View profile

SF Data Weekly - Tinder's Geosharding, Elasticsearch Migrations, Airflow Orchestration on AWS, ML in the Cloud

Revue
 
 
July 23 · Issue #77 · View online
SF Data Weekly
Our Pick
Rules for Crash Course Data Engineering
The comparative roles of data engineers and other data-related positions.
Data Pipelines
Apache Kafka vs. Enterprise Service Bus (EBS) | Confluent
Streaming maturity model used to identify the current situation in large enterprises.
Zero Downtime Elasticsearch Migrations at intermix.io
Building a flexible system: adding an extra flow in the pipeline during migrations.
Build a Concurrent Data Orchestration Pipeline Using Amazon EMR and Apache Livy | AWS Big Data Blog
High level architecture of the built solution.
Data Storage
Geosharded Recommendations Part 1: Sharding Approach
A query in 100 miles circle will only look up 3 out of 55 geoshards.
Comparing Apache Spark, Storm, Flink and Samza Stream Processing Engines - Part 1
How to Migrate an Application from an On-premises Oracle DB to Amazon RDS for PostgreSQL | AWS Database Blog
Data Analysis
Fundamentals of Data Processing for SciFi geeks — Part I
Scalable Multi-node Deep Learning Training Using GPUs in the AWS Cloud  | AWS Machine Learning Blog
Training a ResNet-50 model takes only 50 minutes to achieve >75% accuracy.
Data Visualization
What is a Senior Data Visualization Engineer?
An interactive chart usually starts as a static visualization in a Jupyter notebook.
Data-driven Products
Smart Cities | AWS for Smart, Connected and Sustainable Cities
What makes a city smart? AWS connects the data sources to solutions.
Data Engineering Jobs
Data Architect - SADA Systems Inc.
If you want to post a job for your company, you can do it here.
Did you enjoy this issue?
If you don't want these updates anymore, please unsubscribe here
If you were forwarded this newsletter and you like it, you can subscribe here
Powered by Revue
650 California St., San Francisco, CA 94108