View profile

SF Data Weekly - Uber's Query Engine, Yelp's Move to AWS, Monitoring Data Lakes, MongoDB Change Streams, Kubernetes DBs, Distributed Logging

June 11 · Issue #71 · View online
SF Data Weekly
Our Pick
How Yelp (Mostly) Shut Down Its Own Data Centers and Moved to AWS
Data Pipelines
Comparing Apache Kafka, Amazon Kinesis, Microsoft Event Hubs and Google Pub/Sub
Data pipelines are complex and changing often. Regressions are possible from minor changes to DAGs and tasks, with unintended impacts on tables and data flows which may not be discovered until much later and data has been already used in reports. This training gives practical, real-world examples of tests that can be added to any data pipeline.
How do you know if the data in your data lake is correct?
Kubernetes NodePort vs LoadBalancer vs Ingress? When Should I Use What?
Ingress is a suitable method to expose multiple services with a single load balancer.
Some Things You Should Know Before Using Amazon’s Elasticsearch Service on AWS
Data Storage
Using MongoDB as a Realtime Database With Change Streams
A diagram showing how change streams pushed by MongoDB are handled.
Databases on Kubernetes - How to Recover from Failures, Scale Up and Down in a Few Line Commands
Data Analysis
Breaking Away: The Secrets to Scaling Analytics
Breakaway companies set a sound strategy for scaling analytics.
Kafka-Connect-Xenon: Streaming Live with Kafka Connect!
Data flow and Xenon Architecture.
Data Visualization
Visualizing Google Analytics Campaign Data in Tableau
A view of Click-Through Rate (CTR) - bars and Cost per Click (CPC) - color intensity.
Data-driven Products
Food Discovery with Uber Eats: Building a Query Understanding Engine
A high-level view of a graph data pipeline using multiple data sources.
Data Engineering Jobs
Data Engineer - Splunk
Senior Data Engineer - Data Platform
If you want to post a job for your company, you can do it here.
Did you enjoy this issue?
If you don't want these updates anymore, please unsubscribe here
If you were forwarded this newsletter and you like it, you can subscribe here
Powered by Revue
650 California St., San Francisco, CA 94108