SF Data Weekly

A weekly email of useful links for people interested in building data platforms, curated by the folks at https://www.intermix.io

submissions@sfdata.io

Previous issues

SF Data Weekly - Amazon Payments Analytics, Hive-Kafka Integration, Airbnb Case Study in AWS, Serverless Apps

#99 Jan 14, 2019

SF Data Weekly - Most Clicked Posts of 2018

#98 Jan 2, 2019

SF Data Weekly - Data Skills Survey 2018, Scaling Spark at Airbnb, AWS for IoT, Cassandra vs. TimescaleDB

#97 Dec 24, 2018

#96

Dec 11, 2018

SF Data Weekly - Amazon Redshift at Udemy, Kafka at Pinterest, The Telegraph Data Stack, Quant Research at Facebook
#95

Dec 5, 2018

SF Data Weekly - Facebook's Data @ Scale, Scaling Redshift, Airflow & Luigi Workflows, SQL at Scale, Tableau & plotly.js
#94

Nov 19, 2018

SF Data Weekly - Druid at Airbnb, Netflix's Time Series Storage, Azure Monitor, Automated Testing
#93

Nov 12, 2018

SF Data Weekly - Airbnb Search, Netflix's Recommenders, Kafka & Datalog, Spark's Core Concepts
#92

Nov 8, 2018

SF Data Weekly - Going Serverless, Firebase and MongoDB, Spark Introduction, Azure Analytics
#91

Oct 30, 2018

SF Data Weekly - Airflow Pipelines, Scaling DBs, Azure Analytics, SQL at Scale with Spark
#90

Oct 22, 2018

SF Data Weekly - Data at Uber, Airflow 101, Event Driven Systems & Kafka
#89

Oct 16, 2018

SF Data Weekly - Spark at Facebook, Azure Pipelines, Ebay Data Governance, AWS Deep Learning
#88

Oct 10, 2018

SF Data Weekly - Facebook's Messenger, Blockchain for Storage, Python Pipelines, VPC Network Analysis
#87

Oct 1, 2018

SF Data Weekly - MySQL vs MongoDB, DynamoDB Accelerator, Rankings in DB and Deep Learning Tools, Periscope Data Analytics
#86

Sep 24, 2018

SF Data Weekly - Big Data & AI Landscape, Data Stack at Unsplash, Confluent's Thoughts on ETL, Redshift Cost, Behind Uber Eats
#85

Sep 18, 2018

SF Data Weekly - Airbnb's SpinalTap, Pinterest's Goku DB, Stitch Fix Data Stack, TwoSigma's Flint
#84

Sep 10, 2018

SF Data Weekly - IOPipe's Data Pipeline, 14 Redshift Performance Tips, Spotify and Fitbit Data Analysis, ML at Teads, SimilarWeb's Data Pipeline
#83

Sep 4, 2018

SF Data Weekly - ETL vs ELT, Warehouse vs Data Lake, Kafka Batch to Real-time, Airflow, Databot Pipes
#82

Aug 27, 2018

SF Data Weekly - Mozilla's Data Pipeline, Deep Learning at Twitter, Google Data Collection
#81

Aug 20, 2018

SF Data Weekly - Yelp's Data Stack, Monzo's Data Team, Dropbox's SMR Drives, Netflix's Notebooks, Redshift Vacuum
#80

Aug 13, 2018

SF Data Weekly - Attribution Pipelines, Idempotence, Drake's Lyrics (!), AWS DB Migrations, Reports with Tableau
#79

Aug 7, 2018

SF Data Weekly - Data Engineering at Unruly, Airflow at Plaid, Python Stream Processing, Hadoop Migrations
#78

Jul 30, 2018

SF Data Weekly - Hotels.com's Circus Train, High Performance ETL, Spark on YARN, Gradient Boosting Libraries
#77

Jul 23, 2018

SF Data Weekly - Tinder's Geosharding, Elasticsearch Migrations, Airflow Orchestration on AWS, ML in the Cloud
#76

Jul 16, 2018

SF Data Weekly - ML Workflows at Twitter, Data Pipelines at Blinkist & WalmartLabs, Redshift as OLTP, Werner Vogels on Databases
#75

Jul 9, 2018

SF Data Weekly - Airflow Concepts, 4 WLM Steps in Amazon Redshift, Gaming Events Pipeline, ML Models on Structured Streaming, 10 Python Myths
#74

Jul 1, 2018

SF Data Weekly - Uber's H3, AirBnB's Guide to Data Eng, Why ML Projects Fail, Kafka & Spark Streams
#73

Jun 25, 2018

SF Data Weekly - Netflix's Metacat, Airflow Testing, Decision Making at LinkedIn, What is TensorFlow?
#72

Jun 18, 2018

SF Data Weekly - TensorFlow at Twitter, Mixpanel's Customer Cost, ETL with AWS Lambda, Stream Processing
#71

Jun 11, 2018

SF Data Weekly - Uber's Query Engine, Yelp's Move to AWS, Monitoring Data Lakes, MongoDB Change Streams, Kubernetes DBs, Distributed Logging
#70

Jun 4, 2018

SF Data Weekly - Redshift Spectrum & Data Lakes, Spark ETLs, Alibaba Cloud, Kafka for Developers
#69

May 29, 2018

SF Data Weekly - Eventbrite's Data Pipeline, Learn Kubernetes, Recommender Systems in GCP, Interactive Visualizations
#68

May 21, 2018

SF Data Weekly - Disney's Data Pipeline, Wish Analytics Team, Blockchain vs DBs, DynamoDB Encryptions
#67

May 14, 2018

SF Data Weekly - Airflow Key Concepts, Kafka Internals, Analytics with PySpark, AWS Pipelines
#66

May 7, 2018

SF Data Weekly - Netflix High Availability, Spark vs. Hadoop, Kafka Adoption Journey
#65

Apr 30, 2018

SF Data Weekly - PayPal's Gimel, Redshift Short Query Acceleration, AWS Cost Optimization, Postgres for IoT
#64

Apr 23, 2018

SF Data Weekly - Data Engineers vs Data Scientists, Wayfair's Data Pipeline, Serverless Systems
#63

Apr 16, 2018

SF Data Weekly - Uber's HDFS, Serverless Architectures, Analytics Pipelines, Redshift & Kafka Diagnostics
#62

Apr 9, 2018

SF Data Weekly - GO-JEK's data infrastructure, Spark vs. Redshift, ETL vs ELT, KSQL – 20% discount code from our friends at DataEngConf DataEngConf is a 2-day conference that bridges the
#61

Apr 2, 2018

SF Data Weekly - Mozilla's Data Pipeline, DB Migrations to Kafka, NLP in Spark, Cloud Warehouses
#60

Mar 26, 2018

SF Data Weekly - Data Engineering Report, MongoDB & Redshift, SQL vs NoSQL, TensorFlow
#59

Mar 19, 2018

SF Data Weekly - Wish's Data Infrastructure, Spark Stream-Stream Joins, Kafka Event Sourcing, Python for Analytics
#58

Mar 12, 2018

SF Data Weekly - Apache Arrow, Graph Data with Mongo, SageMaker – New section! Scroll down to the bottom! Each week, we'll send you the 3 best data engineering jobs we
#57

Mar 5, 2018

SF Data Weekly - Uber's Queryparser, Flink at Netflix, Kafka Streams, ETL, Redshift and Pytorch
#56

Feb 26, 2018

SF Data Weekly - Data Engineering Guidelines II, Redshift Performance at Remind, Airflow Pipelines, JupyterLab
#55

Feb 19, 2018

SF Data Weekly - Data Engineers in 2018, KSQL + Kafka Tutorial, DB Migrations in AWS, Airflow
#54

Feb 12, 2018

SF Data Weekly - AirBnB's BinaryAlert, Akka Streams, KSQL, Big Query Weather Data
#53

Feb 5, 2018

SF Data Weekly - LinkedIn’s Data Center Evolution, Scalable Streams, NoSQL Survey, Bitcoin Analysis, Tableau
#52

Jan 29, 2018

SF Data Weekly - LinkedIn's Gobblin, Redshift ETL, Microservice Madness, Data Science at Twitter, Uber Maps
#51

Jan 22, 2018

SF Data Weekly - Airbnb's experimentation platform, Kafka Streams, PostgreSQL and Redshift Spectrum Tutorials
#50

Jan 15, 2018

SF Data Weekly - Coursera's Data Infrastructure, Postgres Internals, AWS Data Lakes, Tableau
#49

Jan 8, 2018

SF Data Weekly - Data Pipeline Marketplace, Redshift Guides, BinaryAlert, NLP for Spark
#48

Jan 3, 2018

SF Data Weekly - LinkedIn Salary, Netflix OCA, Data Products, Redshift Migration, Deep Learning with AWS
#47

Dec 26, 2017

SF Data Weekly - Kafka & Redis Streams, Marketing Data Pipelines, AWS Services in Production, AWS Startups in 2017
#46

Dec 18, 2017

SF Data Weekly - Apache Kafka Monitoring, Scala + Spark, Building an AWS Data Lake, ML NASA
#45

Dec 11, 2017

SF Data Weekly - Spark & Redshift at Dollar Shave Club, Dynamic ETL, Automation in Amazon RDS, Apache Arrow
#44

Dec 4, 2017

SF Data Weekly - LinkedIn's Dali, Data Team Structure, AWS New AI Services
#43

Nov 27, 2017

SF Data Weekly - LinkedIn's Brooklin, Pinterest's Data Infra Revisited, Real-time Data
#42

Nov 20, 2017

SF Data Weekly - Kafka Transactions, ETL with Python, Redshift Warehouses, DataWorks Summit
#41

Nov 13, 2017

SF Data Weekly - Microservices in Kafka, Airflow Tutorial, Google Colaboratory, Spark Summit 2017
#40

Nov 6, 2017

SF Data Weekly - Kafka 1.0, Spark's Future, Pipelines with Airflow, AWS Lambda
#39

Oct 30, 2017

SF Data Weekly - Kafka in Real-Time, Spark is here to Stay, Teradata & Redshift, Data Science Pipelines
#38

Oct 23, 2017

SF Data Weekly - Spotify's AI, Robinhood's Airflow, Amazon Redshift Training, Looker vs. Oracle
#37

Oct 16, 2017

SF Data Weekly - Tracking with Kafka, Benchmarking Redshift, DB Optimization, R-Brain and 500 Data Structures
#36

Oct 9, 2017

SF Data Weekly - Pinterest's Kafka, Yahoo's Vespa, Cloud Storage Growth and Real-Time Analytics
#35

Oct 2, 2017

SF Data Weekly - Facebook's MyRocks, Amazon Athena, Spark Tutorial, D3 Animations – Join us at our next SF Data event this week on Wednesday, October 4th and use the code "WEEKLY" for 5
#34

Sep 25, 2017

SF Data Weekly - Joins in Kafka, More Kafka, MongoDB Goes Public, Pinterest's Test Graph and Microsoft's Power BI platform
#33

Sep 18, 2017

SF Data Weekly - Migrating to Postgres, Apache Ignite, Citymapper Data, Tinder and Divorce Analytics
#32

Sep 11, 2017

SF Data Weekly - Uber's Michelangelo, NYT's Kafka, Intro to Cassandra, Python's popularity
#31

Sep 5, 2017

SF Data Weekly - LinkedIn's Kafka Cruise Control, Facebook's LogDevice, Tesla's battery degradation, Marketing MongoDB
#30

Aug 28, 2017

SF Data Weekly - Pinterest's DoctorKafka, Warehousing w/ Amazon Redshift, GPU DBs, Neural Nets and Analyzing Cryptocurrencies
#29

Aug 21, 2017

SF Data Weekly - The New AWS Glue, Postgres Atomic, DB Replication and Big Data Analytics
#28

Aug 14, 2017

SF Data Weekly - MySQL to Redshift, SMACK Stack, Timescale DB, 3 Stages of Data Science
#27

Aug 8, 2017

SF Data Weekly - Visualizing Shazam, Data Eng Crash Course, Better SQL,
#26

Jul 24, 2017

SF Data Weekly - Spark vs. Redshift, Data Products at AirBnB, Trends in Cloud Computing
#25

Jul 17, 2017

SF Data Weekly - GitHub's Orchestrator, Segment's Commute Time, Kafka at MyHeritage
#24

Jun 26, 2017

SF Data Weekly - Yahoo Bullet, Redshift Spectrum Best Practices, Data Science at Stack Overflow
#23

Jun 14, 2017

SF Data Weekly - Pinterest's Soundwave, Redshift Spectrum, AWS Bandwidth Costs
#22

Jun 5, 2017

SF Data Weekly - Kafka at Reddit, How to Call BS on Big Data, Redshift Performance
#21

May 27, 2017

SF Data Weekly - Airbnb's Data University, Analytics at the NYT, Expedia's Cloud Spend
#20

May 22, 2017

SF Data Weekly - AirBnB's Data Portal, Pinterest's Data Pipelines, Segment's Cost Analytics
#19

May 15, 2017

SF Data Weekly - Kafka at Cloudflare, Typeform's Data Team, DataOps
#18

May 8, 2017

SF Data Weekly - AirBnB's StreamAlert, Kafka at LinkedIn, Redshift at Myntra, Instacart Data
#17

May 1, 2017

SF Data Weekly - Spark at Pinterest, Netflix Data Platform, Productive Data Engineering – This past week Amazon announced earnings. AWS sales increased 42 percent to $3.66 billion. That's a $
#16

Apr 24, 2017

SF Data Weekly - Slack's Infrastructure on AWS, Redshift Spectrum, Spark & Redshift – This past week the AWS Summit 2017 took place in San Francisco . Watch Werner Vogel's keynote. Liste
#15

Apr 18, 2017

SF Data Weekly - Uber's deck.gl, Segment's DynamoDB, Feedvisor's Redshift – This time on a Tuesday vs. the usual Monday - a longer than usual Easter weekend. SQL of all flavors
#14

Apr 10, 2017

SF Data Weekly - Gusto's Data Stack, Microservices & Data, 519 Dashboard UIs – Early bird pricing for DataEngConf in SF from April 26-28 is over. But we got you covered. Use the sp
#13

Apr 3, 2017

SF Data Weekly - Turo's Pricing Engine, FB's Prophet, Real-time Queries with Kafka – Spring has arrived in SF. 50 years ago, in 1967, the Summer of Love started and 100,000 hippies desce
#12

Mar 20, 2017

SF Data Weekly - Uber's Hoodie, DBA of the Future, 50 companies leading AI revolution – This week's issue is heavy on data visualization: color palettes, custom data viz and data-driven int
#11

Mar 13, 2017

SF Data Weekly - LinkedIn's Venice, Lambda Functions, Interactive Queries in Kafka – "Big Data" is going more and more vertical. See MapR's 5 big data trends in healthcare for 2017. Same
#10

Feb 27, 2017

SF Data Weekly - Oscars; Facebook's "Prophet", Google's Spanner, Data Scientist vs. Data Engineer, – Last night's Oscars were predictable until the very end. Nate Silver from FiveThirtyEight only got 7
#9

Feb 13, 2017

SF Data Weekly - Private Event; AirBnB's Superset, Search at Slack, Rise of the Data Engineer – Join us at our next private event "Data Platforms in FinTech", on Feb 23rd at Shasta Ventures in SF!
#8

Feb 5, 2017

SF Data Weekly - IoT pipelines, distributed tracing at Uber, Pinterest mySQL compression, deep learning with little data – Here's a great way to start the week: Create a poster of your git history with Commit Print, with cus
#7

Jan 31, 2017

SF Data Weekly - Issue #7: FB's Scuba; Data Gold Rush; Explaining Data Engineering – It seems like Monday morning is the flavor-du-jour for sending weekly emails. So this time around, SF
#6

Jan 23, 2017

SF Data Weekly - "Data just happens"; DevOps & Databases; Uber's Data-driven Maps; Best 2016 Data Viz – AI & Machine Learning are the sliced bread and peanut butter of 2017, but only SF Data readers un
#5

Jan 9, 2017

SF Data Weekly - Uber's Movement Data Platform – Happy Monday! Uber just launched their new product "Movement", a website that offers access to Uber's
#4

Jan 2, 2017

SF Data Weekly - 2017 Icons, Maps, Colors! – It's 2017, start the year by impressing your colleagues with crazy data architecture skillz with thes
#3

Dec 19, 2016

SF Data Weekly - Larry Ellison is Wrong About Amazon Redshift" – Last issue for this year - enjoy your data conversations!We just published the videos from our recent
#2

Dec 12, 2016

SF Data Weekly - This AI Boom Will Also Bust
#1

Dec 5, 2016

SF Data Weekly - First Issue! – Hi there - our very first issue of SF Data Weekly, an email with useful links for people interested i