SF Data Weekly - Data at Dailymotion, Amazon Payments with Redshift, Streaming Microservices, 5 Warehouse Tips

Dailymotion has been analyzing their data journey that focuses on how data engineers work with data scientists and data analysts to improve production release. The ongoing challenge is to find the right balance between catering to each specific need and being in a generic customer/supplier relationship.

The Amazon Payments Data Engineering team is responsible for data ingestion, transformation, and the computation and storage of data. They recently switched to Amazon Redshift as a payments data warehouse. This post shows their selection process and how they use Redshift.

We’re hearing more and more about Snowflake and BigQuery. Meanwhile, Amazon Redshift is still the #1 cloud warehouse by marketshare.

To help assess where our customers are on their journey to being data-driven we created the Amazon Redshift Maturity Survey and today we’re opening it up for you to take.

The most challenging goal of any application architecture is simplicity, but it is possible to achieve. Neil Avery explores four pillars for enabling scalable development that works across the event-driven enterprise. These pillars minimize `complexity and provide foundational rules for building systems using composition.

When Daniel started his job position in a UK government institution, he noticed there’s huge amount of unused data, stored in text files. Eager to learn more about data engineering, he created a streaming data pipeline and also of found ways of making data more accessible. This is his work using Apache Beam and Dataflow.

In this article, learn exactly what a data warehouse is and most importantly, how to select the right data warehouse for your company. It also shares a few thoughts on modern data warehousing, different architectures, use cases and vendors.

Tilting Point is a new-generation games partner that provides top development studios with expert resources, services, and operational support. Their data engineering team is running daily / hourly batch jobs for reporting on game analytics. This post shows how they reached a near real-time reporting in 5-10 min intervals.

SageMaker is a tool on the Amazon Cloud for developing and deploying ML models. It promises to ease the process of training and deploying models to production at scale providing three very useful functionalities.

Microsoft recently simplified the way machine learning is done in the Azure cloud. This is an intro to recommender systems, an area where Microsoft’s contribution is amongst the most important open-sourced ones.

Tableau invests in advanced analytics (techniques and tools typically beyond those of traditional business intelligence) so that you can get to the root of your questions, no matter how complex they become. This article will focus on Tableau’s approach to predictive analytics, segmentation, what-if analysis, and more.

Blueprint is a newly open-sourced design system by Palantir, implemented as a collection of composable React components and optimized for desktop applications. It has changed the way Palantir scales design and frontend engineering by providing a systematic way to apply unified and consistent design across a diverse set of teams.