SF Data Weekly - EDA, EMR and Flyte

EDA is often done with tools like Python Pandas. This piece compares doing EDA with Pandas to using BigQuery SQL. It has a plethora of side by side examples showing the Pandas and BigQuery approach. One of the better EDA articles we’ve seen in a long time.

Using the Redshift Data API instead of a traditional JDBC connection to connect a Jupyter notebook to Redshift.

Rapid data preparation and transformation for ever-evolving and changing data requirements. Secure & compliant data pipelines. Get started today. [Sponsored Content]

Another A vs B post which turns out to be a well-written and compelling post detailing some of the reasons that Snowflake is easier to use than BigQuery.

“Revolution” might be an overstatement, but this article is a good overview of the features of BigQuery Omni, which is Google’s new serverless version of BigQuery.

Amazon EMR is a big data service offered by AWS to run Apache Spark and other open-source applications on AWS. Amazon EMR Notebooks is a managed environment based on Jupyter Notebook. This piece shows how to schedule and run EMR notebooks using the AWS CLI and chaining notebooks using CloudWatch Events.

Everybody needs to push out an Excel spreadsheet for further analysis, or for a colleague, once in a while. This piece show to output a Pandas DataFrame to Excel with table formatting using XlsxWriter and Python.

For a tool with a big market share, Power BI doesn’t have big mind share. This piece by a Power BI user lists a number of compelling features that she misses while using Tableau.

Amazon QuickSight is a cloud-scale business intelligence (BI) service. One of its enterprise/real-world features is the column- and row-level security, and this piece shows how that’s done.

With the EOL of Python 2, Slack wanted to upgrade their Airflow workflows to Python 3. This article shows how they did it while maintaining transparency to users, and increasing reliability.

Flyte is Lyft’s open source, Python-based orchestration platform for data and machine learning workflows. This piece looks at Flyte Admin, the control plane and gateway to Flyte.