This week small incremental news but a nice detailed ML article at the end for those that missed it.
Apache Airflow 2.1.0 and official Helm Chart 🎉
Airflow release the first official Helm Chart, with documentation page explaining how to use it for newcomers. If you are interested in a post explaining how to run Airflow on GKE with Helm please drop me a message!
On the other hand huge improvement with this new release, we got a cross-DAG dependency UI and a calendar view in the DAG UI. Here the full changelog.
I write SQL everyday can you lint me?
SQLFluff is becoming the best SQL linter in class, here a large overview of all the superpowers that comes with it. I loves the fact the SQLFluff can lint dbt templates out of the box and the rules references that could be a model for all data documentation.
Microsoft announced Power BI in Jupyter notebooks
Wait, Python is getting faster
Two weeks ago Guido Van Rossum said that the Python Core developpers are aiming to 2x Python performance in the next major release 3.11. I think this is a good news for all data developpers out there!
If you interested in the plan to do it, the detailed roadmap is on Github. In the best scenario we could achieve a 5x uplift.
Secoda announced a dbt integration
Secoda (another Data Discovery tool) announced a dbt Cloud integration to their tool. In this news we can learn two things:
- We have out there a lot of Data Discovery tools and we need maybe a consolidation
- dbt is becoming the main transformation tools and everyone trying to integrate it
ML Friday, Machine Learning at Carrefour: tackling promotional product shortage
1 month ago Carrefour posted an article explaining how the tackle promotional product shortage thanks to ML. They tried to explain how they manage the shortage to avoid impacting the NPS.
Join the newsletter to receive the latest updates in your inbox.