Top #data-engineering Tools & Software

Made-With-ML

github

Learn how to develop, deploy and iterate on production-grade ML applications.

airflow

github

Apache Airflow - A platform to programmatically author, schedule, and monitor workflows

applied-ml

github

📚 Papers & tech blogs by companies sharing their work on data science & machine learning in production.

prefect

github

Prefect is a workflow orchestration framework for building resilient data pipelines in Python.

airbyte

github

Open-source data movement for ELT pipelines and AI agents — from APIs, databases & files to warehouses, lakes, and AI applications. Both self-hosted and Cloud.

taipy

github

Turns Data and AI algorithms into production-ready web applications in no time.

xonsh

github

🐚 Python-powered shell. Full-featured, cross-platform and AI-friendly.