Top #data-pipeline Tools & Software

pathway

github

Python ETL framework for stream processing, real-time analytics, LLM pipelines, and RAG.

airflow

github

Apache Airflow - A platform to programmatically author, schedule, and monitor workflows

airbyte

github

Open-source data movement for ELT pipelines and AI agents — from APIs, databases & files to warehouses, lakes, and AI applications. Both self-hosted and Cloud.

shardingsphere

github

Empowering Data Intelligence with Distributed SQL for Sharding, Scalability, and Security Across All Databases.

debezium

github

Change data capture for a variety of databases. Please log issues at https://github.com/debezium/dbz/issues.