Top #pipeline Tools & Software

DeepSpeed

github

DeepSpeed is a deep learning optimization library that makes distributed training and inference easy, efficient, and effective.

jenkins

github

Jenkins automation server

prefect

github

Prefect is a workflow orchestration framework for building resilient data pipelines in Python.

airbyte

github

Open-source data movement for ELT pipelines and AI agents — from APIs, databases & files to warehouses, lakes, and AI applications. Both self-hosted and Cloud.

shardingsphere

github

Empowering Data Intelligence with Distributed SQL for Sharding, Scalability, and Security Across All Databases.

marimo

github

A reactive notebook for Python — run reproducible experiments, query with SQL, execute as a script, deploy as an app, and version with git. Stored as pure Python. All in a modern, AI-native editor.

debezium

github

Change data capture for a variety of databases. Please log issues at https://github.com/debezium/dbz/issues.

dalfox

github

🌙🦊 Dalfox is a powerful open-source XSS scanner and utility focused on automation.