Data Engineering & Pipelines

airflow

apache/airflow

A platform for defining and scheduling workflows, including data and machine learning pipelines.

★ 46.7KStars
⑂ 17.8KForks
2151Open issues
PythonLanguage
Apache-2.0License
Q85Editorial score

Overview

A platform for defining and scheduling workflows, including data and machine learning pipelines. The repository is maintained under apache on GitHub. Its primary language is Python. See the [project README](https://github.com/apache/airflow#readme) for the supported workflows.

Key features

  • A platform for defining and scheduling workflows, including data and machine learning pipelines.

Requirements, installation and quick start

Follow the official installation guide and configure the data sources or runtime required by the example you select.

[Read the upstream installation and quickstart instructions](https://github.com/apache/airflow#readme).

Usage

Begin with a small data workflow, validate source access and outputs, then expand the schedule or processing steps as needed.

[Usage examples and configuration reference](https://github.com/apache/airflow#readme).

Model compatibility and use cases

Model compatibility is not stated in the repository metadata.

License and risk notes

GitHub reports Apache-2.0 for this repository. Review the upstream license file; model weights, datasets and dependencies may have separate terms.

Source review 2026-09-05: GitHub search metadata and repository README. No runtime benchmark performed. License metadata: Apache-2.0.

Release and maintenance

[View upstream releases](https://github.com/apache/airflow/releases).

aistore

NVIDIA/aistore

★ 1.9KGo

OpenBB

OpenBB-finance/OpenBB

★ 72.7KPython