ToolAI.io · GitHub Channel

Data Engineering & Pipelines · Open-source AI Projects on GitHub

A fact-based index of LLM, agent, MCP, RAG and AI development projects, with license, setup, download, screenshots and related resources for each entry.

Public repository data

Project index

5 projects
Data Engineering & Pipelines

aistore

NVIDIA/aistore

AIStore: scalable storage for AI applications

★ 1.9K⑂ 277Go
MITQ94
Data Engineering & Pipelines

langextract

google/langextract

LLM-based structured extraction with source-text alignment and visualization for reviewable document processing.

★ 38.6K⑂ 2.7KPython
Apache-2.0Q90
Data Engineering & Pipelines

spaCy

explosion/spaCy

An engineering-oriented NLP pipeline library for tokenization, entities, syntax, and classification.

★ 33.9K⑂ 4.7KPython
MITQ90
Data Engineering & Pipelines

OpenBB

OpenBB-finance/OpenBB

A financial data platform for research, analysis and AI applications.

★ 72.7K⑂ 7.5KPython
License not detectedQ85
Data Engineering & Pipelines

airflow

apache/airflow

A platform for defining and scheduling workflows, including data and machine learning pipelines.

★ 46.7K⑂ 17.8KPython
Apache-2.0Q85

Recently updated

Craft Anythingstorytold/craft-anything★ 27.4K elephant-goldfish-modelgoogle/elephant-goldfish-model★ 15.6K DeepInferdeepseek-ai/DeepInfer★ 25.7K ComfyUIComfy-Org/ComfyUI★ 136.7K CrewAICrewAIInc/crewAI★ 59.5K Microsoft Agent Frameworkmicrosoft/agent-framework★ 14K

Most starred

ECCaffaan-m/ECC★ 248.8K Hermes Agentnousresearch/hermes-agent★ 227.1K tensorflowtensorflow/tensorflow★ 198.8K AutoGPTSignificant-Gravitas/AutoGPT★ 187.1K ollamaollama/ollama★ 180.2K markitdown — Document conversion and extractionmicrosoft/markitdown★ 174.9K