Agentes y multiagente

TruLens: Evaluation and Tracking for LLM Experiments and AI Agents

truera/trulens

TruLens is an open-source, OpenTelemetry-native evaluation and tracking library for LLM applications and AI agents, enabling developers to trace every step, score performance with LLM judges, and compare app versions.

Abrir repositorio oficial ↗ Página de inicio del proyecto ↗
★ 3,5KEstrellas
⑂ 319Bifurcaciones
54Problemas abiertos
PythonIdioma
MITLicencia
Q@project.QualityScorePuntuación editorial

Capturas de pantalla del proyecto

Captura de pantalla de TruLens: Evaluation and Tracking for LLM Experiments and AI Agents Captura de pantalla de TruLens: Evaluation and Tracking for LLM Experiments and AI Agents Captura de pantalla de TruLens: Evaluation and Tracking for LLM Experiments and AI Agents Captura de pantalla de TruLens: Evaluation and Tracking for LLM Experiments and AI Agents Captura de pantalla de TruLens: Evaluation and Tracking for LLM Experiments and AI Agents Captura de pantalla de TruLens: Evaluation and Tracking for LLM Experiments and AI Agents

Resumen

TruLens helps developers identify where AI agents fail and where costs can be reduced without sacrificing quality. By utilizing OpenTelemetry-native tracing, it captures structured spans for function calls, LLM generations, retrievals, and tool invocations. This allows traces to be exported to any OTLP-compatible backend. The library provides purpose-built evaluators for agentic systems and supports both inline and batch evaluation modes, allowing teams to evaluate applications as they run or over pre-collected datasets.

Características principales

  • OpenTelemetry-based tracing for portability to existing observability backends
  • Seven purpose-built agentic evaluators (LogicalConsistency, ExecutionEfficiency, PlanAdherence, PlanQuality, ToolSelection, ToolCalling, ToolQuality)
  • Inline and batch evaluation modes via the Run API
  • Model Context Protocol (MCP) support for instrumenting tool calls
  • Selector API for targeting specific span attributes during evaluation
  • Leaderboard for comparing app versions by scores, latency, and cost

Requisitos, instalación y guía rápida

Install the core package via pip: `pip install trulens`. Specific LLM provider integrations can be installed alongside the core package (e.g., `pip install trulens trulens-providers-openai` for OpenAI, `pip install trulens trulens-providers-litellm` for LiteLLM, `pip install trulens trulens-providers-google` for Google Gemini, `pip install trulens trulens-providers-bedrock` for AWS Bedrock, `pip install trulens trulens-providers-cortex` for Snowflake Cortex, `pip install trulens trulens-providers-huggingface` for HuggingFace, `pip install trulens trulens-providers-langchain` for LangChain models). App framework integrations are also available (e.g., `pip install trulens trulens-apps-langchain` for LangChain/LangGraph, `pip install trulens trulens-apps-llamaindex` for LlamaIndex).

Uso

Instrument a class method using the `@instrument` decorator from `trulens.core.otel.instrument`, specifying span types and attributes. For inline evaluation, use `tru_recorder` as a context manager while querying your app. For batch evaluation, configure a `RunConfig` with dataset details, add the run to your app via `tru_app.add_run()`, start the run, and compute metrics such as relevance and groundedness.

Compatibilidad de modelos y casos de uso

Supports LLM providers via dedicated packages: OpenAI/Azure OpenAI, LiteLLM (Anthropic, Cohere, Mistral, etc.), Google Gemini, AWS Bedrock, Snowflake Cortex, HuggingFace, and LangChain models.

Notas sobre la licencia y los riesgos

MIT License.

Editorial verification 2026-08-09: repository URL, owner, description, license and repository statistics were reviewed. License metadata: MIT. README was fetched for the channel draft; re-check repository dependencies, releases and model terms before production use.

Lanzamiento y mantenimiento

Not stated in the repository metadata

Firecrawl

firecrawl/firecrawl

★ 161,1KTypeScript

LangChain

langchain-ai/langchain

★ 143,6KPython

RAGFlow

infiniflow/ragflow

★ 86,7KGo