Agentes e multiagente

TruLens: Evaluation and Tracking for LLM Experiments and AI Agents

truera/trulens

TruLens is an open-source, OpenTelemetry-native evaluation and tracking library for LLM applications and AI agents, enabling developers to trace every step, score performance with LLM judges, and compare app versions.

Abrir repositório oficial ↗ Página inicial do projeto ↗
★ 3,5KEstrelas
⑂ 319Forks
54Problemas em aberto
PythonIdioma
MITLicença
Q@project.QualityScorePontuação editorial

Capturas de tela do projeto

Captura de tela de TruLens: Evaluation and Tracking for LLM Experiments and AI Agents Captura de tela de TruLens: Evaluation and Tracking for LLM Experiments and AI Agents Captura de tela de TruLens: Evaluation and Tracking for LLM Experiments and AI Agents Captura de tela de TruLens: Evaluation and Tracking for LLM Experiments and AI Agents Captura de tela de TruLens: Evaluation and Tracking for LLM Experiments and AI Agents Captura de tela de TruLens: Evaluation and Tracking for LLM Experiments and AI Agents

Visão geral

TruLens helps developers identify where AI agents fail and where costs can be reduced without sacrificing quality. By utilizing OpenTelemetry-native tracing, it captures structured spans for function calls, LLM generations, retrievals, and tool invocations. This allows traces to be exported to any OTLP-compatible backend. The library provides purpose-built evaluators for agentic systems and supports both inline and batch evaluation modes, allowing teams to evaluate applications as they run or over pre-collected datasets.

Principais recursos

  • OpenTelemetry-based tracing for portability to existing observability backends
  • Seven purpose-built agentic evaluators (LogicalConsistency, ExecutionEfficiency, PlanAdherence, PlanQuality, ToolSelection, ToolCalling, ToolQuality)
  • Inline and batch evaluation modes via the Run API
  • Model Context Protocol (MCP) support for instrumenting tool calls
  • Selector API for targeting specific span attributes during evaluation
  • Leaderboard for comparing app versions by scores, latency, and cost

Requisitos, instalação e início rápido

Install the core package via pip: `pip install trulens`. Specific LLM provider integrations can be installed alongside the core package (e.g., `pip install trulens trulens-providers-openai` for OpenAI, `pip install trulens trulens-providers-litellm` for LiteLLM, `pip install trulens trulens-providers-google` for Google Gemini, `pip install trulens trulens-providers-bedrock` for AWS Bedrock, `pip install trulens trulens-providers-cortex` for Snowflake Cortex, `pip install trulens trulens-providers-huggingface` for HuggingFace, `pip install trulens trulens-providers-langchain` for LangChain models). App framework integrations are also available (e.g., `pip install trulens trulens-apps-langchain` for LangChain/LangGraph, `pip install trulens trulens-apps-llamaindex` for LlamaIndex).

Uso

Instrument a class method using the `@instrument` decorator from `trulens.core.otel.instrument`, specifying span types and attributes. For inline evaluation, use `tru_recorder` as a context manager while querying your app. For batch evaluation, configure a `RunConfig` with dataset details, add the run to your app via `tru_app.add_run()`, start the run, and compute metrics such as relevance and groundedness.

Compatibilidade do modelo e casos de uso

Supports LLM providers via dedicated packages: OpenAI/Azure OpenAI, LiteLLM (Anthropic, Cohere, Mistral, etc.), Google Gemini, AWS Bedrock, Snowflake Cortex, HuggingFace, and LangChain models.

Observações sobre licença e riscos

MIT License.

Editorial verification 2026-08-09: repository URL, owner, description, license and repository statistics were reviewed. License metadata: MIT. README was fetched for the channel draft; re-check repository dependencies, releases and model terms before production use.

Lançamento e manutenção

Not stated in the repository metadata

Firecrawl

firecrawl/firecrawl

★ 161,1KTypeScript

LangChain

langchain-ai/langchain

★ 143,6KPython

RAGFlow

infiniflow/ragflow

★ 86,7KGo