Tác nhân & Đa tác nhân

TruLens: Evaluation and Tracking for LLM Experiments and AI Agents

truera/trulens

TruLens is an open-source, OpenTelemetry-native evaluation and tracking library for LLM applications and AI agents, enabling developers to trace every step, score performance with LLM judges, and compare app versions.

Mở kho lưu trữ chính thức ↗ Trang chủ dự án ↗
★ 3,5KSố sao
⑂ 319Fork
54Vấn đề đang mở
PythonNgôn ngữ
MITGiấy phép
Q@project.QualityScoreĐiểm biên tập

Ảnh chụp màn hình dự án

Ảnh chụp màn hình của TruLens: Evaluation and Tracking for LLM Experiments and AI Agents Ảnh chụp màn hình của TruLens: Evaluation and Tracking for LLM Experiments and AI Agents Ảnh chụp màn hình của TruLens: Evaluation and Tracking for LLM Experiments and AI Agents Ảnh chụp màn hình của TruLens: Evaluation and Tracking for LLM Experiments and AI Agents Ảnh chụp màn hình của TruLens: Evaluation and Tracking for LLM Experiments and AI Agents Ảnh chụp màn hình của TruLens: Evaluation and Tracking for LLM Experiments and AI Agents

Tổng quan

TruLens helps developers identify where AI agents fail and where costs can be reduced without sacrificing quality. By utilizing OpenTelemetry-native tracing, it captures structured spans for function calls, LLM generations, retrievals, and tool invocations. This allows traces to be exported to any OTLP-compatible backend. The library provides purpose-built evaluators for agentic systems and supports both inline and batch evaluation modes, allowing teams to evaluate applications as they run or over pre-collected datasets.

Tính năng chính

  • OpenTelemetry-based tracing for portability to existing observability backends
  • Seven purpose-built agentic evaluators (LogicalConsistency, ExecutionEfficiency, PlanAdherence, PlanQuality, ToolSelection, ToolCalling, ToolQuality)
  • Inline and batch evaluation modes via the Run API
  • Model Context Protocol (MCP) support for instrumenting tool calls
  • Selector API for targeting specific span attributes during evaluation
  • Leaderboard for comparing app versions by scores, latency, and cost

Yêu cầu, cài đặt và bắt đầu nhanh

Install the core package via pip: `pip install trulens`. Specific LLM provider integrations can be installed alongside the core package (e.g., `pip install trulens trulens-providers-openai` for OpenAI, `pip install trulens trulens-providers-litellm` for LiteLLM, `pip install trulens trulens-providers-google` for Google Gemini, `pip install trulens trulens-providers-bedrock` for AWS Bedrock, `pip install trulens trulens-providers-cortex` for Snowflake Cortex, `pip install trulens trulens-providers-huggingface` for HuggingFace, `pip install trulens trulens-providers-langchain` for LangChain models). App framework integrations are also available (e.g., `pip install trulens trulens-apps-langchain` for LangChain/LangGraph, `pip install trulens trulens-apps-llamaindex` for LlamaIndex).

Cách sử dụng

Instrument a class method using the `@instrument` decorator from `trulens.core.otel.instrument`, specifying span types and attributes. For inline evaluation, use `tru_recorder` as a context manager while querying your app. For batch evaluation, configure a `RunConfig` with dataset details, add the run to your app via `tru_app.add_run()`, start the run, and compute metrics such as relevance and groundedness.

Khả năng tương thích của mô hình và trường hợp sử dụng

Supports LLM providers via dedicated packages: OpenAI/Azure OpenAI, LiteLLM (Anthropic, Cohere, Mistral, etc.), Google Gemini, AWS Bedrock, Snowflake Cortex, HuggingFace, and LangChain models.

Ghi chú về giấy phép và rủi ro

MIT License.

Editorial verification 2026-08-09: repository URL, owner, description, license and repository statistics were reviewed. License metadata: MIT. README was fetched for the channel draft; re-check repository dependencies, releases and model terms before production use.

Phát hành và bảo trì

Not stated in the repository metadata

Firecrawl

firecrawl/firecrawl

★ 161,1KTypeScript

LangChain

langchain-ai/langchain

★ 143,6KPython

RAGFlow

infiniflow/ragflow

★ 86,7KGo