エージェントとマルチエージェント

TruLens: Evaluation and Tracking for LLM Experiments and AI Agents

truera/trulens

TruLens is an open-source, OpenTelemetry-native evaluation and tracking library for LLM applications and AI agents, enabling developers to trace every step, score performance with LLM judges, and compare app versions.

公式リポジトリを開く ↗ プロジェクトホームページ ↗
★ 3.5Kスター
⑂ 319フォーク数
54未解決の問題
Python言語
MITライセンス
Q@project.QualityScore編集部スコア

プロジェクトのスクリーンショット

TruLens: Evaluation and Tracking for LLM Experiments and AI Agentsのスクリーンショット TruLens: Evaluation and Tracking for LLM Experiments and AI Agentsのスクリーンショット TruLens: Evaluation and Tracking for LLM Experiments and AI Agentsのスクリーンショット TruLens: Evaluation and Tracking for LLM Experiments and AI Agentsのスクリーンショット TruLens: Evaluation and Tracking for LLM Experiments and AI Agentsのスクリーンショット TruLens: Evaluation and Tracking for LLM Experiments and AI Agentsのスクリーンショット

概要

TruLens helps developers identify where AI agents fail and where costs can be reduced without sacrificing quality. By utilizing OpenTelemetry-native tracing, it captures structured spans for function calls, LLM generations, retrievals, and tool invocations. This allows traces to be exported to any OTLP-compatible backend. The library provides purpose-built evaluators for agentic systems and supports both inline and batch evaluation modes, allowing teams to evaluate applications as they run or over pre-collected datasets.

主な機能

  • OpenTelemetry-based tracing for portability to existing observability backends
  • Seven purpose-built agentic evaluators (LogicalConsistency, ExecutionEfficiency, PlanAdherence, PlanQuality, ToolSelection, ToolCalling, ToolQuality)
  • Inline and batch evaluation modes via the Run API
  • Model Context Protocol (MCP) support for instrumenting tool calls
  • Selector API for targeting specific span attributes during evaluation
  • Leaderboard for comparing app versions by scores, latency, and cost

要件、インストール、クイックスタート

Install the core package via pip: `pip install trulens`. Specific LLM provider integrations can be installed alongside the core package (e.g., `pip install trulens trulens-providers-openai` for OpenAI, `pip install trulens trulens-providers-litellm` for LiteLLM, `pip install trulens trulens-providers-google` for Google Gemini, `pip install trulens trulens-providers-bedrock` for AWS Bedrock, `pip install trulens trulens-providers-cortex` for Snowflake Cortex, `pip install trulens trulens-providers-huggingface` for HuggingFace, `pip install trulens trulens-providers-langchain` for LangChain models). App framework integrations are also available (e.g., `pip install trulens trulens-apps-langchain` for LangChain/LangGraph, `pip install trulens trulens-apps-llamaindex` for LlamaIndex).

使用方法

Instrument a class method using the `@instrument` decorator from `trulens.core.otel.instrument`, specifying span types and attributes. For inline evaluation, use `tru_recorder` as a context manager while querying your app. For batch evaluation, configure a `RunConfig` with dataset details, add the run to your app via `tru_app.add_run()`, start the run, and compute metrics such as relevance and groundedness.

モデルの互換性とユースケース

Supports LLM providers via dedicated packages: OpenAI/Azure OpenAI, LiteLLM (Anthropic, Cohere, Mistral, etc.), Google Gemini, AWS Bedrock, Snowflake Cortex, HuggingFace, and LangChain models.

ライセンスとリスクに関する注意事項

MIT License.

Editorial verification 2026-08-09: repository URL, owner, description, license and repository statistics were reviewed. License metadata: MIT. README was fetched for the channel draft; re-check repository dependencies, releases and model terms before production use.

リリースとメンテナンス

Not stated in the repository metadata

Firecrawl

firecrawl/firecrawl

★ 161.1KTypeScript

LangChain

langchain-ai/langchain

★ 143.6KPython

RAGFlow

infiniflow/ragflow

★ 86.7KGo