에이전트 및 멀티 에이전트

TruLens: Evaluation and Tracking for LLM Experiments and AI Agents

truera/trulens

TruLens is an open-source, OpenTelemetry-native evaluation and tracking library for LLM applications and AI agents, enabling developers to trace every step, score performance with LLM judges, and compare app versions.

공식 저장소 열기 ↗ 프로젝트 홈페이지 ↗
★ 3.5K별점
⑂ 319포크 수
54미해결 이슈
Python언어
MIT라이선스
Q@project.QualityScore편집 점수

프로젝트 스크린샷

TruLens: Evaluation and Tracking for LLM Experiments and AI Agents 스크린샷 TruLens: Evaluation and Tracking for LLM Experiments and AI Agents 스크린샷 TruLens: Evaluation and Tracking for LLM Experiments and AI Agents 스크린샷 TruLens: Evaluation and Tracking for LLM Experiments and AI Agents 스크린샷 TruLens: Evaluation and Tracking for LLM Experiments and AI Agents 스크린샷 TruLens: Evaluation and Tracking for LLM Experiments and AI Agents 스크린샷

개요

TruLens helps developers identify where AI agents fail and where costs can be reduced without sacrificing quality. By utilizing OpenTelemetry-native tracing, it captures structured spans for function calls, LLM generations, retrievals, and tool invocations. This allows traces to be exported to any OTLP-compatible backend. The library provides purpose-built evaluators for agentic systems and supports both inline and batch evaluation modes, allowing teams to evaluate applications as they run or over pre-collected datasets.

주요 기능

  • OpenTelemetry-based tracing for portability to existing observability backends
  • Seven purpose-built agentic evaluators (LogicalConsistency, ExecutionEfficiency, PlanAdherence, PlanQuality, ToolSelection, ToolCalling, ToolQuality)
  • Inline and batch evaluation modes via the Run API
  • Model Context Protocol (MCP) support for instrumenting tool calls
  • Selector API for targeting specific span attributes during evaluation
  • Leaderboard for comparing app versions by scores, latency, and cost

요구 사항, 설치 및 빠른 시작

Install the core package via pip: `pip install trulens`. Specific LLM provider integrations can be installed alongside the core package (e.g., `pip install trulens trulens-providers-openai` for OpenAI, `pip install trulens trulens-providers-litellm` for LiteLLM, `pip install trulens trulens-providers-google` for Google Gemini, `pip install trulens trulens-providers-bedrock` for AWS Bedrock, `pip install trulens trulens-providers-cortex` for Snowflake Cortex, `pip install trulens trulens-providers-huggingface` for HuggingFace, `pip install trulens trulens-providers-langchain` for LangChain models). App framework integrations are also available (e.g., `pip install trulens trulens-apps-langchain` for LangChain/LangGraph, `pip install trulens trulens-apps-llamaindex` for LlamaIndex).

사용 정보

Instrument a class method using the `@instrument` decorator from `trulens.core.otel.instrument`, specifying span types and attributes. For inline evaluation, use `tru_recorder` as a context manager while querying your app. For batch evaluation, configure a `RunConfig` with dataset details, add the run to your app via `tru_app.add_run()`, start the run, and compute metrics such as relevance and groundedness.

모델 호환성 및 사용 사례

Supports LLM providers via dedicated packages: OpenAI/Azure OpenAI, LiteLLM (Anthropic, Cohere, Mistral, etc.), Google Gemini, AWS Bedrock, Snowflake Cortex, HuggingFace, and LangChain models.

라이선스 및 위험 참고 사항

MIT License.

Editorial verification 2026-08-09: repository URL, owner, description, license and repository statistics were reviewed. License metadata: MIT. README was fetched for the channel draft; re-check repository dependencies, releases and model terms before production use.

릴리스 및 유지 관리

Not stated in the repository metadata

Firecrawl

firecrawl/firecrawl

★ 161.1KTypeScript

LangChain

langchain-ai/langchain

★ 143.6KPython

RAGFlow

infiniflow/ragflow

★ 86.7KGo