Agentes e multiagente

Opik: Open-Source LLM Observability, Evaluation and Agent Tracing

comet-ml/opik

Opik is an Apache-2.0-licensed platform for tracing, evaluating, debugging and monitoring LLM applications, RAG systems and multi-step agent workflows. It supports self-hosting, a hosted Comet.com option, client SDKs, a REST API, automated evaluations, prompt experimentation and production dashboards.

★ 21,1KEstrelas
⑂ 1,7KForks
163Problemas em aberto
PythonIdioma
Apache-2.0Licença
Q@project.QualityScorePontuação editorial

Visão geral

The repository contains the Opik server backend, web application and components for tracing, datasets, experiments, evaluations, prompt management, online evaluation and agent optimization. It is framework-agnostic and can capture LLM calls, conversations, retrieval steps, tool executions and nested agent activity as trace trees. Evaluation capabilities include code-based metrics, datasets, experiments and LLM-as-a-judge metrics for hallucination detection, moderation and RAG assessment.

Principais recursos

  • Nested tracing for LLM calls, conversations, retrieval steps, tool executions and multi-step agents
  • Datasets, experiments, code-based metrics and LLM-as-a-judge evaluation
  • RAG evaluation metrics including answer relevance and context precision
  • Hallucination detection and moderation evaluation
  • Prompt management and Prompt Playground
  • Production dashboards and online evaluation rules
  • Feedback scores for traces and spans through the Python SDK or UI
  • PyTest integration for CI/CD evaluation
  • Agent Optimizer and guardrails capabilities
  • Python, TypeScript and Ruby-via-OpenTelemetry client support plus a REST API
  • Hosted and self-hosted deployment options
  • Docker Compose scripts and Kubernetes/Helm deployment guidance

Requisitos, instalação e início rápido

Python SDK quick start: `pip install opik` or `uv pip install opik`, followed by `opik configure`. For local self-hosting, clone `https://github.com/comet-ml/opik.git`, enter the `opik` directory and run `./opik.sh` on Linux or macOS. On Windows, run `powershell -ExecutionPolicy ByPass -c ".\\opik.ps1"`. The scripts also expose `--infra`, `--backend` and `--guardrails` profiles. Kubernetes installation through Helm is documented for larger-scale deployments. A hosted Comet.com option is also available.

Uso

After configuring the Python SDK, import `track` from `opik` and decorate a function with `@track`. Calls to that function, including nested calls, are then logged as traces. For a local installation, the README states that the web interface is available at `http://localhost:5173` after startup. Applications can alternatively log through supported integrations, OpenTelemetry, other client SDKs or the REST API.

Compatibilidade do modelo e casos de uso

Opik is described as framework-agnostic rather than tied to one model. The supplied README excerpt explicitly lists integrations involving Anthropic, Amazon Bedrock, BytePlus, Cloudflare Workers AI, Cohere, DeepSeek, Fireworks AI, Google Gemini, Groq and other frameworks or platforms. It also names integrations such as Google ADK, AG2, Agno, Autogen, BeeAI, CrewAI, Dify, DSPy and Flowise AI. OpenAI appears in the repository topics, but explicit OpenAI integration details are not included in the supplied excerpt. MCP support is not stated in the repository metadata.

Observações sobre licença e riscos

Repository metadata identifies the license as Apache-2.0. The README states that the server, web application and core observability and evaluation platform can be self-hosted without a commercial license. Review the repository license before deployment or redistribution: https://api.github.com/licenses/apache-2.0

Lançamento e manutenção

The README directs users to the changelog and specifically warns that version 1.7.0 includes important updates and breaking changes. The repository metadata reports the default branch as `main` and the latest supplied push timestamp as 2026-08-03T09:50:41Z. A specific current release version is not stated in the repository metadata.

Firecrawl

firecrawl/firecrawl

★ 161,1KTypeScript

LangChain

langchain-ai/langchain

★ 143,6KPython

RAGFlow

infiniflow/ragflow

★ 86,7KGo