Opik

Opik, built by Comet, is an open source platform under the Apache 2.0 license for tracing, evaluating, and monitoring large language model applications, RAG pipelines, and agentic workflows. Comet spent years as an experiment tracking and model monitoring company before launching Opik for teams building on top of models rather than training their own, and Opik has become one of the fastest growing projects in the space, passing twelve thousand GitHub stars within its first year. It runs as a managed cloud service with a generous free tier, or self hosted through Docker or Kubernetes for teams that need full control of their data.

Opik instruments application code to capture structured traces of every LLM call, tool call, and agent step, with inputs, outputs, token usage, latency, and cost, organized into spans with support for distributed tracing, multimodal logging, and agent graph visualization for frameworks like LangGraph. A custom query language helps filter trace data, and cost tracking surfaces where to optimize. Integrations span more than sixty tools including OpenAI, Anthropic, LangChain, LangGraph, CrewAI, LlamaIndex, DSPy, and LiteLLM, and it reaches low code platforms like Dify and Flowise.

Evaluation is central: more than thirty built in metrics for hallucination, RAG quality, relevance, and agent specific scoring, LLM as a judge, human annotation queues, prompt versioning with a playground, and online evaluation rules that score production traces in real time. An Agent Optimizer SDK offers multiple algorithms to tune prompts, parameters, and tool selection, and Opik Guardrails add production safety such as PII redaction and topic blocking. Pricing keeps the full open source feature set free to self host, adds a free cloud tier, and offers Pro and Enterprise cloud plans that differ mainly in usage limits, data retention, identity management, and support. Enterprise adds custom hosting, advanced authentication, and increased regulatory compliance.