Helicone vs LiteLLM (2026)

Model providers connect through the gateway. Helicone also works with frameworks such as the Vercel AI SDK, LiteLLM, OpenRouter and CrewAI, sends webhooks out, and offers a Zapier app that runs chat completions through the gateway. These route model traffic and send events. None of them connects an agent to a real system to take actions.

SourceHelicone, docs.helicone.ai/llms.txt (integrations, webhooks, Zapier, web search)read 2026-09-21

LiteLLM's MCP Gateway gives agents one fixed endpoint for every registered MCP server, with access controlled per key and team and server authentication for OAuth, OAuth passthrough, on-behalf-of, AWS SigV4 and JWT signer schemes. On the Responses and Chat Completions endpoints the proxy fetches the MCP server's tools and, when configured to, executes the returned tool calls itself. So agents take authenticated action in outside systems through the gateway and do not just hand a suggestion back.

Sourcedocs.litellm.ai/docs/mcpread 2026-09-22

Per request, the gateway routes and falls back between providers, and prompts are assembled from versioned templates at call time. There are no workflows that sequence, branch or retry agent steps, or that combine deterministic nodes with agent steps.

SourceHelicone, docs.helicone.ai/llms.txt (provider routing, prompt assembly)read 2026-09-21

With an MCP tool's require_approval set to never, the proxy executes the model's tool calls and feeds the results back into the model before returning the answer, so the gateway runs a multi-step agent loop itself, and the A2A Agent Gateway routes calls to registered agents. There is no stated workflow definition, deterministic node or versioned flow, so sequencing, branching and retries that mix deterministic steps with agent steps are not covered.

Sourcedocs.litellm.ai/docs/mcpread 2026-09-22

Alerts fire when an application crosses error thresholds or cost limits. Automated weekly reports deliver usage, cost and performance summaries to email or Slack, and webhooks send request events to the customer's endpoints.

SourceHelicone, docs.helicone.ai/llms.txt (alerts, reports, webhooks) and helicone.ai/pricingread 2026-09-21

Work reaches agents through LiteLLM when a caller sends a request, and model, MCP and A2A calls are all invoked by the client. There is no stated schedule, event, webhook or inbound queue that starts an agent run, and budget alerts notify people and do not wake an agent. A gateway that only answers callers has no way to wake an agent on its own.

Sourcedocs.litellm.ai/docs/a2aread 2026-09-22

Model traffic is routed and logged, web search can be added to Anthropic models through the gateway, and datasets hold logged requests for evaluation and fine tuning. There is no document ingestion, index, retrieval layer or knowledge API that grounds an agent in company data.

SourceHelicone, docs.helicone.ai/llms.txt (web search, datasets)read 2026-09-21

The /rag/ingest endpoint is an all in one ingestion pipeline (upload, chunk, embed, write to a vector store) into OpenAI vector stores, Bedrock Knowledge Bases, Vertex AI RAG Engine, Gemini or AWS S3 Vectors, and /rag/query searches the ingested content and generates a response from it. Vector store access is permissioned per team in the gateway. New customer knowledge enters a persistent index without retraining, so the retrieval structure over the customer's knowledge is maintained, persists, scales past the context window and stays queryable.

Sourcedocs.litellm.ai/docs/rag_ingestread 2026-09-22

Logged requests, sessions and prompt versions persist, and the gateway can automatically clear old tool uses and thinking blocks from long running agent sessions. Agents get no session, workflow or long term memory to read and write.

SourceHelicone, docs.helicone.ai/llms.txt (sessions, context editing)read 2026-09-21

The proxy's Memory Management API (/v1/memory, LiteLLM v1.83.10 or later with PostgreSQL connected) keeps entries across sessions, scoped per user and team with role-based read and write rules, and supports create, read, update, list by key prefix and delete. Memory lives in the gateway's own Postgres, and one user's entries can be deleted without touching the rest. The application reads entries and places them in the prompt. The scope is stated, but no lifetime is.

Sourcedocs.litellm.ai/docs/proxy/memoryread 2026-09-22

With LLM Security enabled, Helicone screens each user message with Meta's Prompt Guard, and optionally Llama Guard across 14 threat categories. It blocks detected threats with an error response before the request is processed. Moderation filters harmful content, and custom rate limits cap requests by count or cost.

SourceHelicone, docs.helicone.ai/features/advanced-usage/llm-security and llms.txt (moderations, custom rate limits); docs.helicone.ai/features/advanced-usage/llm-security.mdread 2026-09-21

Guardrails run before, during or after a call and can be scoped per key and team (PII masking, prompt injection and secret detection, content moderation and third-party guardrail providers), and the MCP Gateway adds per-key and per-team tool permissions and guardrails on tool results. These are constraints the customer controls on what an agent may do.

When an MCP tool's require_approval is set to anything other than never, the proxy returns the tool calls to the client so they can be reviewed and executed manually, and that review happens in the customer's own client. LiteLLM itself has no stated surface where a person reviews and approves an agent action before it commits.

Sourcedocs.litellm.ai/docs/mcpread 2026-09-22

SAML SSO comes on the Enterprise plan, with identity integration through the customer's own provider, and each plan sets its organizations and seats. SOC 2 Type II and HIPAA compliance come from the Team plan up, with InfoSec reviews and customized MSAs on Enterprise. Provider keys are encrypted, data sits in EU or US regions, and retention is configurable. Helicone names no roles or permissions inside an organization.

SourceHelicone, helicone.ai/pricing and docs.helicone.ai/references/data-autonomyread 2026-10-01

Access control covers SSO for the Admin UI through Okta, Azure AD, Google Workspace or any OIDC or SAML provider (free for up to five users, an Enterprise license beyond that), SCIM, JWT authentication against the customer's own identity provider, role-based access control across organizations, teams and user roles, IP allowlists, public and private route controls, key rotation and external secret managers.

LiteLLM reports SOC 2 Type II, with the current report available through the LiteLLM Trust Center. Audit logs with retention policies add to that access model, identity integration and attestation. Most controls sit in the Enterprise tier.

Sourcedocs.litellm.ai/docs/enterpriseread 2026-09-22

Every request through the gateway or SDK is logged with prompt, completion, tokens, cost, latency and errors. Helicone groups them into sessions for agents and pipelines, adds user analytics and custom properties, and lets teams query analytics directly with HQL, a SQL interface with row level security. Data export is available. Retention runs 7 days on Hobby, 1 month on Pro, 3 months on Team and indefinitely on Enterprise, where it is also configurable.

SourceHelicone, docs.helicone.ai/llms.txt (sessions, HQL, data export) and helicone.ai/pricingread 2026-09-21

The gateway records what each caller's agent did through it, covering request and response content for model and agent calls with user, key and team attribution, latency and cost (shown in the A2A Agent Gateway's Logs tab for invoked agents), and Prometheus metrics in the open source core.

Enterprise adds per-key or per-team log routing to Langfuse, LangSmith, Arize and other callbacks, audit logs of admin actions with retention policies kept apart from request logs, and log export to GCS or Azure Blob. Traffic that passes through the gateway can be inspected step by step, with audit logs kept apart from runtime traces.

Sourcedocs.litellm.ai/docs/a2aread 2026-09-22

Customers choose an EU or US region for data residency on Helicone Cloud. The open source platform self hosts with Docker or Kubernetes and Helm, and the Enterprise plan includes on premises deployment.

SourceHelicone, docs.helicone.ai/references/data-autonomy and llms.txt (self-hosting, Docker, Kubernetes), and helicone.ai/pricing; docs.helicone.ai/references/data-autonomy.mdread 2026-09-21

Deployment is self hosted. Official Docker images, a Helm chart and a Terraform module run on the customer's own Postgres and Redis, with one-click deploy into AWS, GCP or Azure and fully air-gapped installation. Enterprise includes air-gapped deployment and multi-region deployment under one license with an admin and worker split, and the customer's data and keys never leave its own infrastructure. The customer can run it in its own cloud, on premises or fully air gapped.

Sourcelitellm.airead 2026-09-22

A model registry and pricing data cover the models the gateway reaches, and cookbooks explain how to build agents. There are no ready made agents, templates or packaged workflows a customer adopts.

SourceHelicone, docs.helicone.ai/llms.txt (model registry, cookbooks)read 2026-09-21

LiteLLM offers no agents of its own for a customer to adopt. The AI Hub on the Enterprise tier lists the models, agents, MCP servers and skills the customer has registered, which is a directory of the customer's own assets, and the Google AI Studio managed agents LiteLLM supports live entirely on Google's side, with LiteLLM as the auth and routing layer. Nothing there is a ready made workflow, template or role specific agent, and a model or tool catalog is integration, not a prebuilt pack.

Sourcedocs.litellm.ai/docs/enterpriseread 2026-09-22

The Helicone AI Gateway gives one OpenAI compatible API across more than 100 providers, with automatic provider routing, fallbacks, caching and custom rate limits by request count or cost. Provider keys are stored encrypted in a vault, so customers bring their own keys or use gateway credits.

SourceHelicone, docs.helicone.ai/llms.txt (AI Gateway overview, provider routing, error handling and fallback, custom rate limits) and docs.helicone.ai/references/data-autonomyread 2026-09-21

The customer chooses the model. One OpenAI-compatible API reaches 140+ providers and 1,800+ models, with models set in the customer's own configuration and swapped without changing application code, the customer's internal, fine-tuned and self-hosted models behind the same key, load balancing across providers, regions and keys, lowest-cost routing, and an Auto Router that sends prompts to model tiers the customer configures.

Sourcelitellm.airead 2026-09-22

A REST API covers requests, sessions, scores, evaluations, prompts, webhooks and trace logging. Alongside it sit an OpenAI compatible gateway API, SDK integrations, webhooks, a Zapier app, and a Helicone MCP server that lets MCP compatible assistants query observability data. The codebase is open source.

SourceHelicone, docs.helicone.ai/llms.txt (REST reference, MCP server, webhooks, Zapier)read 2026-09-21

A Python SDK and a proxy with an OpenAI compatible REST API ship with LiteLLM, and the gateway's own features are callable from outside. The A2A Agent Gateway serves a proxied agent card for each registered agent, pinned to A2A 0.3 or 1.0, and agents are invoked through the A2A SDK or the OpenAI SDK. Memory Management and ingestion have their own REST endpoints (/v1/memory, /v1/rag/ingest) with curl and Python examples.

Sourcedocs.litellm.ai/docs/a2aread 2026-09-22

Scores and user feedback on requests and sessions come in through the API. Helicone curates request data into datasets for fine tuning and evaluation, provides a playground and versioned prompts deployed through the gateway, and tracks score distributions over time. Customers cannot run an application against fixtures or datasets before production, and there are no configurable quality gates.

SourceHelicone, docs.helicone.ai/llms.txt (eval scores, datasets, prompts, evaluations API) and helicone.ai/pricingread 2026-09-21

Shadow evaluations sample a key's, team's or user's live traffic, send each sampled request through a candidate router configuration without returning that answer to the client, and have an LLM judge compare it blind against the answer the current model served. A job runs up to 30 days and can compare several configurations on the same traffic before anything changes, and after a switch each request carries its routing decision and savings. A change is evaluated against the customer's own traffic, with a judge verdict.

Sourcedocs.litellm.ai/docs/auto_router/evaluateread 2026-09-22

The product covers observability, the AI Gateway, prompts, datasets, security features and self hosting. No agent controls a browser, desktop or computer.

SourceHelicone, docs.helicone.ai/llms.txtread 2026-09-21

The product is a gateway for models, MCP servers and agents, and there is no stated browser, desktop or remote computer session that LiteLLM runs for an agent. The sandboxes in the Managed Agents Platform announcement, and the swap of OpenAI's Code Interpreter for E2B or OpenSandbox, are code execution, not control of a real interface that an agent drives itself.

Sourcelitellm.airead 2026-09-22