LiteLLM vs Portkey (2026)

LiteLLM's MCP Gateway gives agents one fixed endpoint for every registered MCP server, with access controlled per key and team and server authentication for OAuth, OAuth passthrough, on-behalf-of, AWS SigV4 and JWT signer schemes. On the Responses and Chat Completions endpoints the proxy fetches the MCP server's tools and, when configured to, executes the returned tool calls itself. So agents take authenticated action in outside systems through the gateway and do not just hand a suggestion back.

Sourcedocs.litellm.ai/docs/mcpread 2026-09-22

Agents authenticate once to Portkey's MCP Gateway, which sits between them and MCP servers. For each tool call, the gateway checks that user's access to the server and tool, then injects the upstream credentials (OAuth, API keys or custom headers, with external identity providers such as Okta and Auth0). Servers and individual tools are provisioned per workspace and user. Through the gateway, agents take authenticated action in outside systems.

Sourcedocs.portkey.ai/docs/product/mcp-gatewayread 2026-09-22

With an MCP tool's require_approval set to never, the proxy executes the model's tool calls and feeds the results back into the model before returning the answer, so the gateway runs a multi-step agent loop itself, and the A2A Agent Gateway routes calls to registered agents. There is no stated workflow definition, deterministic node or versioned flow, so sequencing, branching and retries that mix deterministic steps with agent steps are not covered.

Sourcedocs.litellm.ai/docs/mcpread 2026-09-22

The Agent Gateway is a proxy between agent clients and the customer's own agent servers. It handles authentication, access control and observability for registered A2A agents, and the agents and their steps run on the customer's servers. Portkey has no workflow definitions and runs no agents of its own, so nothing in it sequences or branches agent steps.

Sourcedocs.portkey.ai/docs/product/agent-gatewayread 2026-09-22

Work reaches agents through LiteLLM when a caller sends a request, and model, MCP and A2A calls are all invoked by the client. There is no stated schedule, event, webhook or inbound queue that starts an agent run, and budget alerts notify people and do not wake an agent. A gateway that only answers callers has no way to wake an agent on its own.

Sourcedocs.litellm.ai/docs/a2aread 2026-09-22

Work reaches agents through Portkey only when a client sends a request. Model, MCP and A2A calls are all made by the caller, and the Agent Gateway proxies them to the customer's agent servers. Portkey names no schedule, event, webhook or inbound queue that starts an agent run. The gateway waits for callers and does not wake an agent on its own.

Sourcedocs.portkey.ai/docs/product/agent-gatewayread 2026-09-22

The /rag/ingest endpoint is an all in one ingestion pipeline (upload, chunk, embed, write to a vector store) into OpenAI vector stores, Bedrock Knowledge Bases, Vertex AI RAG Engine, Gemini or AWS S3 Vectors, and /rag/query searches the ingested content and generates a response from it. Vector store access is permissioned per team in the gateway. New customer knowledge enters a persistent index without retraining, so the retrieval structure over the customer's knowledge is maintained, persists, scales past the context window and stays queryable.

Sourcedocs.litellm.ai/docs/rag_ingestread 2026-09-22

Embedding and file requests route through Portkey to providers, and responses are cached. Portkey maintains no retrieval structure over the customer's documents, and routing embedding calls builds none.

Sourceportkey.ai/pricingread 2026-09-22

The proxy's Memory Management API (/v1/memory, LiteLLM v1.83.10 or later with PostgreSQL connected) keeps entries across sessions, scoped per user and team with role-based read and write rules, and supports create, read, update, list by key prefix and delete. Memory lives in the gateway's own Postgres, and one user's entries can be deleted without touching the rest. The application reads entries and places them in the prompt. The scope is stated, but no lifetime is.

Sourcedocs.litellm.ai/docs/proxy/memoryread 2026-09-22

Features cover routing, caching, observability, guardrails, prompt management and the MCP and agent gateways. None of them is a memory layer that keeps an agent's state or a user's context across sessions. The response cache holds model responses.

Sourceportkey.ai/pricingread 2026-09-22

Guardrails run before, during or after a call and can be scoped per key and team (PII masking, prompt injection and secret detection, content moderation and third-party guardrail providers), and the MCP Gateway adds per-key and per-team tool permissions and guardrails on tool results. These are constraints the customer controls on what an agent may do.

When an MCP tool's require_approval is set to anything other than never, the proxy returns the tool calls to the client so they can be reviewed and executed manually, and that review happens in the customer's own client. LiteLLM itself has no stated surface where a person reviews and approves an agent action before it commits.

Sourcedocs.litellm.ai/docs/mcpread 2026-09-22

Inputs and outputs pass through Portkey Guardrails, which act on the verdict automatically by denying the request, flagging it and letting it through, retrying or falling back. The MCP Gateway applies content filters per server, per user tool provisioning and rate limits on tool calls. Org and workspace admins can enforce guardrails, budgets and rate limits, so the customer sets these limits on what an agent may do. Portkey describes no step where a person reviews, approves or must validate an agent action before it commits.

Sourcedocs.portkey.ai/docs/product/guardrailsread 2026-09-22

Access control covers SSO for the Admin UI through Okta, Azure AD, Google Workspace or any OIDC or SAML provider (free for up to five users, an Enterprise license beyond that), SCIM, JWT authentication against the customer's own identity provider, role-based access control across organizations, teams and user roles, IP allowlists, public and private route controls, key rotation and external secret managers.

LiteLLM reports SOC 2 Type II, with the current report available through the LiteLLM Trust Center. Audit logs with retention policies add to that access model, identity integration and attestation. Most controls sit in the Enterprise tier.

Sourcedocs.litellm.ai/docs/enterpriseread 2026-09-22

Enterprise plans get SSO through OIDC. Portkey also has fine grained role based access control, encrypted storage of provider keys, TLS 1.2+ in transit and AES-256 at rest. It claims compliance with SOC 2, ISO 27001, GDPR and HIPAA, with regular third party audits and penetration tests, and its Trust Center is at trust.portkey.ai. Enterprise includes SOC 2 Type 2 and custom BAAs. SCIM works with Okta and Microsoft Entra, and Portkey supports audit logs, JWT authentication and API key rotation.

Sourcedocs.portkey.ai/docs/product/enterprise-offering/security-portkeyread 2026-09-22

The gateway records what each caller's agent did through it, covering request and response content for model and agent calls with user, key and team attribution, latency and cost (shown in the A2A Agent Gateway's Logs tab for invoked agents), and Prometheus metrics in the open source core.

Enterprise adds per-key or per-team log routing to Langfuse, LangSmith, Arize and other callbacks, audit logs of admin actions with retention policies kept apart from request logs, and log export to GCS or Azure Blob. Traffic that passes through the gateway can be inspected step by step, with audit logs kept apart from runtime traces.

Sourcedocs.litellm.ai/docs/a2aread 2026-09-22

Every tool call through the MCP Gateway is logged with the user, parameters and response, and the log can be filtered by server, user or time range. Model requests carry logs, traces, feedback, custom metadata, filters and alerts on every plan, with retention set per plan. Enterprise adds audit logs, logs export and OpenTelemetry export.

Sourcedocs.portkey.ai/docs/product/mcp-gatewayread 2026-09-22

Deployment is self hosted. Official Docker images, a Helm chart and a Terraform module run on the customer's own Postgres and Redis, with one-click deploy into AWS, GCP or Azure and fully air-gapped installation. Enterprise includes air-gapped deployment and multi-region deployment under one license with an admin and worker split, and the customer's data and keys never leave its own infrastructure. The customer can run it in its own cloud, on premises or fully air gapped.

Sourcelitellm.airead 2026-09-22

Enterprise offers private cloud deployment, VPC hosting and data isolation, and the open source gateway can be hosted by the customer. Hybrid deployments run on the customer's AWS (EKS, ECS, Marketplace), GCP and Azure (AKS, ACA), and air gapped deployment is available. A Gateway URLs setting points the dashboard at the customer's own self hosted AI and MCP gateways. Some of these options are Enterprise only.

Sourceportkey.ai/pricingread 2026-09-22

LiteLLM offers no agents of its own for a customer to adopt. The AI Hub on the Enterprise tier lists the models, agents, MCP servers and skills the customer has registered, which is a directory of the customer's own assets, and the Google AI Studio managed agents LiteLLM supports live entirely on Google's side, with LiteLLM as the auth and routing layer. Nothing there is a ready made workflow, template or role specific agent, and a model or tool catalog is integration, not a prebuilt pack.

Sourcedocs.litellm.ai/docs/enterpriseread 2026-09-22

No ready made agents, workflows, templates or role specific agents ship with Portkey. The Prompt Library holds the customer's own templates, and the Skills Registry shares skills the team writes. The prebuilt assets are guardrail checks and partner guardrail integrations, which constrain what agents do.

Sourcedocs.portkey.ai/docs/product/guardrailsread 2026-09-22

The customer chooses the model. One OpenAI-compatible API reaches 140+ providers and 1,800+ models, with models set in the customer's own configuration and swapped without changing application code, the customer's internal, fine-tuned and self-hosted models behind the same key, load balancing across providers, regions and keys, lowest-cost routing, and an Auto Router that sends prompts to model tiers the customer configures.

Sourcelitellm.airead 2026-09-22

Model choice sits with the customer. Portkey's Universal API puts many model providers behind one interface. Routing configs the customer writes set providers, model overrides, fallbacks, load balancing weights, retries, conditional routing and circuit breakers. Private model deployments on AWS, GCP and Azure are supported too.

Sourceportkey.ai/pricingread 2026-09-22

A Python SDK and a proxy with an OpenAI compatible REST API ship with LiteLLM, and the gateway's own features are callable from outside. The A2A Agent Gateway serves a proxied agent card for each registered agent, pinned to A2A 0.3 or 1.0, and agents are invoked through the A2A SDK or the OpenAI SDK. Memory Management and ingestion have their own REST endpoints (/v1/memory, /v1/rag/ingest) with curl and Python examples.

Sourcedocs.litellm.ai/docs/a2aread 2026-09-22

Developers get Portkey's own API and SDKs. The same feedback call works through the Node and Python SDKs and the REST API, and Portkey publishes an OpenAPI specification. Prompts are served through a Prompt API, and the Agent Gateway acts as a server to downstream agent clients for registered A2A agents.

Sourcedocs.portkey.ai/docs/product/observability/feedbackread 2026-09-22

Shadow evaluations sample a key's, team's or user's live traffic, send each sampled request through a candidate router configuration without returning that answer to the client, and have an LLM judge compare it blind against the answer the current model served. A job runs up to 30 days and can compare several configurations on the same traffic before anything changes, and after a switch each request carries its routing decision and savings. A change is evaluated against the customer's own traffic, with a judge verdict.

Sourcedocs.litellm.ai/docs/auto_router/evaluateread 2026-09-22

Canary testing sends a set share of production traffic to a new model or prompt through a routing config, on every plan. Each request carries guardrail verdicts (pass or fail per check, counted in the log drawer) and weighted feedback scores sent through the Feedback API, which can be analyzed by metadata. Guardrails can deny, retry or fall back on a failed check. Together these let a team compare a change against current traffic.

Sourcedocs.portkey.ai/docs/product/ai-gateway/canary-testingread 2026-09-22

The product is a gateway for models, MCP servers and agents, and there is no stated browser, desktop or remote computer session that LiteLLM runs for an agent. The sandboxes in the Managed Agents Platform announcement, and the swap of OpenAI's Code Interpreter for E2B or OpenSandbox, are code execution, not control of a real interface that an agent drives itself.

Sourcelitellm.airead 2026-09-22

The gateway routes model calls, including provider computer use tools, whose actions the customer's own code carries out. Portkey describes no browser, desktop or remote computer session that it runs for an agent, and it wires in no headless fetch or third party browser engine.

Sourceportkey.ai/pricingread 2026-09-22