LiteLLM

LiteLLM's MCP Gateway gives agents one fixed endpoint for every registered MCP server, with access controlled per key and team and server authentication documented for OAuth, OAuth passthrough, on-behalf-of, AWS SigV4 and JWT signer schemes. On the Responses and Chat Completions endpoints the proxy fetches the MCP server's tools, and when configured to, executes the returned tool calls itself. That is an authenticated action framework, so agents take authenticated action in outside systems through the gateway rather than handing a suggestion back.

Sourcedocs.litellm.ai/docs/mcpread 2026-09-22

With an MCP tool's require_approval set to never, the proxy executes the model's tool calls and feeds the results back into the model before returning the answer, so the gateway runs a multi-step agent loop itself, and the A2A Agent Gateway routes calls to registered agents. No workflow definition, deterministic node or versioned flow is documented, so sequencing, branching and retries that mix deterministic steps with agent steps are not covered.

Sourcedocs.litellm.ai/docs/mcpread 2026-09-22

The /rag/ingest endpoint is an all in one ingestion pipeline (upload, chunk, embed, write to a vector store) into OpenAI vector stores, Bedrock Knowledge Bases, Vertex AI RAG Engine, Gemini or AWS S3 Vectors, and /rag/query searches the ingested content and generates a response from it; vector store access is permissioned per team in the gateway. New customer knowledge enters a persistent index without retraining, so the retrieval structure over the customer's knowledge is maintained, persists, scales past the context window and stays queryable.

Sourcedocs.litellm.ai/docs/rag_ingestread 2026-09-22

Guardrails run before, during or after a call and can be scoped per key and team (PII masking, prompt injection and secret detection, content moderation and third-party guardrail providers), and the MCP Gateway adds per-key and per-team tool permissions and guardrails on tool results. These are constraints the customer controls on what an agent may do.

When an MCP tool's require_approval is set to anything other than never, the proxy returns the tool calls to the client so they can be reviewed and executed manually; that review happens in the customer's own client. LiteLLM itself documents no surface where a person reviews and approves an agent action before it commits.

Sourcedocs.litellm.ai/docs/mcpread 2026-09-22

Access control is documented on the Enterprise page: SSO for the Admin UI through Okta, Azure AD, Google Workspace or any OIDC or SAML provider (free for up to five users, an Enterprise license beyond that), SCIM, JWT authentication against the customer's own identity provider, role-based access control across organizations, teams and user roles, IP allowlists, public and private route controls, key rotation and external secret managers.

LiteLLM's Data Privacy and Security page (docs.litellm.ai/docs/data_security) states SOC 2 Type II, with the current report available through the LiteLLM Trust Center. That pairs a named access model with identity integration and an attestation, and audit logs with retention policies add to it. Most controls sit in the Enterprise tier.

Sourcedocs.litellm.ai/docs/enterpriseread 2026-09-22

The gateway records what each caller's agent did through it: request and response content for model and agent calls with user, key and team attribution, latency and cost (the A2A Agent Gateway page shows this in the Logs tab for invoked agents), and Prometheus metrics in the open source core.

The Enterprise page adds per-key or per-team log routing to Langfuse, LangSmith, Arize and other callbacks, audit logs of admin actions with retention policies kept apart from request logs, and log export to GCS or Azure Blob. Traffic that passes through the gateway can be inspected step by step, with audit logs kept apart from runtime traces.

Sourcedocs.litellm.ai/docs/a2aread 2026-09-22

The proxy's Memory Management API (/v1/memory, LiteLLM v1.83.10 or later with PostgreSQL connected) keeps entries across sessions, scoped per user and team with role-based read and write rules, and supports create, read, update, list by key prefix and delete, so a buyer can say where memory lives (the gateway's own Postgres) and delete one user's entries without touching the rest. The application reads entries and places them in the prompt. The scope is stated, but no lifetime is.

Sourcedocs.litellm.ai/docs/proxy/memoryread 2026-09-22

Deployment is self hosted. The homepage documents official Docker images, a Helm chart and a Terraform module, running on the customer's own Postgres and Redis, one-click deploy into AWS, GCP or Azure, and fully air-gapped installation. The pricing page FAQ (litellm.ai/pricing) confirms air-gapped deployment on Enterprise and states that the customer's data and keys never leave its own infrastructure, and the Enterprise docs page adds multi-region deployment under one license with an admin and worker split. The customer can run it in its own cloud, on premises or fully air gapped.

Sourcelitellm.airead 2026-09-22

No agents of LiteLLM's own are offered for a buyer to adopt. The AI Hub on the Enterprise page publishes a page of the models, agents, MCP servers and skills the customer has registered, which is a directory of the customer's own assets, and the Google AI Studio managed agents LiteLLM supports live entirely on Google's side, with LiteLLM as the auth and routing layer. Nothing there is a ready made workflow, template or role specific agent; a model or tool catalog is integration, not a pack.

Sourcedocs.litellm.ai/docs/enterpriseread 2026-09-22

Work reaches agents through LiteLLM when a caller sends a request: model, MCP and A2A calls are all invoked by the client. No schedule, event, webhook or inbound queue that starts an agent run is documented; budget alerts notify people rather than wake an agent. A gateway that only answers callers has no way to wake an agent on its own.

Sourcedocs.litellm.ai/docs/a2aread 2026-09-22

The customer chooses. The homepage documents one OpenAI-compatible API to 140+ providers and 1,800+ models, with models set in the customer's own configuration and swapped without changing application code, the customer's internal, fine-tuned and self-hosted models behind the same key, load balancing across providers, regions and keys, lowest-cost routing, and an Auto Router that sends prompts to model tiers the customer configures.

Sourcelitellm.airead 2026-09-22

A Python SDK and a proxy with a documented OpenAI compatible REST API ship with LiteLLM, and the gateway's own features are callable from outside. The A2A Agent Gateway serves a proxied agent card for each registered agent, pinned to A2A 0.3 or 1.0, and agents are invoked through the A2A SDK or the OpenAI SDK; the Memory Management and /rag/ingest pages document REST endpoints (/v1/memory, /v1/rag/ingest) with curl and Python examples.

Sourcedocs.litellm.ai/docs/a2aread 2026-09-22

Shadow evaluations sample a key's, team's or user's live traffic, send each sampled request through a candidate router configuration without returning that answer to the client, and have an LLM judge compare it blind against the answer the current model served; a job runs up to 30 days and can compare several configurations on the same traffic before anything changes, and after a switch each request carries its routing decision and savings. A change is evaluated against the customer's own traffic, with a judge verdict.

Sourcedocs.litellm.ai/docs/auto_router/evaluateread 2026-09-22

The product is a gateway for models, MCP servers and agents; no browser, desktop or remote computer session that LiteLLM runs for an agent is documented. The sandboxes the Managed Agents Platform announcement describes, and the blog's swap of OpenAI's Code Interpreter for E2B or OpenSandbox, are code execution, not control of a real interface that an agent drives itself.

Sourcelitellm.airead 2026-09-22