LiteLLM vs nexos.ai (2026)

LiteLLM's MCP Gateway gives agents one fixed endpoint for every registered MCP server, with access controlled per key and team and server authentication for OAuth, OAuth passthrough, on-behalf-of, AWS SigV4 and JWT signer schemes. On the Responses and Chat Completions endpoints the proxy fetches the MCP server's tools and, when configured to, executes the returned tool calls itself. So agents take authenticated action in outside systems through the gateway and do not just hand a suggestion back.

Sourcedocs.litellm.ai/docs/mcpread 2026-09-22

Workspace Integrations reach Google Drive, Mail and Calendar and Microsoft Outlook, OneDrive, SharePoint, Teams and Calendar on the productivity side, Atlassian, GitHub and GitLab through their MCP servers, and Slack, HiBob, Looker, Google Ads and BigQuery. Organization admins activate them and each user connects through the provider's own sign in. Agents take write actions through these integrations, such as sending a message, creating a ticket or updating a record.

Sourcedocs.nexos.ai/workspace/integrationsread 2026-09-22

With an MCP tool's require_approval set to never, the proxy executes the model's tool calls and feeds the results back into the model before returning the answer, so the gateway runs a multi-step agent loop itself, and the A2A Agent Gateway routes calls to registered agents. There is no stated workflow definition, deterministic node or versioned flow, so sequencing, branching and retries that mix deterministic steps with agent steps are not covered.

Sourcedocs.litellm.ai/docs/mcpread 2026-09-22

A visual Flow in Agent Builder shows the steps an agent will take to complete its task, with optional human in the loop review steps, and agents run multi step work across connected tools on demand or on a schedule. There are no deterministic nodes, branching or multi agent handoff.

Sourcedocs.nexos.ai/workspace/agents/agent-builderread 2026-09-22

Work reaches agents through LiteLLM when a caller sends a request, and model, MCP and A2A calls are all invoked by the client. There is no stated schedule, event, webhook or inbound queue that starts an agent run, and budget alerts notify people and do not wake an agent. A gateway that only answers callers has no way to wake an agent on its own.

Sourcedocs.litellm.ai/docs/a2aread 2026-09-22

A schedule can be set on an agent (hourly, daily, monthly or yearly), so it runs automatically in the background. Agents can run periodically, for example to produce a morning summary of Slack updates, and scheduled runs appear in the agent's Activity history. Agents are also reachable from Slack, Microsoft Teams, Telegram and WhatsApp.

Sourcedocs.nexos.ai/changelog/changelog/latest-updatesread 2026-09-22

The /rag/ingest endpoint is an all in one ingestion pipeline (upload, chunk, embed, write to a vector store) into OpenAI vector stores, Bedrock Knowledge Bases, Vertex AI RAG Engine, Gemini or AWS S3 Vectors, and /rag/query searches the ingested content and generates a response from it. Vector store access is permissioned per team in the gateway. New customer knowledge enters a persistent index without retraining, so the retrieval structure over the customer's knowledge is maintained, persists, scales past the context window and stays queryable.

Sourcedocs.litellm.ai/docs/rag_ingestread 2026-09-22

Files added to a Project are chunked, embedded and stored in the Project's context index, and nexos.ai uses Retrieval-Augmented Generation to pull relevant context from them when answering. Projects can be shared with view or edit access. Agents take files from a Project as their knowledge base to give sourced answers, and that knowledge can be updated or removed at any time. New knowledge enters the persistent index without retraining and stays queryable beyond the context window.

Sourcedocs.nexos.ai/workspace/projectsread 2026-09-22

The proxy's Memory Management API (/v1/memory, LiteLLM v1.83.10 or later with PostgreSQL connected) keeps entries across sessions, scoped per user and team with role-based read and write rules, and supports create, read, update, list by key prefix and delete. Memory lives in the gateway's own Postgres, and one user's entries can be deleted without touching the rest. The application reads entries and places them in the prompt. The scope is stated, but no lifetime is.

Sourcedocs.litellm.ai/docs/proxy/memoryread 2026-09-22

User Memory keeps context about each user across chats and agents. The app adds context as the user works, and the user can view saved memories, edit them, delete one or all, or turn Memory off, alongside personalization details and custom instructions the user writes. Memory is scoped per user, in Chat and Agents, and users can delete their own entries, but nexos.ai states no lifetime for it.

Sourcedocs.nexos.ai/workspace/chat/memory-and-personalizationread 2026-09-22

Guardrails run before, during or after a call and can be scoped per key and team (PII masking, prompt injection and secret detection, content moderation and third-party guardrail providers), and the MCP Gateway adds per-key and per-team tool permissions and guardrails on tool results. These are constraints the customer controls on what an agent may do.

When an MCP tool's require_approval is set to anything other than never, the proxy returns the tool calls to the client so they can be reviewed and executed manually, and that review happens in the customer's own client. LiteLLM itself has no stated surface where a person reviews and approves an agent action before it commits.

Sourcedocs.litellm.ai/docs/mcpread 2026-09-22

Before any action that changes something in another tool, such as sending a message, creating a ticket or updating a record, Human-in-the-Loop Approvals pause the agent and show an approval card naming the action, the integration and its key details. The person approves (the agent acts and continues) or declines (the action is not taken and the run stops). Approvals appear in the conversation for chat agents and in Activity history for scheduled agents, and are switched on per integration inside each agent's settings.

Sourcedocs.nexos.ai/workspace/agents/human-in-the-loop-hitl-approvalsread 2026-09-22

Access control covers SSO for the Admin UI through Okta, Azure AD, Google Workspace or any OIDC or SAML provider (free for up to five users, an Enterprise license beyond that), SCIM, JWT authentication against the customer's own identity provider, role-based access control across organizations, teams and user roles, IP allowlists, public and private route controls, key rotation and external secret managers.

LiteLLM reports SOC 2 Type II, with the current report available through the LiteLLM Trust Center. Audit logs with retention policies add to that access model, identity integration and attestation. Most controls sit in the Enterprise tier.

Sourcedocs.litellm.ai/docs/enterpriseread 2026-09-22

Access is managed per user, as organization admins invite, update and remove users and assign each a role and team, and accounts can be managed through the customer's identity provider with linked SSO sign in. Company wide guardrails with per model overrides and exclusions apply across the workspace. nexos.ai states SOC 2 Type II and ISO 27001 certification, with reports in its Trust Center.

Sourcedocs.nexos.ai/workspace/settings/organization/usersread 2026-09-22

The gateway records what each caller's agent did through it, covering request and response content for model and agent calls with user, key and team attribution, latency and cost (shown in the A2A Agent Gateway's Logs tab for invoked agents), and Prometheus metrics in the open source core.

Enterprise adds per-key or per-team log routing to Langfuse, LangSmith, Arize and other callbacks, audit logs of admin actions with retention policies kept apart from request logs, and log export to GCS or Azure Blob. Traffic that passes through the gateway can be inspected step by step, with audit logs kept apart from runtime traces.

Sourcedocs.litellm.ai/docs/a2aread 2026-09-22

Every API request is recorded in Observability: which model was used, latency, cost, token usage and errors, with full request details, and admins can drill into a call to see the exact messages and responses, filtering by time, model, user or team. A Metrics dashboard shows usage, cost and health, and log spans are also recorded. Scheduled agents keep an Activity history of their runs and approvals, and retention is stated.

Sourcedocs.nexos.ai/workspace/settings/observabilityread 2026-09-22

Deployment is self hosted. Official Docker images, a Helm chart and a Terraform module run on the customer's own Postgres and Redis, with one-click deploy into AWS, GCP or Azure and fully air-gapped installation. Enterprise includes air-gapped deployment and multi-region deployment under one license with an admin and worker split, and the customer's data and keys never leave its own infrastructure. The customer can run it in its own cloud, on premises or fully air gapped.

Sourcelitellm.airead 2026-09-22

The platform and most available models are hosted in Europe, and the EU gateway runs Claude, GPT, Gemini and Mistral EU hosted with zero data retention. nexos.ai gives no region list, customer environment option or way to choose a region.

Sourcenexos.ai/eu-gatewayread 2026-09-22

LiteLLM offers no agents of its own for a customer to adopt. The AI Hub on the Enterprise tier lists the models, agents, MCP servers and skills the customer has registered, which is a directory of the customer's own assets, and the Google AI Studio managed agents LiteLLM supports live entirely on Google's side, with LiteLLM as the auth and routing layer. Nothing there is a ready made workflow, template or role specific agent, and a model or tool catalog is integration, not a prebuilt pack.

Sourcedocs.litellm.ai/docs/enterpriseread 2026-09-22

Users can start agents from premade templates, an extensive template library comes with every plan, and a prompt library of assistant instructions is organized by department, with entries such as a product strategy evaluator.

Sourcedocs.nexos.ai/workspace/agents/prompt-libraryread 2026-09-22

The customer chooses the model. One OpenAI-compatible API reaches 140+ providers and 1,800+ models, with models set in the customer's own configuration and swapped without changing application code, the customer's internal, fine-tuned and self-hosted models behind the same key, load balancing across providers, regions and keys, lowest-cost routing, and an Auto Router that sends prompts to model tiers the customer configures.

Sourcelitellm.airead 2026-09-22

Users pick any of 200+ models per chat, Project or agent, or let Smart Routing choose, and organization admins add models with their own guardrails, timeouts, ordered fallback models and a trusted-model badge. The Gateway API exposes the same catalog to the customer's code.

Sourcedocs.nexos.ai/workspace/settings/organization/modelsread 2026-09-22

A Python SDK and a proxy with an OpenAI compatible REST API ship with LiteLLM, and the gateway's own features are callable from outside. The A2A Agent Gateway serves a proxied agent card for each registered agent, pinned to A2A 0.3 or 1.0, and agents are invoked through the A2A SDK or the OpenAI SDK. Memory Management and ingestion have their own REST endpoints (/v1/memory, /v1/rag/ingest) with curl and Python examples.

Sourcedocs.litellm.ai/docs/a2aread 2026-09-22

The Gateway API at api.nexos.ai has a published OpenAPI reference covering Chat, Anthropic native Messages, Responses, audio, images, embeddings and storage endpoints, plus management endpoints for teams, users, agents, assistants, company API keys, budget limits and usage, with Python, Go and .NET code examples. Agents can be managed from outside through it.

Sourcedocs.nexos.ai/gateway-apiread 2026-09-22

Shadow evaluations sample a key's, team's or user's live traffic, send each sampled request through a candidate router configuration without returning that answer to the client, and have an LLM judge compare it blind against the answer the current model served. A job runs up to 30 days and can compare several configurations on the same traffic before anything changes, and after a switch each request carries its routing decision and savings. A change is evaluated against the customer's own traffic, with a judge verdict.

Sourcedocs.litellm.ai/docs/auto_router/evaluateread 2026-09-22

Models can be compared side by side on the same prompt in nexos.ai Chat, the customer's own assistants can be added to Compare Models to test them against other models on performance and output differences, and Agent Builder has a Try Agent window for testing on real scenarios before launch. These let a person inspect and compare an agent's outputs by eye. There are no scored test cases, pass rate, judge verdict, quality gate or optimization loop.

Sourcedocs.nexos.ai/workspace/chatread 2026-09-22

The product is a gateway for models, MCP servers and agents, and there is no stated browser, desktop or remote computer session that LiteLLM runs for an agent. The sandboxes in the Managed Agents Platform announcement, and the swap of OpenAI's Code Interpreter for E2B or OpenSandbox, are code execution, not control of a real interface that an agent drives itself.

Sourcelitellm.airead 2026-09-22

The Web Scraper fetches and analyzes web pages from URLs found in web search results, uploaded files and agent instructions, so agents continue tasks without a person supplying the link. The fetch is headless, and there is no hosted browser, desktop session or remote computer control that an agent drives.

Sourcedocs.nexos.ai/changelog/changelog/latest-updatesread 2026-09-22