LiteLLM

LiteLLM, built by BerriAI, is an open source AI gateway that gives teams a single OpenAI compatible interface to call more than a hundred large language model providers, including OpenAI, Anthropic, Gemini, Azure, and AWS Bedrock. It ships in two forms that share a name: a lightweight Python SDK that runs inside application code as a translation layer, and a proxy server, the gateway, deployed as a centralized service for a team or organization. The project is MIT licensed, has more than 40,000 GitHub stars, and has become the default open source choice for teams that want to own the gateway layer themselves.

The gateway removes the friction of juggling provider specific SDKs, auth patterns, and request formats. Point existing OpenAI SDK code at the proxy and it translates, routes, retries, and bills across providers with no application changes. Production features include virtual keys, per key and per team budget tracking, rate limits, load balancing, automatic fallback across deployments, spend dashboards for chargeback, prompt management, and logging that exports to tools like Langfuse, Datadog, and OpenTelemetry. A database such as Postgres and a cache such as Redis back the proxy for key management, budgets, and analytics.

Because the core is open source and self hosted, LiteLLM is a common fit for strict data residency, air gapped environments, and regulated workloads where third party SaaS is not an option, and it adds no markup on provider calls. A commercial LiteLLM Enterprise license, sold through a sales process and referenced publicly around $250 a month for the entry tier, unlocks the governance features a security team expects: single sign on, granular role based access control, audit logs, JWT authorization, guardrails, and dedicated support, all still self hosted. The practical tradeoff is ownership: the software is free, but the operating burden, from database operations to patching and on call, sits with the adopting team.