In short
LiteLLM is an open-source Python library and proxy server, maintained by BerriAI, that gives applications one OpenAI-compatible interface to more than 100 large language model providers. Companies run its proxy as an LLM gateway with virtual keys, budgets and logging. Because it stores provider API keys, attackers now target exposed and unpatched LiteLLM servers.
Key takeaways
- What it is
- An MIT-licensed Python SDK and self-hosted proxy from BerriAI that routes calls to 100+ LLM providers through one OpenAI-compatible API.
- How it works
- Apps send requests to the proxy, which checks a virtual key, applies budgets and logging, then forwards the call to the right provider.
- Why it matters
- The gateway holds provider keys. NVD lists LiteLLM flaws CVE-2026-42208 and CVE-2026-42271 in CISA's Known Exploited Vulnerabilities catalog.
- What Odin Research saw
- Staged scripts written to pull keys from LiteLLM tables through SQL injection, and a workspace reproducing CVE-2026-42271.
- What to do first
- Upgrade to 1.83.7 or later, keep the PostgreSQL database private, and store the salt key in a secret manager.
What is LiteLLM?
LiteLLM is an open-source library and proxy server that gives applications one interface to more than 100 large language model (LLM) providers. Developers use its Python SDK to switch models without rewriting code. Companies run its proxy as a self-hosted LLM gateway with virtual keys, budgets, logging and an admin UI.
The project is maintained by BerriAI and published on GitHub under the MIT license. Its documentation names OpenAI, Anthropic, Vertex AI and Amazon Bedrock among supported providers. Requests and responses use the OpenAI format, whichever provider sits behind them.
LiteLLM has two parts:
- The Python SDK, with a router that adds retries and fallbacks across deployments.
- The proxy server (LLM gateway), with per-key, team and user budgets, spend tracking, central logging, guardrails, caching and an admin UI.
The proxy is the part that matters for security. It sits between every internal app and every AI provider. It holds the provider keys.
How LiteLLM works
An application sends an OpenAI-style request to the proxy. The documentation's examples serve it on port 4000. The proxy checks the caller's virtual key, applies the key's budget and rate limits, logs the request and forwards it to the configured provider.
In production, the proxy uses a PostgreSQL database. Odin Research's analysis in Monograph 04 notes that request metadata lands in a spend log table and model credentials in a model table. Virtual key hashes sit in a token view.
Two secrets protect this data:
- The master key (
LITELLM_MASTER_KEY) is the proxy admin credential and the admin UI password. The documentation says it must start withsk-and should live in a secret manager. - The salt key (
LITELLM_SALT_KEY) encrypts the provider API keys stored in the database. The documentation warns not to change it after adding a model, because stored credentials would become unreadable.
LiteLLM can also connect models to tools through the Model Context Protocol (MCP). That feature is behind one of its most serious flaws.
LiteLLM vulnerabilities attackers have used
Two LiteLLM flaws are listed in CISA's Known Exploited Vulnerabilities catalog, according to their NVD records. Both are fixed in version 1.83.7.
- CVE-2026-42208 (CVSS 3.1 9.8). The API key check placed the caller's Authorization header value straight into a database query. That allowed unauthenticated SQL injection through any LLM API route. It affects 1.81.16 up to 1.83.7. CISA added it to KEV on 8 May 2026.
- CVE-2026-42271 (CVSS 3.1 8.8). Two MCP test endpoints accepted a full server configuration, including the command to run. Any user with a valid proxy key could start a command on the proxy host. It affects 1.74.2 up to 1.83.7. CISA added it to KEV on 8 June 2026.
The impact is wider than one server. A breached gateway can expose every provider key it holds, plus the prompts and responses in its logs. Treat it like any other non-human identity store.
What Odin Research observed
Monograph 04 looked at 349 AI-stack records on 104 hosts. Setting aside 16 disclosure archives left 333 records on 103 hosts, a mix of staged offensive scripts and dual-use admin scripts.
- 50 records aimed at LiteLLM databases. They made up 14.3% of the set, across two hosts whose addresses are withheld. Scripts were written to query the spend logs and token view through SQL injection.
- Injection through the Authorization header. One pair of scripts was written to inject SQL through the HTTP Authorization header to reach Anthropic-style API keys in the model table. NVD describes CVE-2026-42208 as SQL injection through that same header.
- A CVE-2026-42271 workspace. One host, withheld because it may be legitimate, held a container workspace reproducing the MCP command flaw. Odin did not test it.
- Encryption limits the damage. LiteLLM encrypts stored provider keys with the salt key. A table dump yields plaintext keys only if payloads were logged unmasked or the salt key sits next to the database.
These were staged files on older hosts. Odin Research did not verify any result on a target. The same report also covers AI offense frameworks such as CyberStrikeAI and HexStrike AI.
How to detect attacks on LiteLLM
- SQL injection on gateway routes. Alert on requests whose URI or headers mention LiteLLM table names together with a real injection operator, such as
UNION SELECTorpg_sleep. - Calls to MCP test endpoints. Requests to
/mcp-rest/test/connectionor/mcp-rest/test/tools/listfrom ordinary users deserve review, especially on unpatched versions. - Shells from the proxy. A LiteLLM or MCP server process that starts a shell or interpreter is a strong signal.
- Unexpected database clients. Only the gateway hosts should connect to the LiteLLM PostgreSQL database on 5432 or 5433.
Monograph 04 carries Suricata signatures for LiteLLM SQL injection and external MCP access. It adds a Sentinel KQL hunt for unexpected database clients, a Sigma rule for shells started by LiteLLM or MCP processes and a YARA rule for key harvesters. All are in the Defense Kit.
How to secure LiteLLM
- Patch. Upgrade to 1.83.7 or later to fix both known exploited flaws.
- Keep the database private. Bind PostgreSQL to a private network. The documentation recommends verified TLS for database connections.
- Separate the secrets. Store the master key and salt key in a secret manager, away from the database host.
- Harden the runtime. The documentation recommends
LITELLM_MODE="PRODUCTION", which stops it loading credentials from a local.envfile, turning off debug logging and keeping prompt storage off for sensitive work. - Limit who can reach it. Do not expose the proxy or admin UI to the internet. Give each app its own virtual key with a budget. See AI agent identity and access.
- Rotate after an incident. If you suspect exposure, rotate every provider key the gateway held. See how to respond to leaked credentials.
LiteLLM compared with other AI-stack targets
| Target | What it holds | Staged attack path in Monograph 04 |
|---|---|---|
| LiteLLM gateway | Provider keys, spend logs, virtual keys | SQL injection to key tables; MCP test endpoint flaw |
| MCP servers | Access to internal tools and data | OAuth token replay and SSE session probing |
| OpenClaw workspaces | Agent config files with API keys | Searches for exposed config files, then key tests |
| Apps with a custom model base URL | Trust in model output | Rogue OpenAI-compatible servers that return template injection |
Frequently asked questions about LiteLLM
What is LiteLLM?
LiteLLM is an open-source Python SDK and proxy server from BerriAI. It gives applications one OpenAI-compatible interface to more than 100 LLM providers, with virtual keys, budgets, logging and an admin UI.
Is LiteLLM safe to use?
Yes, when patched and deployed privately. Versions before 1.83.7 carry two flaws in CISA's Known Exploited Vulnerabilities catalog. Keep the proxy and its database off the internet and store its secrets in a secret manager.
What is CVE-2026-42271?
A LiteLLM flaw that let any user with a valid proxy key run commands on the proxy host through two MCP test endpoints. NVD lists versions 1.74.2 up to 1.83.7 as affected and rates it CVSS 3.1 8.8.
What is CVE-2026-42208?
A LiteLLM SQL injection flaw in the API key check, reachable without authentication through the Authorization header. NVD rates it CVSS 3.1 9.8. It affects 1.81.16 up to 1.83.7.
What port does LiteLLM use?
The LiteLLM documentation examples serve the proxy on port 4000. Its PostgreSQL database usually listens on 5432, which should never be reachable from the internet.
Who maintains LiteLLM?
BerriAI maintains LiteLLM. The project is hosted on GitHub under BerriAI/litellm and released under the MIT license.
Can attackers steal API keys from LiteLLM?
They can try. Odin Research found staged scripts written to pull keys from LiteLLM tables. LiteLLM encrypts stored provider keys with a salt key, so a dump yields plaintext only if logs were unmasked or the salt key was exposed.
How do I detect attacks on LiteLLM?
Watch for SQL injection operators on gateway routes, calls to /mcp-rest/test/ endpoints, shells started by the proxy and unexpected database clients. Odin Research publishes matching rules in its Defense Kit.
Sources
- LiteLLM documentation, Getting started. docs.litellm.ai/docs/ (checked 21 November 2026)
- LiteLLM documentation, Production deployment. docs.litellm.ai/docs/proxy/prod (checked 21 November 2026)
- NVD, CVE-2026-42208. nvd.nist.gov/vuln/detail/CVE-2026-42208 (checked 21 November 2026)
- NVD, CVE-2026-42271. nvd.nist.gov/vuln/detail/CVE-2026-42271 (checked 21 November 2026)
Related tools and terms
What is CyberStrikeAI?
CyberStrikeAI is a public AI-driven offensive security platform. It spreads its operator interface, API and agent listener across several ports on one host and exposes tools through MCP.
AI offensive toolWhat is PentAGI?
PentAGI is an open-source autonomous penetration testing system that coordinates AI agents and security tools in containers.
AI offensive toolWhat is HexStrike AI?
HexStrikeAI is a public project that exposes many security tools to AI agents through the Model Context Protocol (MCP).
AI security toolWhat is DeepAudit?
DeepAudit is an open-source AI code auditing tool. Application security teams use it to find flaws in source code.
AI security toolWhat is AutoCVE?
AutoCVE is an AI-assisted tool that turns new vulnerability advisories into HTTP checks.
AI security toolWhat is Cairn?
Cairn is a reconnaissance tool that keeps an asset graph built from DNS zones, TLS certificates and HTTP banners.
Odin Research
Get the rules, not just the definition
Every Odin Research report ships with verified indicators and detection rules you can load today.