Skip to content
NewCybercrime Report 2026 · Eight Chapters · A Post Is Not An Incident → Explore The ChaptersCybercrime Report 2026 → Explore

Odin Research / What is / AI tools and agents

What Is LiteLLM? The LLM Gateway and Why Attackers Target It

One proxy that holds the keys to every AI provider you use: what LiteLLM does, which flaws attackers have used, and how to lock it down.

By
Odin by Brandefense
Updated
21 Nov 2026
Reading time
5 min read
Type
LLM gateway

In short

LiteLLM is an open-source Python library and proxy server, maintained by BerriAI, that gives applications one OpenAI-compatible interface to more than 100 large language model providers. Companies run its proxy as an LLM gateway with virtual keys, budgets and logging. Because it stores provider API keys, attackers now target exposed and unpatched LiteLLM servers.

Key takeaways

What it is
An MIT-licensed Python SDK and self-hosted proxy from BerriAI that routes calls to 100+ LLM providers through one OpenAI-compatible API.
How it works
Apps send requests to the proxy, which checks a virtual key, applies budgets and logging, then forwards the call to the right provider.
Why it matters
The gateway holds provider keys. NVD lists LiteLLM flaws CVE-2026-42208 and CVE-2026-42271 in CISA's Known Exploited Vulnerabilities catalog.
What Odin Research saw
Staged scripts written to pull keys from LiteLLM tables through SQL injection, and a workspace reproducing CVE-2026-42271.
What to do first
Upgrade to 1.83.7 or later, keep the PostgreSQL database private, and store the salt key in a secret manager.

What is LiteLLM?

LiteLLM is an open-source library and proxy server that gives applications one interface to more than 100 large language model (LLM) providers. Developers use its Python SDK to switch models without rewriting code. Companies run its proxy as a self-hosted LLM gateway with virtual keys, budgets, logging and an admin UI.

The project is maintained by BerriAI and published on GitHub under the MIT license. Its documentation names OpenAI, Anthropic, Vertex AI and Amazon Bedrock among supported providers. Requests and responses use the OpenAI format, whichever provider sits behind them.

LiteLLM has two parts:

  • The Python SDK, with a router that adds retries and fallbacks across deployments.
  • The proxy server (LLM gateway), with per-key, team and user budgets, spend tracking, central logging, guardrails, caching and an admin UI.

The proxy is the part that matters for security. It sits between every internal app and every AI provider. It holds the provider keys.

How LiteLLM works

An application sends an OpenAI-style request to the proxy. The documentation's examples serve it on port 4000. The proxy checks the caller's virtual key, applies the key's budget and rate limits, logs the request and forwards it to the configured provider.

In production, the proxy uses a PostgreSQL database. Odin Research's analysis in Monograph 04 notes that request metadata lands in a spend log table and model credentials in a model table. Virtual key hashes sit in a token view.

Two secrets protect this data:

  • The master key (LITELLM_MASTER_KEY) is the proxy admin credential and the admin UI password. The documentation says it must start with sk- and should live in a secret manager.
  • The salt key (LITELLM_SALT_KEY) encrypts the provider API keys stored in the database. The documentation warns not to change it after adding a model, because stored credentials would become unreadable.

LiteLLM can also connect models to tools through the Model Context Protocol (MCP). That feature is behind one of its most serious flaws.

LiteLLM vulnerabilities attackers have used

Two LiteLLM flaws are listed in CISA's Known Exploited Vulnerabilities catalog, according to their NVD records. Both are fixed in version 1.83.7.

  • CVE-2026-42208 (CVSS 3.1 9.8). The API key check placed the caller's Authorization header value straight into a database query. That allowed unauthenticated SQL injection through any LLM API route. It affects 1.81.16 up to 1.83.7. CISA added it to KEV on 8 May 2026.
  • CVE-2026-42271 (CVSS 3.1 8.8). Two MCP test endpoints accepted a full server configuration, including the command to run. Any user with a valid proxy key could start a command on the proxy host. It affects 1.74.2 up to 1.83.7. CISA added it to KEV on 8 June 2026.

The impact is wider than one server. A breached gateway can expose every provider key it holds, plus the prompts and responses in its logs. Treat it like any other non-human identity store.

What Odin Research observed

Monograph 04 looked at 349 AI-stack records on 104 hosts. Setting aside 16 disclosure archives left 333 records on 103 hosts, a mix of staged offensive scripts and dual-use admin scripts.

  • 50 records aimed at LiteLLM databases. They made up 14.3% of the set, across two hosts whose addresses are withheld. Scripts were written to query the spend logs and token view through SQL injection.
  • Injection through the Authorization header. One pair of scripts was written to inject SQL through the HTTP Authorization header to reach Anthropic-style API keys in the model table. NVD describes CVE-2026-42208 as SQL injection through that same header.
  • A CVE-2026-42271 workspace. One host, withheld because it may be legitimate, held a container workspace reproducing the MCP command flaw. Odin did not test it.
  • Encryption limits the damage. LiteLLM encrypts stored provider keys with the salt key. A table dump yields plaintext keys only if payloads were logged unmasked or the salt key sits next to the database.

These were staged files on older hosts. Odin Research did not verify any result on a target. The same report also covers AI offense frameworks such as CyberStrikeAI and HexStrike AI.

How to detect attacks on LiteLLM

  • SQL injection on gateway routes. Alert on requests whose URI or headers mention LiteLLM table names together with a real injection operator, such as UNION SELECT or pg_sleep.
  • Calls to MCP test endpoints. Requests to /mcp-rest/test/connection or /mcp-rest/test/tools/list from ordinary users deserve review, especially on unpatched versions.
  • Shells from the proxy. A LiteLLM or MCP server process that starts a shell or interpreter is a strong signal.
  • Unexpected database clients. Only the gateway hosts should connect to the LiteLLM PostgreSQL database on 5432 or 5433.

Monograph 04 carries Suricata signatures for LiteLLM SQL injection and external MCP access. It adds a Sentinel KQL hunt for unexpected database clients, a Sigma rule for shells started by LiteLLM or MCP processes and a YARA rule for key harvesters. All are in the Defense Kit.

How to secure LiteLLM

  1. Patch. Upgrade to 1.83.7 or later to fix both known exploited flaws.
  2. Keep the database private. Bind PostgreSQL to a private network. The documentation recommends verified TLS for database connections.
  3. Separate the secrets. Store the master key and salt key in a secret manager, away from the database host.
  4. Harden the runtime. The documentation recommends LITELLM_MODE="PRODUCTION", which stops it loading credentials from a local .env file, turning off debug logging and keeping prompt storage off for sensitive work.
  5. Limit who can reach it. Do not expose the proxy or admin UI to the internet. Give each app its own virtual key with a budget. See AI agent identity and access.
  6. Rotate after an incident. If you suspect exposure, rotate every provider key the gateway held. See how to respond to leaked credentials.

LiteLLM compared with other AI-stack targets

TargetWhat it holdsStaged attack path in Monograph 04
LiteLLM gatewayProvider keys, spend logs, virtual keysSQL injection to key tables; MCP test endpoint flaw
MCP serversAccess to internal tools and dataOAuth token replay and SSE session probing
OpenClaw workspacesAgent config files with API keysSearches for exposed config files, then key tests
Apps with a custom model base URLTrust in model outputRogue OpenAI-compatible servers that return template injection

Frequently asked questions about LiteLLM

What is LiteLLM?

LiteLLM is an open-source Python SDK and proxy server from BerriAI. It gives applications one OpenAI-compatible interface to more than 100 LLM providers, with virtual keys, budgets, logging and an admin UI.

Is LiteLLM safe to use?

Yes, when patched and deployed privately. Versions before 1.83.7 carry two flaws in CISA's Known Exploited Vulnerabilities catalog. Keep the proxy and its database off the internet and store its secrets in a secret manager.

What is CVE-2026-42271?

A LiteLLM flaw that let any user with a valid proxy key run commands on the proxy host through two MCP test endpoints. NVD lists versions 1.74.2 up to 1.83.7 as affected and rates it CVSS 3.1 8.8.

What is CVE-2026-42208?

A LiteLLM SQL injection flaw in the API key check, reachable without authentication through the Authorization header. NVD rates it CVSS 3.1 9.8. It affects 1.81.16 up to 1.83.7.

What port does LiteLLM use?

The LiteLLM documentation examples serve the proxy on port 4000. Its PostgreSQL database usually listens on 5432, which should never be reachable from the internet.

Who maintains LiteLLM?

BerriAI maintains LiteLLM. The project is hosted on GitHub under BerriAI/litellm and released under the MIT license.

Can attackers steal API keys from LiteLLM?

They can try. Odin Research found staged scripts written to pull keys from LiteLLM tables. LiteLLM encrypts stored provider keys with a salt key, so a dump yields plaintext only if logs were unmasked or the salt key was exposed.

How do I detect attacks on LiteLLM?

Watch for SQL injection operators on gateway routes, calls to /mcp-rest/test/ endpoints, shells started by the proxy and unexpected database clients. Odin Research publishes matching rules in its Defense Kit.

Sources

  1. LiteLLM documentation, Getting started. docs.litellm.ai/docs/ (checked 21 November 2026)
  2. LiteLLM documentation, Production deployment. docs.litellm.ai/docs/proxy/prod (checked 21 November 2026)
  3. NVD, CVE-2026-42208. nvd.nist.gov/vuln/detail/CVE-2026-42208 (checked 21 November 2026)
  4. NVD, CVE-2026-42271. nvd.nist.gov/vuln/detail/CVE-2026-42271 (checked 21 November 2026)

Related tools and terms

All Odin Research explanations · Brandefense glossary

Odin Research

Get the rules, not just the definition

Every Odin Research report ships with verified indicators and detection rules you can load today.

Open the Defense Kit