In an analysis highlighting the persistent friction between developer agility and enterprise cloud security, cloud security firm Wiz Research revealed that nearly 10% of publicly exposed instances of LiteLLM—a popular open-source proxy gateway for managing and load-balancing large language model (LLM) APIs—were operating with default master credentials or without authentication enabled.

LiteLLM is widely deployed by engineering teams to provide a unified OpenAI-compatible endpoint that routes developer requests across diverse model providers, including OpenAI, Anthropic Claude, Google Gemini, and AWS Bedrock. Because the gateway centralizes billing, rate limiting, and credential management, compromising the gateway provides an adversary with unrestricted access to an organization's downstream AI infrastructure.

The sk-1234 Exposure Vector

LiteLLM's official quick-start documentation historically provided an example configuration demonstrating proxy deployment with a sample master key: master_key: "sk-1234". In production deployments where developers copied the sample configuration file directly or failed to override the variable via secure environment vaults, the proxy remained open to external takeover.

Wiz identified that an attacker possessing this master key could query the LiteLLM admin endpoints (/key/generate, /model/info, and /spend/keys) to achieve complete platform compromise:

  • Plaintext Extraction of Upstream API Keys: Read API tokens for OpenAI, Anthropic, Cohere, and cloud provider LLM accounts stored within the proxy database.
  • Cloud IAM Credential Theft: On containerized deployments hosted on AWS ECS, EKS, or Azure Container Instances, attackers could leverage the proxy's administrative tools to query the local cloud Instance Metadata Service (IMDS) and extract temporary IAM assume-role credentials.
  • Prompt Injection and Data Interception: Inspect in-flight enterprise prompt traffic, sensitive source code snippets, and customer queries routed through the gateway.

Immediate Hardening Actions for AI Infrastructure

Check Vulnerable State Hardened Configuration
Master Key LITELLM_MASTER_KEY="sk-1234" or unset Generate 64-char cryptographically secure secret via secret manager
Network Ingress 0.0.0.0:4000 exposed to WAN Internal VPC ingress only; enforce mutual TLS or reverse proxy auth
IMDS Access IMDSv1 enabled on gateway host Enforce IMDSv2 with hop limit = 1 to block container extraction