LiteLLM in Docker: The Salt Key You Can Never Change

Contents

Figure 1: LiteLLM in Docker: The Salt Key You Can Never Change — one command to start, a dozen decisions to keep it running

Getting the LiteLLM proxy running in Docker takes one command. Getting it running safely takes about a dozen decisions, and the docs scatter them across three pages. This guide pulls them together.

One honest limit first: Docker's client and Compose are installed here, but there is no daemon, so I never started a container or pulled an image. I did run the real docker compose config to validate my compose file. Every production rule below comes from LiteLLM's docs unless I mark it as mine.

Start With the Quick Start

LiteLLM's quick start offers two paths [1].

The Compose path downloads docker-compose.quickstart.yml, writes a .env containing both keys generated with openssl rand -hex 32, and starts it with docker compose -f docker-compose.quickstart.yml up -d. You get two services on port 4000: the gateway and a Postgres database for models, keys and spend logs [1].

The plain docker run path uses the image docker.litellm.ai/berriai/litellm:latest, mounts your config to /app/config.yaml, passes --config /app/config.yaml and exposes -p 4000:4000 [1].

The plain path has a catch. Without a database, max_budget is not enforced. **The budgets page is blunt about it: litellm_settings.max_budget "fails open" because global spend is only loaded when a database client exists, so "requests keep being served past the limit," and a warning appears once at startup [5]. Virtual keys cannot be resolved either, and fail with No connected db. [5]

If you need budgets or virtual keys, you need Postgres.

Keys: Three Things You Cannot Undo

1. LITELLM_MASTER_KEY. The proxy "refuses to start without it," and it doubles as the Admin UI password by default, so the docs say to treat it "like a root password" [1].

One correction to a common assumption: the sk- prefix is "a convention, not a requirement" [1]. Nothing breaks if your master key starts differently; what matters is that it is long, random and out of source control.

2. LITELLM_SALT_KEY. It encrypts provider API keys stored in the database. It "has no in-place rotation," so changing it "makes every stored credential unreadable until you re-enter it" [1]. Set it once.

3. Provider keys. Pass them as environment variables and reference them as os.environ/... in config [1].

The quick start generates both keys with openssl rand -hex 32, and its compose file refuses to start without them [1]. The Deploy page prefers your cloud's secret manager over plain Kubernetes Secrets [3].

Pin the Image Tag

Both the Production and Deploy pages say the same thing: pin a version, do not use latest. The Deploy page's comment is blunt: "pin a version tag rather than latest or a moving tag, so rollbacks are deterministic" [3].

That "or a moving tag" is the part people miss. main-stable is a moving tag, and the Production page's own Docker example uses ghcr.io/berriai/litellm:main-stable, while the Deploy page's Kubernetes manifest uses :latest [2][3]. The pages do not follow their own rule everywhere, so pin an explicit version anyway.

Official images live at ghcr.io/berriai and mirror to docker.litellm.ai/berriai. The ghcr.io image bundles the Prisma toolchain that Postgres needs [3].

There is more than one variant, which the deployment pages never mention [4]:

Image	What it is
`ghcr.io/berriai/litellm`	Core proxy, including the Prisma toolchain
`ghcr.io/berriai/litellm-non_root`	Non-root variant
`ghcr.io/berriai/litellm-spend_logs`	Spend-logs sidecar
`ghcr.io/berriai/litellm-database`	Legacy alias, kept for existing deployments

All variants are signed with cosign from v1.83.0, and the recommended production pattern is a digest pin — image: ghcr.io/berriai/litellm@sha256:<digest> — which survives tag mutation entirely [4].

The Production Settings That Matter

From the Production page [2]:

Setting	Value	Why
`general_settings.trusted_proxy_ranges`	Your load balancer's CIDRs, or `[]` if clients connect directly	If unset, the proxy warns at startup and only the weaker per-username sign-in limit applies
`LITELLM_MODE="PRODUCTION"`	env var	Disables `load_dotenv()`, which would otherwise load credentials from a local `.env`
`set_verbose: False`, `json_logs: true`, `LITELLM_LOG="ERROR"`	litellm settings	Turns off debug logging and silences FastAPI's per-request info logs
`general_settings.disable_error_logs: True`	`True`	Exceptions are written to the database by default and bloat the spend logs table under sustained provider errors
`litellm_settings.request_timeout`	Example `600`; default `6000`	Fails hanging requests instead of holding connections open
`general_settings.proxy_batch_write_at`	`60`	Batches spend writes instead of one per request; above roughly 1,000 rps also use the Redis transaction buffer
`general_settings.database_connection_pool_limit`	Default `10`	Size as MAX_DB_CONNECTIONS / (instances × workers)

The connection math is worth writing down. 3 instances × 4 workers × 10 connections = 120 connections against your database [2]. An autoscaler that owns the replica count multiplies the number you thought you set.

Size and Scale

The sizing is per worker: 1 vCPU and 4Gi of memory, as both requests and limits. The page's YAML comments say 1*num_workers and 4*num_workers, and it spells out the arithmetic: a container running 8 workers wants 8 vCPU and 32Gi [2].

On a single VM, set NUM_WORKERS to the vCPU count. On Kubernetes, use --num_workers 1 and scale pods horizontally. The Production page recommends scaling on CPU at about 60% "because a replica is not useful the moment it is created," and notes the Helm startup probe allows up to 300 seconds for a first readiness check, while both charts ship higher defaults, 80 and 70 [2].

Once you run more than one instance, add Redis 7.0 or newer [2][3].

Health Checks

The Deploy page uses /health/liveliness for the Kubernetes liveness probe and /health/readiness for readiness, both on port 4000 with initialDelaySeconds: 120 and periodSeconds: 15 [3]. Load balancers should check /health/readiness [3].

The Production page adds the part that trips people up: /health/readiness "always returns 200 OK" so pods stay operational during a database outage [2]. A green check does not prove the database is up.

What actually degrades during an outage is auth: virtual keys already in the in-memory cache keep working until user_api_key_cache_ttl expires, which defaults to 60 seconds, while the master key and config-file models keep working throughout [2]. That is the difference between "the proxy is up" and "the proxy is useful."

I did not add a container-level HEALTHCHECK to my compose file, because the pages do not say whether the image ships curl or another tool for it.

Database Migrations

By default, LiteLLM runs prisma migrate deploy on startup. The Production page says to set DISABLE_SCHEMA_UPDATE=true on serving pods and run migrations through a separate job, which the Helm charts do automatically [2][3].

The pages do not show how to do that with plain Docker. For a single Compose stack, leaving the default means the proxy migrates its own database on startup. That is simple, but think twice before running several replicas that all try to migrate at once. That last point is my inference, not a docs statement.

The Production page documents one related multi-replica trap you can fix today: background jobs. Budget resets, spend-log pruning and usage exports are jobs that should happen once, and each worker registers them [2]. LITELLM_JOB_ROLE controls this [2]:

`LITELLM_JOB_ROLE`	What the process registers
unset, or `all`	Every job (the default)
`worker`	Every job, including the single-owner ones
`serving`	No single-owner job

For a multi-pod deployment without Redis, setting LITELLM_JOB_ROLE=serving on the traffic-serving replicas is the only way to get single execution [2]. That is a one-line Compose change and it belongs in production.

For TLS to Postgres, use sslmode=verify-full&sslrootcert=/certs/global-bundle.pem in DATABASE_URL. LiteLLM rewrites verify-full and sslrootcert into the params its Postgres driver understands (sslmode=require, sslcert=<bundle>, sslaccept=strict) on DATABASE_URL, DIRECT_URL and related variables [2]. SSL_CERT_FILE does not affect the database driver.

A Compose File That Follows These Rules

My file pins the tag through a required variable, mounts config read-only, keeps secrets as required environment variables, uses LITELLM_MODE=PRODUCTION, binds port 4000 to 127.0.0.1 behind your load balancer, and gives Postgres a volume and a pg_isready healthcheck. The LiteLLM service waits for the database to be healthy.

services:
  litellm:
    image: ghcr.io/berriai/litellm:${LITELLM_VERSION:?pin an explicit version}
    command: ["--config", "/app/config.yaml", "--num_workers", "1"]
    ports:
      - "127.0.0.1:4000:4000"
    volumes:
      - ./config.yaml:/app/config.yaml:ro
    environment:
      LITELLM_MODE: PRODUCTION
      LITELLM_LOG: ERROR
      LITELLM_JOB_ROLE: serving
      LITELLM_MASTER_KEY: ${LITELLM_MASTER_KEY:?set a random master key}
      LITELLM_SALT_KEY: ${LITELLM_SALT_KEY:?set once, never change}
      DATABASE_URL: ${DATABASE_URL:?postgres required for keys and budgets}
    depends_on:
      db:
        condition: service_healthy

  db:
    image: postgres:16
    environment:
      POSTGRES_PASSWORD: ${POSTGRES_PASSWORD:?set a database password}
      POSTGRES_DB: litellm
    volumes:
      - pgdata:/var/lib/postgresql/data
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U postgres -d litellm"]
      interval: 5s
      timeout: 5s
      retries: 10
    # not published; only the gateway needs to reach it

volumes:
  pgdata:

The required-variable pattern is the cheapest safety win: with no secrets set, docker compose config fails before anything starts.

My checker resolves the file with the real tool and flags problems. A deliberately bad file produced 11 findings, including latest, a placeholder master key, no DATABASE_URL, no Postgres volume and port 4000 open on all interfaces. The good file produced none. Some findings are the docs' rules and some are my opinions, and the script labels which is which.

Pre-Launch Checklist

Pin the image tag or digest, and keep a rollback tag.
Set the master key and salt key, store them in a secret manager and never rotate the salt.
Run Postgres with a volume and TLS.
Set trusted_proxy_ranges and put a load balancer in front.
Point probes at /health/readiness (and /health/liveliness on Kubernetes).
Turn off verbose logging, and turn on JSON logs.
Set LITELLM_JOB_ROLE=serving and add Redis before running a second instance.
Test one request with a virtual key, not the master key.

Common Pitfalls

Running :latest because the quick start does. It is a demo default, not a production one.
Treating main-stable as a pinned tag. The Deploy page calls it a moving tag.
Rotating the salt key "for hygiene." Every stored provider credential stops decrypting.
Assuming master keys must start with sk-. That is a convention.
Reading a 200 from /health/readiness as "database is fine." It returns 200 during an outage by design.
Leaving the default connection pool at 10 per worker. Multiply by replicas before you are surprised.
Running several replicas with the default LITELLM_JOB_ROLE. Background jobs run once per worker.
Exposing port 4000 on 0.0.0.0 on a VM. Bind it to loopback and let the load balancer in.
Expecting max_budget to hold without Postgres. It fails open.

Best Book to Read

Cover	Book	Description	Get it
	Docker Deep Diveby Nigel Poulton	images, volumes, health checks and Compose explained from the layer up, which is the background this checklist assumes.	View on Amazon

As an Amazon Associate I earn from qualifying purchases. Links on this page are affiliate links, and I may earn a commission at no extra cost to you.

Wrapping Up

A production LiteLLM container is mostly about the things around it: a pinned image, two keys you cannot lose, a real database and a few settings that keep logs and connections sane. The salt key is the one you can never change, and the readiness probe is the one that will lie to you.

Start from the compose pattern, run the checker and boot it on a machine with a daemon. Then send one request with a virtual key before you call it done.

If the gateway is new to you, the LiteLLM proxy guide covers the quickstart and the virtual keys and budgets guide covers what Postgres buys you. For traffic decisions once it is running, see the load balancing guide and the error triage guide.

Frequently Asked Questions

Can I run LiteLLM in Docker without a database?

You can start it, but budgets and virtual keys stop working. LiteLLM's budgets page says every budget is enforced against spend read from the database, so litellm_settings.max_budget fails open there: the global budget check is skipped and requests keep being served past the limit. A warning is logged once at startup, and virtual keys fail with No connected db. Run with Postgres if a budget is part of how you bound spend.

Must the LiteLLM master key start with sk-?

No. The quickstart page says the sk- prefix in the generated value is a convention, not a requirement, so a master key without it still works. What matters is that the key is long, random, kept out of source control and rotated if it leaks, because it authorizes every management API call and by default doubles as the Admin UI password.

What happens if I change LITELLM_SALT_KEY after adding models?

Every provider credential stored in the database becomes unreadable until you re-enter it. The salt key encrypts and decrypts those stored credentials and has no in-place rotation, which is why the docs say to set it once, keep the generated value and treat it as permanent. Rotating it is a data migration, not a config change.

Which image tag should I use for LiteLLM in production?

An explicit version tag, or a digest. The deploy page says to pin a version tag rather than latest or a moving tag so rollbacks are deterministic, and the image security page recommends pinning by digest with image: ghcr.io/berriai/litellm@sha256:. Official images are published to ghcr.io/berriai and mirrored at docker.litellm.ai/berriai, with a non-root variant and a spend-logs sidecar signed with cosign since v1.83.0.

Why does LiteLLM readiness return 200 when the database is down?

Because the Production page says /health/readiness always returns 200 so pods remain operational during a database outage. That means a green readiness check does not prove the database is reachable. Virtual keys already in the in-memory auth cache keep authenticating until user_api_key_cache_ttl expires, which defaults to 60 seconds, while the master key and config-file models keep working throughout.

How many database connections does a LiteLLM deployment use?

Multiply it out: database_connection_pool_limit defaults to 10 per worker process, so 3 instances times 4 workers times 10 connections is 120 connections against Postgres. The Production page recommends sizing the limit as MAX_DB_CONNECTIONS / (instances × workers) so your replicas cannot exhaust the database.

Sources

[1] LiteLLM, Docker quickstart — the docker-compose.quickstart.yml download, the .env built with openssl rand -hex 32, the docker compose -f ... up -d command, the gateway plus Postgres on port 4000, the docker.litellm.ai/berriai/litellm:latest run command with --config /app/config.yaml and -p 4000:4000, the master key as root credential that doubles as the Admin UI password, the note that sk- is a convention, the salt key's lack of in-place rotation, and os.environ/ for provider keys: docs.litellm.ai/docs/proxy/docker_quick_start
[2] LiteLLM, Production — trusted_proxy_ranges and its startup warning, LITELLM_MODE="PRODUCTION" disabling load_dotenv(), the logging trio, disable_error_logs, request_timeout with a 600 example against a 6000 default, proxy_batch_write_at: 60 with the Redis transaction buffer above ~1000 rps, database_connection_pool_limit defaulting to 10 with the 3×4×10=120 example, per-worker 1 vCPU/4Gi sizing, the 60% CPU target with 300-second startup probes and 80/70 chart defaults, Redis 7.0+ for multiple instances, /health/readiness always returning 200 during an outage with user_api_key_cache_ttl at 60 seconds, DISABLE_SCHEMA_UPDATE=true on serving pods, LITELLM_JOB_ROLE, and the sslmode=verify-full rewriting to driver params: docs.litellm.ai/docs/proxy/prod
[3] LiteLLM, Deploy — image registries at ghcr.io/berriai mirrored to docker.litellm.ai/berriai, the ghcr.io image bundling the Prisma toolchain, "pin a version tag rather than latest or a moving tag", the note that Redis is required once you run more than one instance and that the migrations job runs once per upgrade with DISABLE_SCHEMA_UPDATE=true on proxies, the liveness and readiness probes with initialDelaySeconds: 120 and periodSeconds: 15, the :latest and main-stable images in the page's own manifests, and preferring a cloud secret manager: docs.litellm.ai/docs/proxy/deploy
[4] LiteLLM, Docker image security — images signed with cosign from v1.83.0, the litellm, litellm-non_root, litellm-spend_logs and legacy litellm-database variants, and the recommendation to pin by digest with image: ghcr.io/berriai/litellm@sha256:<digest>: docs.litellm.ai/docs/proxy/docker_image_security
[5] LiteLLM, Budgets — budgets are enforced against spend read from the database, litellm_settings.max_budget failing open with requests served past the limit, the single startup warning when a budget is set with no database connected, and virtual keys failing with No connected db.: docs.litellm.ai/docs/proxy/users