Configuration

Full reference for crabllm.toml — providers, virtual keys, aliases, pricing, extensions, and storage.

CrabLLM is configured via a TOML file, passed with --config:

crabllm --config crabllm.toml

The --bind flag overrides the listen address.

Environment variables

Strings containing ${VAR} are expanded from environment variables at startup. Unknown variables expand to empty string. Use this for secrets:

api_key = "${OPENAI_API_KEY}"

Top-level fields

Field	Type	Default	Description
`listen`	string	required	Address to bind, e.g. `"0.0.0.0:8080"`
`shutdown_timeout`	integer	`30`	Graceful shutdown timeout in seconds

Providers

Each provider is a named entry under [providers]:

[providers.my_openai]
kind = "openai"
api_key = "${OPENAI_API_KEY}"
models = ["gpt-4o", "gpt-4o-mini"]

Field	Type	Default	Description
`kind`	string	required	Provider type (see Providers)
`api_key`	string	`""`	API key for authentication
`base_url`	string	per-kind	Base URL override
`models`	list	`[]`	Model names this provider serves
`weight`	integer	`1`	Routing weight for load balancing
`max_retries`	integer	`2`	Max retries on transient errors
`timeout`	integer	`30`	Per-request timeout in seconds
`api_version`	string	—	API version (Azure only)
`region`	string	—	AWS region (Bedrock only)
`access_key`	string	—	AWS access key (Bedrock only)
`secret_key`	string	—	AWS secret key (Bedrock only)

Virtual keys

[[keys]]
name = "team-a"
key = "sk-team-a-secret"
models = ["gpt-4o", "claude-sonnet-4-20250514"]

[[keys]]
name = "admin"
key = "sk-admin-secret"
models = ["*"]

Field	Type	Description
`name`	string	Human-readable key name (used in usage tracking)
`key`	string	The bearer token clients send
`models`	list	Allowed models. `["*"]` means all

When no keys are configured, authentication is disabled.

Aliases

[aliases]
gpt4 = "gpt-4o"
claude = "claude-sonnet-4-20250514"

Maps friendly model names to canonical names. Single-hop lookup.

Pricing

[pricing.gpt-4o]
prompt_cost_per_million = 2.50
completion_cost_per_million = 10.00

[pricing.claude-sonnet-4-20250514]
prompt_cost_per_million = 3.00
completion_cost_per_million = 15.00

Per-model token pricing in USD. Used by the budget extension for spend tracking.

Extensions

[extensions.cache]
ttl = 3600

[extensions.rate_limit]
rpm = 60

[extensions.usage]

[extensions.budget]
default_limit = 10000000

[extensions.logging]
level = "info"

See Extensions for details on each.

Storage

[storage]
kind = "memory"

Kind	Feature flag	`path` field
`memory`	none (default)	not used
`sqlite`	`storage-sqlite`	file path, e.g. `"crabllm.db"`
`redis`	`storage-redis`	URL, e.g. `"redis://127.0.0.1:6379"`

See Storage for details.

Full example

listen = "0.0.0.0:8080"
shutdown_timeout = 30

[providers.openai]
kind = "openai"
api_key = "${OPENAI_API_KEY}"
models = ["gpt-4o", "gpt-4o-mini"]
weight = 2
max_retries = 2
timeout = 30

[providers.anthropic]
kind = "anthropic"
api_key = "${ANTHROPIC_API_KEY}"
models = ["claude-sonnet-4-20250514"]

[providers.ollama]
kind = "ollama"
models = ["llama3.2"]

[aliases]
gpt4 = "gpt-4o"
claude = "claude-sonnet-4-20250514"

[[keys]]
name = "default"
key = "${CRABLLM_API_KEY}"
models = ["*"]

[pricing.gpt-4o]
prompt_cost_per_million = 2.50
completion_cost_per_million = 10.00

[extensions.rate_limit]
rpm = 100

[extensions.usage]

[extensions.logging]
level = "info"

[storage]
kind = "sqlite"
path = "crabllm.db"

Configuration

On this page