2. Models & Providers

The CLI lets you switch model, provider, and sampling temperature without restarting. Three commands cover everything: /model, /provider, /temperature.

The provider concept

A provider is a (API_KEY, BASE_URL, MODEL) triple — one set of credentials pointing at one endpoint with one default model. The endpoint speaks OpenAI Chat Completions unless the block says otherwise with a fourth, optional variable, XXXX_API_FORMAT (0.5.0) — see Wire protocol below. Provider names are arbitrary; they come from your .env file.

The convention is XXXX_API_KEY / XXXX_BASE_URL / XXXX_MODEL, where XXXX is the provider name in upper case:

# .env

# Default provider (legacy: just OPENAI_*)
OPENAI_API_KEY=sk-...
OPENAI_BASE_URL=https://api.openai.com/v1
OPENAI_MODEL=gpt-5.4

# Add as many as you want
GEMINI_API_KEY=...
GEMINI_BASE_URL=https://generativelanguage.googleapis.com/v1beta
GEMINI_MODEL=gemini-2.5-pro

DEEPSEEK_API_KEY=sk-...
DEEPSEEK_BASE_URL=https://api.deepseek.com/v1
DEEPSEEK_MODEL=deepseek-chat

LOCAL_API_KEY=any-string
LOCAL_BASE_URL=http://localhost:8000/v1
LOCAL_MODEL=qwen2.5-72b

# Anthropic's own API, natively (0.5.0) — the base URL is the API root, no /v1
CLAUDE_API_KEY=sk-ant-...
CLAUDE_BASE_URL=https://api.anthropic.com
CLAUDE_MODEL=claude-sonnet-5
CLAUDE_API_FORMAT=anthropic-messages

A provider is only listed if all three of its env vars are set. If GEMINI_API_KEY exists but GEMINI_BASE_URL is missing, GEMINI doesn't appear.

Wire protocol

XXXX_API_FORMAT names the protocol spoken to that block's endpoint: openai-completions (the default when unset), anthropic-messages (Anthropic's Messages API) or openai-responses (OpenAI's Responses API, 0.5.3). It is never inferred — not from the URL, the provider name or the model name — so CLAUDE_* pointing at an OpenAI-compatible gateway stays on Chat Completions until you say otherwise, and an unknown value is an error rather than a fallback. What the native wire buys is prompt caching that works and is reported (LLM_PROMPT_CACHE=anthropic) and signed thinking blocks that survive a tool loop. Full notes: configuration reference §2.

/provider — list or switch

> /provider

Lists every detected provider, marks the active one with ✓, prints usage hint.

> /provider GEMINI

Switches credentials + base URL + default model in one shot — and the wire protocol, when the target block's XXXX_API_FORMAT differs (the CLI prints Wire protocol: A → B). The conversation history is preserved — only the LLM client changes. The next turn goes to the new provider. What does not carry over is anything minted by the model being left: thinking artifacts in history (signed thinking blocks included) are dropped, as on any model switch. LLM_EXTRA_BODY is kept, and on a wire change the CLI names its keys, since they were written for the other protocol.

Common errors:

Message	Fix
No providers found in .env	Set at least one XXXX_API_KEY + XXXX_BASE_URL + XXXX_MODEL triple
No API key found for provider 'GEMINI'	GEMINI_API_KEY is missing
No base URL configured for provider 'GEMINI'	GEMINI_BASE_URL is missing
No model configured for provider 'GEMINI'	GEMINI_MODEL is missing
Unknown api_format '…'	GEMINI_API_FORMAT is not openai-completions, anthropic-messages or openai-responses; the session stays on the provider it had

/model — list or switch model on the current provider

> /model

Shows the current model, then queries the provider for the full model list and groups output:

Current Model: gpt-5.4

Available Models:

  Claude:
    • claude-haiku-4-5
    • claude-sonnet-4-6
    • claude-opus-4-7

  OpenAI GPT:
    • gpt-5.4 ✓
    • gpt-5.4-mini

  Other:
    • o3
    • text-embedding-3-small

The list comes from the provider's /models endpoint, so it reflects what the provider actually exposes — not a hardcoded table. Switch with:

> /model claude-sonnet-4-6

/model is scoped to the current provider. If the model you want isn't available there, switch provider first with /provider.

Cross-provider model names

Some providers expose Claude models behind OpenAI-compatible APIs. If /model lists claude-* while you're on a non-Anthropic provider, that's intentional — the provider is proxying.

/temperature — sampling temperature

> /temperature        # show current
Temperature: provider default (not sent)
> /temperature 0.2    # set
Temperature changed from provider default to 0.2

Range: 0.0 to 2.0. Lower = more deterministic, higher = more creative. By default no temperature is set and none is sent, so the provider's own default applies (typically 1.0 for chat; 0.5.7 — before, 0.2 was sent). /temperature off keeps a set value but stops sending it; on sends it again, and says so when there is no value to send.

The change is per session. Restarting the CLI resets to the provider default. If you want a persistent default, set it in .env:

LLM_TEMPERATURE=0.3

On the anthropic-messages wire /temperature and LLM_TEMPERATURE have no effect: the SDK's messages.create has no temperature parameter. A gateway that does take one gets it through LLM_EXTRA_BODY.

Need a request param the CLI has no command for — reasoning_effort, top_p, seed, response_format, or a provider-specific field? Set LLM_EXTRA_BODY to a JSON object; it is forwarded verbatim to the LLM .create() (and inherited by sub-agents):

LLM_EXTRA_BODY='{"reasoning_effort":"high"}'

See Appendix B for parsing/redaction details.

/thinking — thinking depth

> /thinking           # show current
> /thinking high      # set
> /thinking off       # back to the provider default

What it writes depends on the wire, because the protocols name the field differently:

Wire	Field written into extra_body	Levels
openai-completions	reasoning_effort	minimal · low · medium · high (a non-standard word is passed through for the provider to validate)
anthropic-messages	output_config.effort	low · medium · high · xhigh · max, or the levels the provider's Models API marks supported for this model; anything else is refused, not stored
openai-responses	reasoning.effort	minimal · low · medium · high (a non-standard word is passed through, as on Chat Completions). Other reasoning keys you set — summary — are left alone, and a reasoning_effort carried in from another wire is removed, since this API rejects it

On current Anthropic models the effort alone turns adaptive thinking on. There is no auto-recovery on any wire: a model that rejects the field fails every request until /thinking off. The setting is per session and sub-agents inherit it; to make it persistent put the same field in LLM_EXTRA_BODY.

A /provider switch to another wire moves the level /thinking set into the new wire's field, because each wire rejects the others' fields. A level the new wire does not take (minimal on anthropic-messages) is dropped, and the log says so. Only a level that /thinking (or agentao run's thinking) wrote moves, and only while its field still holds it. If you already set the new wire's field yourself, your value wins and the moved level is dropped. A field you set in LLM_EXTRA_BODY is yours: it stays as it is, and a switch to a wire that rejects it fails every request until you change it.

When to switch what

Situation	What to do
Same task, want a smarter model	/model <bigger> (stays on provider)
Same task, want a cheaper model	/model <smaller>
Switch to a different vendor	/provider <NAME> (history kept, credentials swap)
Outputs feel too random / hallucinatory	/temperature 0.2
Outputs feel too rigid / repetitive	/temperature 1.2
Hard problem, want more reasoning	/thinking high
Cost is exploding mid-session	/model to a smaller variant before the next turn

Pitfalls

• Switching mid-tool-call: if you switch model or provider while the agent is still iterating tools, the next iteration uses the new client. Mostly fine, but tool-call format differences between providers can cause a single garbled turn — use /clear after switching if you see this.
• Conversation history is preserved across switches: this is usually what you want, but it means the new model sees the previous model's tool-use traces. Some models are stricter than others about message format mismatches.
• Models you can name aren't always models you can call: /model lists what the provider's /models endpoint reports; some entries (embedding models, fine-tunes you don't have access to) will fail at the next turn. /model <name> also accepts a name that is not in the list, without checking it: a typo is reported only when the next turn fails. Check the error and pick another.

Where to go next

Want to…	Read
Make sure the agent doesn't run dangerous tools on the new model	3. Permissions & Modes
Recover from a context blow-up after switching to a smaller model	7. Context & Status
Run a local model	Set LOCAL_* env vars as shown above; same /provider LOCAL flow

---

Where this fits

The CLI uses cli.agent.set_provider(api_key, base_url, model) and cli.agent.set_model(name) — both are public methods on the Agentao instance. An embedding host can call them too, with the same effect on the same in-memory session. See Part 2.5 · Runtime LLM Switch for the embedded-API equivalents and the threading caveats.

Authoritative help

Command syntax: /help and agentao/cli/help_text.py. Discovery logic for providers: agentao/cli/commands/provider.py:_list_providers_from_env.