Providers and gateways

Cusp talks to models through four wire protocols. Almost every provider, gateway and local server speaks at least one of them, which is how one runtime covers so many models.

When a provider speaks more than one, Cusp prefers the one that preserves the most: Responses for OpenAI and xAI, Messages for Anthropic, Z.ai, Moonshot and MiniMax, and generateContent for Gemini.

ProvidersCopy Anchor Link

A dash in the key prefix column means the key format isn't distinctive. Cusp still identifies the provider by probing, or from the base URL.

Cloud platformsCopy Anchor Link

Cusp recognizes these from the host name (*.openai.azure.com, bedrock…, aiplatform.googleapis.com) and applies each platform's request differences automatically.

GatewaysCopy Anchor Link

A gateway gives you many models behind one key. Cusp treats each gateway as a provider and reads its model list, so you pick models by their gateway slug.

Other OpenAI-compatible gateways (Portkey, Cloudflare AI Gateway, Helicone, and so on) work as a custom endpoint: give Cusp the base URL and any extra headers the gateway needs.

Local modelsCopy Anchor Link

No key is required for local runtimes. For local models Cusp picks the largest loaded model that supports tool calling with at least a 32k context window.

Auto-detectionCopy Anchor Link

When you paste a key, with or without a base URL, Cusp runs a short pipeline:

1. Hints. The key prefix (sk-ant-, sk-or-, AIza, …) and the host of the base URL each point to a provider.
2. Probes. Cusp sends cheap, token-free requests (such as listing models) to each candidate protocol in parallel, with a 6-second timeout per probe.
3. Fingerprints. Even a 401 tells Cusp something: error bodies and headers identify the provider behind an unknown URL.
4. Models. The model list comes from the endpoint, enriched from Cusp's built-in catalog (context window, pricing, capabilities) and public sources.
5. Pick. Cusp chooses a main model (the highest agentic score among models with tool calling) and a small model for auxiliary work: session titles, the auto-mode classifier, compaction and web page extraction.

You can override every result. Auto-detection only fills in a configuration; nothing is hidden.

Per-model-family adaptersCopy Anchor Link

The protocol is only half the story. Models from different labs were trained with different tool conventions, and they behave best when the harness speaks their dialect. Cusp keeps a profile per model family and adapts to it.

Edit formatCopy Anchor Link

Write is available to every family. Fuzzy matching for Edit goes through a ladder (exact, normalized whitespace, trimmed, re-indented, block-anchored) and reports which tier it used, so a near miss doesn't silently edit the wrong place.

Reasoning round-tripCopy Anchor Link

Reasoning models return state that must be sent back on the next turn: Anthropic thinking signatures, OpenAI encrypted reasoning, Gemini thought signatures, and reasoning_content from DeepSeek, Kimi and GLM. Dropping it causes errors on some providers and a quiet loss of quality on others. Cusp stores this state opaquely with each message and replays it byte-exact to the same model. When you switch models mid-session, it sends a safe projection instead and notes the change.

Other adaptationsCopy Anchor Link

• Tool names and descriptions. GPT models see update_plan instead of TodoWrite. Hooks and permission rules always see the canonical name.
• Strict schemas. On OpenAI, tool schemas are sent in strict mode, with optional fields made nullable and mapped back.
• Broken tool calls. Open models sometimes emit malformed JSON or tool calls embedded in text (Hermes, Qwen, GLM, MiniMax and Kimi markup). Cusp repairs JSON, coerces types against the schema, and extracts text-embedded calls before giving up.
• Parallel calls. Turned off for runtimes that don't support them.
• System prompts. Dynamic context goes where each provider handles it best: a mid-conversation system message on Claude, a developer item on OpenAI, a system reminder elsewhere.

Caching and costCopy Anchor Link

Cusp keeps the system prompt and tool list byte-stable across turns so providers can cache them. It sets the cache key each provider understands (prompt_cache_key on OpenAI, the session id on OpenRouter, a conversation header on xAI) and places Anthropic cache breakpoints automatically. Cost is computed per turn from the model's pricing, counting cache reads and writes separately. /cost in the CLI shows the running total.

Retries and fallbacksCopy Anchor Link

Transient errors (rate limits, overloaded servers, dropped streams) are retried with backoff, and the UI tells you what happened: "Rate limited by Anthropic — retrying in 8 s." If a request fails because the endpoint rejects a parameter, Cusp learns from the error and retries without it. You can configure a fallback model (fallbackModel in settings); Cusp stays on one route as long as it works, to keep the cache warm.

Environment variablesCopy Anchor Link

The CLI and daemon pick up keys from these variables:

ANTHROPIC_API_KEY  OPENAI_API_KEY      GEMINI_API_KEY     GOOGLE_API_KEY
OPENROUTER_API_KEY AI_GATEWAY_API_KEY  XAI_API_KEY        DEEPSEEK_API_KEY
ZAI_API_KEY        ZHIPU_API_KEY       MOONSHOT_API_KEY   KIMI_API_KEY
MINIMAX_API_KEY    DASHSCOPE_API_KEY   MIMO_API_KEY       MISTRAL_API_KEY
GROQ_API_KEY       CEREBRAS_API_KEY    TOGETHER_API_KEY   FIREWORKS_API_KEY
DEEPINFRA_API_KEY  NOVITA_API_KEY      SILICONFLOW_API_KEY

In the desktop app, keys you paste are stored in the OS keychain instead.