Caching

Fusion AI Gateway caches completions by default. No custom headers, no custom SDKs, no configuration — point any OpenAI-compatible client at https://api.fusioncode.app/v1 and caching is already on.

There are two layers:

1. Exact-match response cache — byte-identical requests are served from the edge in ~1ms without touching the model provider.
2. Prefix-cache credit — requests that reuse a previous prompt’s prefix (e.g. a coding agent conversation where only the last turn changed) are billed at the cached-input rate for the shared prefix tokens.

Response headers

Every completion response carries cache diagnostics:

Example — a changed-suffix request that reused a 145,007-token prefix:

x-fusion-cache: MISS
x-fusion-cache-prefix: HIT
x-fusion-prefix-cached-tokens: 145007

Cache key composition

A response is reused only when the entire key matches:

sha256(model + messages + max_tokens + api_key + sampling params)

where sampling params include temperature, top_p, stop, response_format, seed, and penalties. Two requests that differ in any of these never share a cache entry, and cached responses are namespaced per API key — no user can read or be billed for another user’s cached completion.

Opting out

Caching is on by default. To force fresh completions for a specific request:

• Send the standard HTTP header Cache-Control: no-cache, or
• Set "cache": false in the request body.

Opted-out requests are neither served from cache nor stored, and they do not contribute fingerprints to the prefix registry.

Usage & savings reporting

GET /v1/usage reports caching effectiveness and the dollar savings:

{
  "cached_tokens_this_month": 817382,
  "prompt_cache_hit_rate_pct": 1.78,
  "cache_hit_count_this_month": 218,
  "cache_savings_usd_this_month": 0.061304,
  "cache_savings_by_model": {
    "deepseek-ai/DeepSeek-V4-Flash-0731": {
      "cached_tokens": 817382,
      "savings_usd": 0.061304
    }
  }
}

Savings are computed as cachedTokens × (inputPrice − cachedPrice) per model — the exact amount prefix caching took off your bill.

Prefix-credit estimation

Upstream providers do not always report cached tokens in their usage responses (prompt_tokens_details: null). Fusion therefore tracks request fingerprints and credits the shared prefix from its own calibrated per-message token accounting:

• Each message’s token count is anchored to the upstream’s exact prompt_tokens total.
• Identical messages keep stable calibrated counts across requests.
• The credit is capped at the current request’s actual prompt_tokens.

The credit appears in your /v1/usage metrics and prompt_log billing immediately, and is reflected in the x-fusion-prefix-cached-tokens header.

TTLs

Per-client rate limits

Accounts can be assigned their own rate limit (admin-managed). When set, the limit applies per account; otherwise the platform defaults apply. A limited account receives 429 responses with a Retry-After header indicating when to retry.