BlazeInferDocs

Session affinity & caching

Prefix-aware caching works by default; an optional x-session-id header adds session affinity.

BlazeInfer caches the prompt prefix (the KV cache) and routes requests with prefix awareness by default. When a request starts with the same messages as an earlier one, such as a later turn of the same conversation, the shared prefix can be served from cache. That makes multi-turn work faster, because the model does not process the whole prompt again on each turn. You do not need to send anything extra for this.

Optional: the x-session-id header

x-session-id is optional. In some cases, for example multimodal conversations that include images, you can enforce additional session affinity by sending a stable x-session-id on every request in the conversation. BlazeInfer then sends every turn of that conversation to the same model replica.

curl https://blaze.e2enetworks.net/v1/chat/completions \
  -H "Authorization: Bearer $BLAZE_API_KEY" \
  -H "x-session-id: conv-8f3a2c" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "deepseek-ai/deepseek-v4.1-flash",
    "messages": [{"role": "user", "content": "Continue our last conversation."}]
  }'

Choose a session ID that is stable for one conversation and unique across conversations. A UUID or a hash of your conversation ID works well.

Observing cache reuse

When a prefix is served from cache, the usage block can report it:

{
  "usage": {
    "prompt_tokens": 812,
    "completion_tokens": 64,
    "total_tokens": 876,
    "prompt_tokens_details": { "cached_tokens": 640 }
  }
}

Coding agents

OpenCode and Kilo Code send a stable x-session-id on every request in a session, so they get session affinity without extra setup. In your own clients and SDK code, add x-session-id yourself when you want it. Behind a LiteLLM proxy, turn on header forwarding so the header reaches BlazeInfer; see LiteLLM.

On this page