Session affinity & caching
Prefix-aware caching works by default; an optional x-session-id header adds session affinity.
BlazeInfer caches the prompt prefix (the KV cache) and routes requests with prefix awareness by default. When a request starts with the same messages as an earlier one, such as a later turn of the same conversation, the shared prefix can be served from cache. That makes multi-turn work faster, because the model does not process the whole prompt again on each turn. You do not need to send anything extra for this.
Optional: the x-session-id header
x-session-id is optional. In some cases, for example multimodal conversations
that include images, you can enforce additional session affinity by sending a
stable x-session-id on every request in the conversation. BlazeInfer then sends
every turn of that conversation to the same model replica.
curl https://blaze.e2enetworks.net/v1/chat/completions \
-H "Authorization: Bearer $BLAZE_API_KEY" \
-H "x-session-id: conv-8f3a2c" \
-H "Content-Type: application/json" \
-d '{
"model": "deepseek-ai/deepseek-v4.1-flash",
"messages": [{"role": "user", "content": "Continue our last conversation."}]
}'Choose a session ID that is stable for one conversation and unique across conversations. A UUID or a hash of your conversation ID works well.
Observing cache reuse
When a prefix is served from cache, the usage block can report it:
{
"usage": {
"prompt_tokens": 812,
"completion_tokens": 64,
"total_tokens": 876,
"prompt_tokens_details": { "cached_tokens": 640 }
}
}Coding agents
OpenCode and Kilo Code send a stable x-session-id on every request in a
session, so they get session affinity without extra setup. In your own clients
and SDK code, add x-session-id yourself when you want it. Behind a LiteLLM
proxy, turn on header forwarding so the header reaches BlazeInfer; see
LiteLLM.