BlazeInferDocs

Structured outputs

Ask a model for JSON, either in the prompt or with a JSON schema.

You can ask a model to return JSON in two ways: describe the shape you want in the prompt, or send a JSON schema in response_format.

Both response_format: {"type": "json_schema", ...} and response_format: {"type": "json_object"} are supported.

Prompt-based JSON

Describe the schema in the system message, then parse the reply. Nothing forces the model to follow the prompt, so be ready for a json.JSONDecodeError:

import json, os
from openai import OpenAI

client = OpenAI(
    base_url="https://blaze.e2enetworks.net/v1",
    api_key=os.environ["BLAZE_API_KEY"],
)

response = client.chat.completions.create(
    model="qwen/qwen3.8-27b",
    messages=[
        {
            "role": "system",
            "content": (
                "Reply with JSON only, matching "
                '{"title": string, "tags": string[]}. No prose.'
            ),
        },
        {"role": "user", "content": "A short article about vector databases."},
    ],
)

data = json.loads(response.choices[0].message.content)

Schema-enforced output

Send the schema in response_format, as with OpenAI:

{
  "model": "google/gemma-4-31b-it",
  "messages": [
    { "role": "user", "content": "Give a title for an article about vector databases." }
  ],
  "response_format": {
    "type": "json_schema",
    "json_schema": {
      "name": "article",
      "strict": true,
      "schema": {
        "type": "object",
        "properties": { "title": { "type": "string" } },
        "required": ["title"]
      }
    }
  }
}

Two things to watch:

  • Give every array a maxItems. Without a limit, the model can repeat items until it reaches max_tokens.
  • Leave room in max_tokens for reasoning, because reasoning counts towards it.

On this page