Structured Output

Structured output is a language model response in a machine-readable form, usually JSON, that a program can parse and use directly. It matters wherever a model’s answer feeds code rather than a person: extracting fields from documents, classifying text into fixed categories, or producing arguments for a tool call.

Three levels of control

The weakest is asking for JSON in the prompt. The model usually complies, but occasionally adds a sentence before the object, drops a closing brace, or renames a field, and the parser fails. Some services offer a JSON mode that constrains successfully completed output to valid JSON without a fixed shape. The strongest is schema-constrained generation: the application supplies a schema, and the service restricts decoding so that successful output conforms to the supported schema. This contract does not promise a successful response to every request. Google’s Gemini API documentation describes this as configuring models to “generate responses that adhere to a provided JSON Schema”, with data extraction and classification among its uses. Services typically support only part of the JSON Schema standard, and the same documentation notes that very large or deeply nested schemas may be rejected.

Valid is not correct

On a successfully completed response, a supported schema constrains form without establishing factual correctness. Google’s Gemini guide requires application-side validation and handling of schema-compliant answers whose values are wrong. An invoice extractor can return a well-formed object with the wrong total, a date copied from the delivery note instead of the due date, or a supplier name that appears nowhere in the document.

A required non-null date leaves no way for a successful schema-compliant response to express a missing due date. This can encourage an invented value, though refusal or failure is also possible. Make absence representable with nullable fields or an explicit not_found value, and evaluate both false answers and incorrect abstentions.

What to check in code

• Evidence. Request a source span with document, page, and field provenance. Its presence alone does not identify the right field: an invoice date can be mistaken for a due date. Verify normalized values against the span using explicit date and amount rules.
• Internal consistency. Reconcile line items, subtotal, tax, discounts, shipping, and rounding under the document’s accounting rules. Check page coverage and continued rows before summing; a plausible date order alone does not prove that dates are correct.
• Reference data. Supplier identifiers, currency codes, and account numbers should match known records.
• Completion. A response cut off at the output limit, or refused, is not a result. Check status and finish reason, reject empty or incomplete results, then parse, validate the schema, and validate meaning.

Cases that fail these checks go to review or a retry, not into the downstream system. Structured output constrains successful response formats. Completion handling, semantic checks, and sampled human audits address different failure modes; none proves that every accepted value is correct.

How structured output fits with tokens, context, latency, and model choice is worked through in Valid JSON, Wrong Date.

References:

• Google AI for Developers, Gemini API: Structured outputs (Creative Commons Attribution 4.0).
• Gemini generateContent API: FinishReason.