ALGOTECHNIC
All articles
8 min read

Structured Outputs and Interfaces: Enforce Schemas Without Surrendering Validation

Schema-constrained generation makes model output machine-readable, but production systems still need semantic checks, bounded retries and authorised tool execution.

  • structured generation
  • JSON Schema
  • tool use
A language model's token stream passes through a glowing schema frame into typed fields, while a separate validation gate remains ahead.

A model response becomes an interface when another program depends on its shape. Prompting for JSON treats that shape as an instruction. Schema-constrained generation treats it as a decoding constraint. The distinction removes a common class of formatting failures, but it does not make the response correct, meaningful or safe to act on.

The production design should therefore preserve several gates. Define a small contract, constrain generation where the provider supports it, parse the completed payload, validate application rules, then decide whether the caller may execute an action. A valid object can cross the first three gates and still fail the last two.

A pipeline moves from schema and prompt through constrained decoding, parsing, semantic validation and authorised execution, with separate policies for refusals, truncation, transport failures and rejected actions.
Schema adherence belongs early in the interface pipeline. Only a complete, parsed and semantically valid proposal reaches authorisation; each failure class has its own retry policy.

Design the contract, not a container for prose

Start with the smallest object the consumer can handle. Give fields stable, descriptive names; use enums for genuinely closed vocabularies and arrays only when the consumer accepts several items. Close objects with additionalProperties: false when the interface must reject unknown keys. Make absence explicit with a nullable field rather than letting the model invent a value to fill a mandatory string.

OpenAI’s August release illustrates why these choices need provider-specific tests. Its strict Structured Outputs accepts a subset of JSON Schema. Objects must list every field as required and disallow additional properties; a nullable union can represent an optional value. Unsupported constructs produce a request error, not a model answer that needs another attempt. Generate the wire schema and application type from one definition where possible, and version both when the contract changes.

Consider a proposal to move an appointment:

{
  "type": "object",
  "properties": {
    "appointment_id": { "type": "string" },
    "requested_start": { "type": "string" },
    "reason": { "type": ["string", "null"] }
  },
  "required": ["appointment_id", "requested_start", "reason"],
  "additionalProperties": false
}

This contract says nothing about whether the identifier exists, the timestamp parses or the new time is available. That is deliberate. A schema can express types, required keys, numeric ranges and some string constraints. It is a poor place to hide a booking policy in a sprawling regular expression. Complex constraints also increase the grammar that the inference system must compile and maintain.

Leave room for legitimate negative outcomes. An extraction contract may need an empty result or an explicit not_found status. Forcing every request into a success-shaped object encourages plausible fabrication when the input contains nothing to extract.

Constrain tokens without confusing form with meaning

Autoregressive models normally choose from the whole vocabulary at each step. Guided generation adds a parser or state machine that tracks the prefix already produced. Tokens that cannot extend that prefix into an allowed document have their probabilities masked to zero; the model samples only from the remaining continuations.

Willard and Louf’s 2023 work made the mechanism concrete for regular expressions and context-free grammars. Their finite-state formulation indexes vocabulary tokens by valid parser states, moving work into preprocessing and an in-memory index. Context-free approaches can represent nesting that a plain finite-state machine cannot. OpenAI’s implementation likewise converts its supported schema subset into a context-free grammar, preprocesses it and caches the resulting artefact. A new schema therefore adds first-request latency; complexity is an operating cost, not just schema-authoring convenience.

This is stronger than JSON mode’s promise of parseable JSON. A document can be valid JSON while omitting appointment_id or supplying an object where a string belongs. A supported strict schema constrains those structural choices. It does not choose the right appointment, and masking does not verify evidence behind a value. The model can produce a beautifully typed request for the wrong record.

Constraints also do not guarantee timely termination. The prefix can remain grammatically extendable until an output limit interrupts it. Treat completion status as part of the interface rather than assuming that every returned string reached a valid final state.

Parse first, then validate the application

Parsing turns bytes into data; schema validation checks that data against declared assertions; semantic validation decides whether the application can use it. Keep these stages visible in logs and error handling. A JSON syntax error, a missing key and an unavailable appointment need different responses.

Use a strict parser on a complete payload. Do not repair truncation by appending braces, silently coerce unexpected types or execute fields from a growing stream. Bound payload size and nesting, and account for numeric precision when values cross into a language with narrower number types. Typed SDK helpers reduce boilerplate, but their successful deserialisation is not a business-rule decision.

JSON Schema’s own specification makes the next distinction clear. Its format keyword is primarily an annotation in the 2020-12 draft, and format assertion is optional and configurable. Even a syntactically valid email or date does not establish that the mailbox exists or the date is available. Application validators must parse requested_start, compare it with the relevant time zone and calendar, and check cross-field rules. Pydantic’s model validators offer one implementation route for checks that depend on several fields.

Semantic checks often need current state outside the model response. Resolve the appointment identifier against the authorised data source, inspect its status and test the requested slot against current availability. Do not ask the model to certify facts that the database can decide. Record rule identifiers with rejections so operators can distinguish bad extraction from a legitimate but impossible request.

Truth requires evidence beyond both schema and validator. If the response asserts a policy reason or summarises a source, evaluation must test that claim against the source. A validator can enforce an allowed label while the selected label remains wrong.

Give every failure a bounded policy

Retries are useful only when another attempt can change the outcome. Set a total deadline and a small attempt budget, then classify the failure before spending either.

  • Refusal: inspect the explicit refusal field before parsing normal content. Return an allowed alternative or route for review. Do not automatically rephrase a safety refusal until the model complies.
  • Truncation or another early stop: reject the partial object. A bounded retry may use a larger output allowance or a smaller task, but repeating an unchanged request can reproduce the failure. Persistent truncation indicates a contract or task-design problem.
  • Transport failure: timeouts, dropped connections and server errors occur outside schema enforcement. Retry only within the request deadline, using backoff and jitter. A lost response leaves delivery uncertain, so preserve the original request identity and never assume that silence means no work occurred.
  • Parse or schema failure: strict generation should remove many formatting repairs, but retain the branch for unsupported features, non-strict fallbacks and implementation bugs. Return specific validator errors only when a repair attempt is permitted; never loop without a cap.
  • Semantic rejection: prefer clarification, corrected input or a deterministic failure over repeated sampling. Another plausible object does not repair an unavailable slot.

Streaming changes presentation, not these rules. Partial objects can update a progress display, but the consumer must wait for a completed, non-refused response before final parsing or action. A broken stream is a transport failure with an incomplete payload, not a valid prefix to execute.

A tool call is a proposal, not authority

Function calling supplies a name and structured arguments. It does not supply permission. OpenAI’s original 2023 function-calling announcement warned that untrusted tool output could steer the model towards unintended actions and recommended confirmation before actions with real-world impact. Schema enforcement does not change that boundary.

Resolve the returned name against a server-side allowlist of fixed implementations. Never evaluate generated code or turn a model-supplied string directly into a database query. Validate the complete arguments, re-check semantic rules against current state and authorise the actual user for this specific appointment and operation. Require confirmation when the consequence warrants it, then execute with least-privilege credentials, a timeout and an audit record.

Side-effecting tools need their own failure semantics. Use idempotency keys or transactional deduplication where the tool supports them, so a transport retry cannot move an appointment twice. Keep model-request retries separate from tool-request retries: the former may produce a different proposal, while the latter may repeat an effect. Treat returned tool content as untrusted data, not as a new instruction channel.

The useful guarantee is narrow but valuable: supported constraints can make structure dependable enough for software to consume. The interface remains responsible for everything the structure cannot express. Keep parsing, validation, authorisation and execution explicit, and a schema becomes a reliable boundary rather than an excuse to surrender one.

Sources and further reading