Kairox Arch Prep
STUDY NOTES · LESSON 10 OF 12 · 6 MIN READ

Structured output and the validation-retry loop

How to get reliable JSON from Claude: structured outputs, strict tools, tool_choice and its limits, honest schemas, truncation checks and capped retries.

In 30 seconds

  • Constrain the shape: structured outputs for the response, strict tool use for tool inputs.
  • Forced tool_choice ("any" or a named tool) is model-dependent; some current models reject it.
  • Design honest schemas: nullable fields for data that may be missing, enums for fixed vocabularies.
  • Validate in code, retry with the exact error, cap retries, and route the rest to people.
Structured output with validation and a capped retry loopRequest+ JSON schemaResponsestructuredstop_reasonmax_tokens = cut offValidateschema + rulesfail → retry with the exact error (max N)StorepassHuman reviewafter N failures
Constrain the shape, check for truncation, validate what the schema cannot express, and retry with the exact error — a limited number of times.

Constrain the shape

Asking for "valid JSON" in prose works most of the time, which is not good enough for a pipeline. Structured outputs take a JSON schema for the response; strict tool use validates a tool's inputs against its schema. Either way, the shape is defined by the request, not hoped for.

tool_choice, and its limits

The options are auto (Claude decides), any (must call some tool), a named tool (must call that one) and none. Forcing a tool is model-dependent: several current models, and requests using manual extended thinking, reject any and named-tool choices. Where that happens, use auto with strict tools, or structured outputs.

Schemas that do not force invention

If due_date is a required string but some invoices have no due date, the model is pushed to make one up. Make it nullable and say "return null when the document does not state it". Free-text categories drift ("Electronics", "Elec."), so use an enum with a short definition for each value.

"due_date": {"type": ["string", "null"], "description": "ISO date, or null if not stated"},
"category": {"type": "string", "enum": ["electronics", "furniture", "services"]}

Check, validate, retry — with limits

First check the stop reason: max_tokens means the output was cut off. Then validate what a schema cannot express, such as totals adding up. If validation fails, retry with the original input plus the specific error — "due_date is after the invoice date" — rather than resending the same request. Cap the retries and send persistent failures to a person instead of filling gaps with defaults.

Scale

Large, non-urgent extraction jobs suit the Message Batches API, which runs asynchronously at a lower price; very large archives are split across several batches.

Part of Domain 4: Prompt Engineering & Structured Output (20% of the exam). Explanations are our own, checked against Anthropic's documentation on November 22, 2026.

Common traps

TrapWhy it failsDo this instead
"Return valid JSON" with no schemaOccasional malformed output breaks the pipelineStructured outputs or strict tools
Required field the source often lacksPushes the model to invent valuesNullable field + instruction
Retrying with the same requestSame input, same failureRetry with the exact validation error, capped

Check yourself

Two original practice questions on this lesson. Try them before opening the answers.

Check yourself 1

Scenario: Structured data extraction

An extraction pipeline must always return JSON matching a fixed schema so downstream code can parse it. Which approach gives the most reliable structure?

  1. A. Ask for valid JSON in the prompt, include the schema, and parse whatever comes back
  2. B. Set a very high max_tokens so the JSON object is never cut off before it closes
  3. C. Ask for a markdown table and convert it to JSON with a regular-expression parser
  4. D. Use structured outputs with a JSON schema, check the stop reason, then validate business rules
Show answer and explanation

Answer: D. Structured outputs (a JSON schema supplied in output_config.format) constrain the response to the schema; you still check the stop reason for truncation and validate rules the schema cannot express. Strict tool use (strict: true) is the equivalent for tool inputs. Prompt-only JSON requests fail occasionally; regex over tables is brittle; a large max_tokens prevents truncation but not schema errors.

Check yourself 2

Scenario: Structured data extraction

Your validator finds that an extracted record is missing the required due_date field. What is the most effective retry strategy?

  1. A. Resend the identical request, since a second sample will often include the field
  2. B. Fill in today's date so the record passes validation and can continue downstream
  3. C. Retry with the input plus the exact validation error, and cap the number of retries
  4. D. Retry with a higher temperature so the model is more likely to produce a new answer
Show answer and explanation

Answer: C. Validation-retry works best when the model gets precise feedback about what failed, with a retry cap and a review queue for persistent failures. Resending the same request repeats the same conditions; raising temperature adds randomness rather than information; inventing a value silently corrupts data.

Keep going