Kairox Arch Prep
STUDY NOTES · LESSON 12 OF 12 · 6 MIN READ

Reliability: retries, human review and provenance

Rate limits and backoff, circuit breakers, idempotent batch jobs, routing work to human review, and keeping sources visible in multi-source answers.

In 30 seconds

  • Retry only transient failures, with exponential backoff, jitter and a cap.
  • When a dependency keeps failing, stop calling it and switch to a defined degraded mode.
  • Batch jobs need per-item status and idempotent processing.
  • Spend human review where errors are likely, and keep every claim tied to its source.
Deciding how to handle a failed call: retry with backoff, or report clearlyCall failsTransient?429 · 5xx · timeoutBack offjitter · retry-aftercap attemptsCircuit breakerdegraded modePermanentbad input · not foundReport clearlyno blind retriesyesnostill failing?
Retry only what is transient, with backoff and a cap; stop hammering a failing dependency; never retry a permanent error blindly.

Rate limits and transient errors

An HTTP 429 means you are sending too much too fast. Retrying immediately — or opening more connections — makes it worse. Back off exponentially, add random jitter so many clients do not retry in lockstep, honour any retry-after information, and stop after a set number of attempts. Server errors and timeouts get the same treatment. Errors that will not change on retry — bad input, a record that does not exist — should be reported clearly instead.

When a dependency is down

If the model endpoint or a tool's back end fails repeatedly, a circuit breaker stops sending requests for a while and switches the system to a degraded mode decided in advance: a fallback path, a queue, or an honest "we're running slowly" message. It closes again when health returns.

Jobs with thousands of items

Give each item its own status and make processing idempotent, so a retry never creates duplicates. Retry transient failures with backoff and move items that keep failing to a dead-letter queue for review. Restarting the whole job after one failure wastes work and can repeat side effects.

Human review where it counts

If people can check 2,000 of 50,000 extracted records a month, send them the ones most likely to be wrong: records that fail validation, miss key fields, or fall in categories with poor measured accuracy. Add a small random sample to keep those signals honest. Reviewing the first 2,000 or a purely random set wastes scarce attention.

Provenance and uncertainty

When an answer combines several sources, each figure should keep its source and date. If sources disagree, or one cannot be dated, say so in the output rather than averaging or quietly choosing. When part of the work fails — a source is unavailable — report the gap with what was attempted, so readers know what the answer does not cover.

Prove it with numbers

A reliability improvement is a measurement, not a demo: success, escalation and error rates on the same evaluation set before and after the change, then production monitoring.

Part of Domain 5: Context Management & Reliability (15% of the exam). Explanations are our own, checked against Anthropic's documentation on November 22, 2026.

Common traps

TrapWhy it failsDo this instead
Retrying 429s immediatelyAdds load to an overloaded serviceBackoff + jitter + cap, honour retry-after
Restarting a batch after one failureWasted work and duplicate side effectsPer-item status, idempotency, dead-letter queue
Averaging conflicting sourcesHides the disagreementShow each value with its source; flag the conflict

Check yourself

Two original practice questions on this lesson. Try them before opening the answers.

Check yourself 1

Scenario: Claude Code for continuous integration

Your service calls the Claude API in bursts and sometimes receives HTTP 429 responses. What is the correct client behaviour?

  1. A. Treat 429 as a permanent failure, log it, and drop the affected request
  2. B. Retry with exponential backoff and jitter, honour retry-after, and cap attempts
  3. C. Retry immediately so the request lands as soon as capacity frees up again
  4. D. Open more parallel connections so requests spread across more capacity
Show answer and explanation

Answer: B. 429 means rate limited: back off exponentially, add jitter so clients do not retry in lockstep, honour retry-after when present, and cap attempts. Immediate retries and extra connections increase the load that caused the limit, and dropping requests loses work that would succeed shortly.

Check yourself 2

Scenario: Multi-agent research system

One of four research subagents fails because its main source is unavailable. The other three succeed. What should flow back to the coordinator and into the final report?

  1. A. An estimate of what the failed source would probably have said, marked low confidence
  2. B. A structured failure: what was attempted, any partial findings, and the gap left
  3. C. A generic error string so the coordinator knows to retry the whole research run
  4. D. Nothing from the failed subagent; the report should only include successful work
Show answer and explanation

Answer: B. Errors should propagate with enough structure for the coordinator to decide: what failed, what partial results exist, and which part of the question is now uncovered, so the report can state the gap. Silently dropping the failure overstates coverage, a bare error forces a costly full retry, and inventing probable content is fabrication.

Keep going