Idempotency

Ticket #8823 needs a $4,200 credit. Atlas calls issue_credit. Eight seconds later the connection drops with no response.

Did the money move?

Atlas v0 listed this as one of the six things it could not survive, and the previous chapter established why it cannot be papered over: every repetition mechanism in this book assumes repeating is free. That includes retry policies, replay from a checkpoint, and durable re-execution in Part X. For issue_credit repeating is not free, and this chapter is how you make it free.

Three outcomes, and the third is the common one

The mental model that causes the bug is that a call either succeeded or failed. There are three states, and you are almost always in the third:

   ①  succeeded    you got a 200. easy.
   ②  failed       you got a 4xx before anything happened. also easy.
   ③  unknown      the connection died, the timeout fired, the process
                   was killed. you have no idea.

And a detail that should change your instinct about which way state ③ leans: most network timeouts happen on the response, not on the request. The server received it, did the work, and the reply was lost coming back. "Unknown" is not a coin flip. It skews toward it happened, which is the direction that costs money.

There is no protocol fix for this. You cannot make the network tell you. The only available move is to make the question stop mattering: design the operation so that performing it twice and performing it once produce the same world.

Two different things repeat the call

Here is where an agent differs from an ordinary distributed system, and it is the part that gets designed wrong.

The second row is the load-bearing one. When the model sees Error: request timed out in its context and decides to call issue_credit again, that is not a retry from where it is standing. It is a fresh judgment about what to do next, made by something with no memory of having a previous attempt in flight. It cannot supply a stable key on the second call, because supplying one would require knowing there was a first. And if you ask it for one anyway, you get the invented-identifier failure with a new name.

Which gives this chapter's rule, and it is a sibling of one you have already met:

Arguments come from the model. Authority comes from your code. Idempotency keys come from your code too.

Reported rates put agent tool-call repetition at 15–30%: timeouts, validation errors, and the model simply changing its mind. This is not an edge case you handle in a later hardening pass. It is a third of your write traffic.

Deriving the key

The key must be identical across every repetition of the same logical operation, and different for genuinely different operations, without anyone asking the model. Three parts do it:

   idempotency_key = hash(
       run_id           ← this ticket's run. scopes the key to one job.
     + tool_name        ← same args, different tool, different operation
     + canonical(args)  ← sorted keys, normalized values
   )

   run 8823 · issue_credit · {account:4471, cents:420000}
        └──────────────▶  ik_9f2c41…   (stable across every repeat)

import { createHash } from 'node:crypto';

export function idempotencyKey(
  runId: string,
  toolName: string,
  args: Record<string, unknown>,
): string {
  // Sorted keys — an unstable serialization is an unstable key.
  const canonical = JSON.stringify(args, Object.keys(args).sort());
  return createHash('sha256')
    .update(`${runId}\0${toolName}\0${canonical}`)
    .digest('hex')
    .slice(0, 32);
}

// The model never sees this. It is added by the dispatcher, next to
// the requester check, and sent as a header the tool's API honours.

The canonicalization is not fussiness. An unsorted JSON.stringify produces different bytes for the same object depending on insertion order, and a key that changes when nothing changed is not a key. This is the same determinism requirement as a cacheable prompt prefix, for the same reason.

Note what the argument hash buys: if the model calls again with $4,200 the key matches and nothing new happens; if it calls with $4,900 the key differs and a second credit is issued, which is correct, because that is a different operation. A model correcting its own mistake must not be silently deduplicated.

Tell the model it was a replay

Silent dedup is right for your infrastructure and wrong for the model, and this is the subtlety that separates a working design from one that produces a strange transcript.

If the second issue_credit returns a fresh-looking success, the model has now seen two successful credit issuances in its context. Tool results are prompt text; it will reason over what it reads. The reply that reaches the customer may cheerfully mention two credits, or the total may be wrong in a summary. Nothing failed, and the answer is incorrect.

So report the replay honestly:

{
  "status": "already_applied",
  "credit_id": "cr_8823_1",
  "amount_cents": 420000,
  "applied_at": "2026-08-09T14:02:11Z",
  "note": "This credit was already issued during this run. No new credit was created."
}

That is not an error and should not carry is_error. It is the truthful answer to "what happened when I called this," written for the consumer that has to explain it to a customer. This is the same discipline as errors as instructions, applied to a success.

Natural idempotency beats the synthetic kind

Keys are the fallback. The better move is an operation that cannot be doubled because of how it is shaped. That is a schema decision, and therefore squarely a Part VIII concern.

The pattern is to prefer the absolute over the relative:

Every left-hand column entry describes a change; every right-hand entry describes a desired state. Declarative tools are idempotent by construction, and the cost is usually just naming the thing you are writing.

The last row is the honest limit. A class ⑤ external write cannot be made naturally idempotent, because the deduplication would have to live inside someone else's mail server. For those the key must be checked before the send, and the send must be the only effect in its unit of work. That is the one-effect-per-node rule from Part VII arriving from a completely different direction, and it is worth noticing that two independent lines of argument landed on the same constraint.

Every irreversible write ships with a read that confirms it

The catalogue rule that falls out of state ③.

If issue_credit times out and you have no way to ask whether credit cr_8823_1 exists, "unknown" is permanently unresolved and your only recovery is a person querying a database at 2am. So:

An irreversible write tool is incomplete without a paired read that can confirm its outcome by reference.

issue_credit ships with get_credit(reference). send_reply ships with get_delivery_status(message_ref). The pair is what turns state ③ into state ① or ② on demand: for your retry logic, for the trace, and for the human who eventually asks what happened.

That read also has to be genuinely cheap and genuinely read-only, or you have solved the problem by adding a smaller copy of it.

What this chapter is not doing

Keys expire. The conventional retention is around twenty-four hours, which interacts with agents in a specific and unpleasant way: a run that pauses for three days waiting on a human approval outlives the window, and the "retry" that follows the resume is a brand-new request as far as the dedup layer is concerned.

Sizing that window, storing the keys, and the honest accounting of what "exactly once" can and cannot mean are Idempotency in Practice in Part XVI. What belongs here is the design: the key's provenance, the shape of the operation, the honesty of the replay result, and the paired read.

Atlas, concretely

The last row is the payoff worth stating explicitly. Four of Atlas's tools needed real work in this chapter and the rest needed none, because the read/write split already did the sorting. Idempotency is expensive to retrofit and nearly free to design in, and the difference between those two experiences is entirely whether you classified your tools before you shipped them.

Takeaways

• Three outcomes, not two: succeeded, failed, and unknown. Most timeouts happen on the response, so "unknown" leans toward it happened.
• You cannot make the network answer the question. You can make the question stop mattering.
• Two things repeat a call. Your infrastructure knows it is repeating; the model does not. From where it sits, calling again is a fresh decision.
• Therefore idempotency keys come from your code, never from the model. That is the same rule as authority, for the same reason.
• Derive the key from run ID, tool name, and a canonically serialized argument hash. Sorted keys: an unstable serialization is an unstable key.
• Different arguments must produce a different key. A model correcting a wrong amount is issuing a new operation, not retrying an old one. Mature payment APIs reject a reused key with changed parameters for exactly that reason.
• Dedup silently for infrastructure, never for the model. Return an explicit already_applied result, because the model reasons over what it reads and will otherwise believe it acted twice.
• Prefer declarative operations over incremental ones. set_priority('high') converges; increment_priority() compounds.
• External writes cannot be made naturally idempotent, because the dedup would live in someone else's system. Check the key before sending, and let the send be the only effect in its unit of work.
• Every irreversible write needs a paired read that confirms its outcome by reference. Without it, "unknown" is unresolvable.
• Agents repeat 15–30% of tool calls. This is a third of your write traffic, not an edge case.

A retry is only safe if whatever decides to retry was told enough to decide. Next: Errors as Instructions, because the string a tool returns is a prompt, read by something that has to recover from it.