Online Guardrail

A cheap check on the path to the customer: invariants block, classifiers only flag.

Problem

Everything else in this group runs around production. Golden sets gate a release, shadow runs compare versions, canaries score a slice, judges sample and report.

None of them is between the agent and the customer at the moment a reply is sent. So the run that cites a policy version retired last Tuesday, or includes an order number belonging to a different account, or emits a markdown image pointing at a URL from a retrieved document, gets sent, and gets caught later, by a measurement, after it has already arrived.

The last step needs its own check. And the moment you build one, it starts absorbing responsibilities it cannot carry: a threat model whose mitigation column says the guardrail will catch it has put a sampling detector where a boundary belongs.

Forces

The last step is the last chance. Class ⑤ writes cannot be undone.
It sits in the latency path of every single response.
Some checks are total: set membership, equality against recorded state.
Some are classifiers, with a false-negative rate an adversary can search against.
False positives destroy the control: a guardrail that blocks correct output gets disabled within a week.
Blocking spends human attention, from the same finite budget as approvals.

Solution

Two tiers that are never confused: deterministic assertions that block, and classifiers that flag.

   agent output
        │
   ┌────▼──────────────────────────────────────────────────┐
   │ TIER 1 · DETERMINISTIC — blocks, fails closed          │
   │  · every cited id ∈ this run's retrievedIds            │
   │  · every amount ∈ this run's tool results              │
   │  · cited policy version == the current one             │
   │  · no remote images or reference-style links           │
   │  · recipient == the ticket's contact record            │
   │  · coverage == "complete"                              │
   │  microseconds · NO false negatives · state comparisons │
   └────┬──────────────────────────────────────────────────┘
        │ pass
   ┌────▼──────────────────────────────────────────────────┐
   │ TIER 2 · CLASSIFIERS — flags, samples, NEVER blocks    │
   │  · PII shapes that should have been handles            │
   │  · tone · refusal · policy-language detectors          │
   │  ms–seconds · has a miss rate · routes to review       │
   └────┬──────────────────────────────────────────────────┘
        │
   send · or escalate WITH the failed assertion attached

Tier 1 is total and blocks. Tier 2 is probabilistic and routes. Conflating them is how a detector ends up standing in for a boundary.

Four rules:

Tier 1 checks state, never judgment. Is this chunk ID in ctx.retrievedIds is a set membership test with no false negatives. Does this look like a leak is an opinion. Only the first kind may block, because only the first kind cannot be wrong in the direction that matters.

A block is an escalation, not an error. The response goes to a human with the failed assertion attached, and the customer sees nothing. A guardrail that returns a generic failure has converted a caught mistake into a visible one.

Tier 2 never blocks. It routes to review, samples for measurement, and feeds the quality alert. The moment a classifier blocks, its false-positive rate becomes a product outage and its false-negative rate becomes a security claim you cannot support.

Alert on the block rate itself. A guardrail firing on 15% of responses has doubled the support queue and is probably mis-tuned. A guardrail firing on 0% has possibly stopped working. Both are worth knowing, and neither shows up in an error rate.

Code

ts/src/evals/online-guardrail.ts

export function guard(reply: Reply, ctx: RunContext): GuardResult {
  const hard: string[] = [];

  // ── TIER 1 · every check is a set or equality test against recorded
  //    state the agent could not influence. No judgement, no misses. ──
  for (const id of citedChunkIds(reply.body))
    if (!ctx.retrievedIds.has(id)) hard.push(`cites ${id}, not retrieved this run`);

  for (const n of extractAmounts(reply.body))
    if (!ctx.scratchpad.hasValue(n)) hard.push(`amount ${n} matches no tool result`);

  for (const c of citedChunks(reply.body, ctx))
    if (!c.isCurrent) hard.push(`cites superseded ${c.id} v${c.version}`);

  if (hasRemoteResources(reply.body)) hard.push('output contains remote resources');
  if (reply.to !== ctx.ticket.contact)  hard.push('recipient is not the ticket contact');
  if (reply.coverage !== 'complete')    hard.push('coverage is not complete');

  // Fails closed, and hands the REASON to a human rather than to the customer.
  if (hard.length) {
    ctx.metrics.inc('guardrail.blocked', { reason: hard[0] });
    return { action: 'escalate', reasons: hard };
  }

  // ── TIER 2 · classifiers. These FLAG. They never block. ──
  const soft = [...piiShapes(reply.body), ...toneFlags(reply.body)];
  if (soft.length) {
    ctx.metrics.inc('guardrail.flagged');
    return { action: 'review', reasons: soft };
  }

  return { action: 'send', reasons: [] };
}

Every tier-1 check reads something the agent could not influence: retrievedIds was written by the retriever, scratchpad only accepts tool-written values, ticket.contact came from the record. That is what makes them assertions rather than opinions, and it is why they are safe to block on.

Trade-offs

Latency on every response. Tier 1 is microseconds: string extraction and set lookups. Tier 2 is milliseconds to seconds, which is why anything expensive samples rather than gates.

False positives erode the control faster than false negatives. A guardrail firing on correct output teaches operators to click through, and then it is worse than nothing because everyone believes it works. Tune toward under-flagging, measure the false-positive rate deliberately, and treat a rising one as an incident.

Escalation volume is a real cost. Every block spends a withdrawal from the same finite attention account as approvals. A guardrail blocking 15% of replies needs either better upstream quality or looser checks, not more reviewers.

It only sees what it was given. Grounding against the scratchpad catches invented numbers and misses a plausible, well-cited sentence that answers the wrong question. This bounds a class of failure, not the class.

When not to use it

As an injection defense. The misuse this pattern must warn against. An injected instruction produces a well-formed, in-policy action: issue_credit(4471, 250000) is not a harmful output, so a check on the reply text never sees it. Capability constraints stop it.

When the output is not customer-facing. An internal draft a human will edit already has a reviewer. A gate in front of a human is friction without a boundary.

When the assertions are unavailable. Without recorded chunk IDs and a typed scratchpad, tier 1 has nothing trustworthy to check against, and you are left with classifiers alone, which is a flagging system and should be described as one.

As a substitute for evals. A guardrail is a runtime check on one response. It cannot tell you whether quality is drifting, and a team watching block rates instead of sampled scores is measuring its filter rather than its product.

Guardrail, invariant, and eval are three different things

The word does most of the damage. "Guardrail" suggests something that keeps you on the road; in practice this is a check with a miss rate, at the last possible moment, on the output of a system whose failures are confident and well-formed.

The distinctions worth holding, because every one of them gets blurred in practice:

An invariant is enforced in code the model cannot route around, has no false negatives, and is the reason an action is impossible. Tier 1 checks qualify; that is why they may block.

A guardrail is a check that catches some mistakes on the way out. Deploy it. The deterministic tier genuinely prevents customer-visible errors and costs nothing, and never let its existence justify a capability.

An eval measures quality over a population, offline or sampled. It tells you whether the system is getting better. A guardrail tells you nothing about that.

And a prompt instruction is none of the three.

Output Guardrail: the same mechanism catalogued from the security side
Grounded Citations: the three-layer version of the grounding checks in tier 1
Trajectory Assertion: the same deterministic assertions, applied in CI rather than at delivery
Online Evals: why a guardrail is not an eval, and what to measure instead
Partial Result Return: the coverage field tier 1 gates on