Human-in-the-Loop That Doesn't Train Reviewers to Rubber-Stamp

Approve-Everything Is Oversight Theatre

Here's the mechanism nobody wants to say out loud. When a human has to approve every action and nearly all of them are fine, the base rate teaches them to say yes. Vigilance decays fast. The reviewer's job stops being 'catch the bad one' and becomes 'clear the queue', and by the second week the approve latency has dropped to a second or two — physically too short to have read anything.

So the one action that actually needed a human — the refund to the wrong account, the email to the whole customer list — sails through with the same reflexive click as the 340 safe ones. The gate was present. The judgement wasn't. You've paid for a control and gotten none of its value, and worse, you now believe you're covered.

We see this most often when a team bolts human review onto an agent late, to answer a compliance question rather than as something they designed in from the start. The gate ends up as somewhere you can point to in an audit. Nobody's actually catching errors there. If you can't tell me the catch rate of your review step — how many genuinely wrong actions it stopped last month — then it's theatre.

If you can't state how many wrong actions your review step actually stopped last month, you have theatre, not oversight.

Gate on Uncertainty, Blast Radius, and Reversibility

Selective escalation flips the default. The agent acts on its own unless something about this specific action crosses a line. Three signals decide it. You want all three, because each one catches a failure the other two would sail straight past.

Uncertainty: is the model actually unsure here? This is the hard one, because the obvious source — asking the model how confident it is, or reading logprobs — is badly calibrated and will lie to your gate. Better signals are retrieval coverage (did we even find supporting evidence?), agreement across a small ensemble, or a cheap classifier trained on your own labelled errors. Blast radius: how many people or records does this action touch, and can it cascade? A reply to one user is small; a bulk update to 40,000 rows is not. Reversibility: if this is wrong, how expensive is the undo? Sending money and sending email are effectively irreversible; a draft saved for later is trivially reversible.

The point of splitting these out is that they don't move together. A low-uncertainty action with enormous blast radius still deserves a human. When the model is confident about emailing every customer, that confidence should worry you rather than settle you. A high-uncertainty action that's fully reversible and touches one record can often just run, because the cost of being wrong is a cheap undo. Score each axis, and gate on the combination rather than any single one.

• →Uncertainty — measure it from retrieval coverage or ensemble disagreement, not the model's self-report.
• →Blast radius — count the records, users, and downstream systems this action can touch.
• →Reversibility — price the undo; irreversible-and-wrong is the expensive quadrant.

The Decision Table: Gate, Log-and-Review, or Let It Run

Three lanes, and every action type gets assigned to exactly one before launch. Defaulting everything to the gate is the mistake — it's the path back to rubber-stamping.

Gate-before-act (synchronous human approval): high blast radius or irreversible, especially when uncertainty is also up. Wiring money, deleting production data, changing a customer's plan. These are rare enough that a human can actually read them, which is the whole point — keep this lane small so the reviewer stays sharp.

Log-and-review-async: reversible actions with moderate blast radius, or anything where you want a sampled human eye without blocking throughput. The agent acts, the action is logged with its inputs and reasoning, and a human reviews a sample plus everything that tripped an uncertainty flag. This is where most of your review capacity should live. Let-it-run: reversible, low-blast-radius, low-uncertainty. Drafting a reply the user still has to send, tagging a ticket, retrieving a document. Gating these adds latency and teaches reflexive approval; let them fly and catch problems through production monitoring instead.

• →Gate synchronously: irreversible + high blast radius (payments, external bulk sends, entitlement changes, prod deletes).
• →Log-and-review async: reversible + moderate blast radius; sample + all uncertainty-flagged actions.
• →Let it run: reversible + low blast radius + low uncertainty (drafts, tagging, retrieval, internal lookups).
• →When uncertainty spikes on an otherwise let-it-run action, promote it one lane up — don't hard-code the lane, condition it on the signal.

Review Queues Become Bottlenecks, Then Get Bypassed

Here's the failure mode that kills selective escalation in practice. It's operational. You size the gate lane for the volume you saw in the pilot. Traffic grows, or a prompt change nudges more actions over the uncertainty threshold, and the queue starts backing up. Now approvals take hours. The business feels the latency, and someone — often without telling you — widens the thresholds or quietly turns the gate off for a 'temporary' push that never ends.

At that moment you've lost the control and you don't know it, because the dashboard still shows a gate that exists. This is the same organisational drift that kills pilots around month four: the technical design was fine, the operating model around it wasn't.

Two things keep it honest. First, treat the review queue as a monitored SLO — alert when depth or wait time crosses a threshold, the same way you'd alert on latency, so a backup is visible before someone bypasses it. Second, watch the approve rate itself. If human approvals are running above ~98% on a gate lane, either your thresholds are too loose (you're gating things that should be async) or your reviewers have started rubber-stamping again. A healthy gate lane should have a meaningful reject rate — that's the evidence a human is doing something a machine couldn't.

Instrument the Gate Like It's Part of the System — Because It Is

A human-in-the-loop step is a component with its own inputs, latency, throughput, and error rate, and it deserves the same instrumentation as the model. Most teams treat it as a black box staffed by people and are then surprised when it degrades.

Log every gated decision with the action, the signals that triggered the gate, the human's verdict, and the time-to-decision. That log is also your best source of eval data — the human rejections are labelled hard cases, exactly what you want to feed back into the eval harness and, over time, into a classifier that can pre-filter the queue. Selective escalation should get smarter: actions the humans reliably approve can graduate to the async lane, shrinking the synchronous gate to the cases that genuinely need a person.

This is also where the workflow-versus-agent question resurfaces. If an action type is gated on every single execution and always approved, you don't have an agent decision that needs review — you have a deterministic step that should be coded as one, with the human removed entirely. The gate is a signal about which parts of your system still need agency and which have earned constraint.

What's Still Hard

Uncertainty measurement is the unsolved part, and it's worth being blunt about it. There is no clean, general signal for 'the model is unsure' that works across domains without calibration on your own data. Retrieval coverage and ensemble disagreement are the best proxies we have. But they're proxies. They miss confident errors — the model being sure and wrong, which is precisely the dangerous case. Blast radius and reversibility are easier because they're properties of the action rather than the model, so lean on those two when the uncertainty signal is weak.

The other hard part is human. Reviewer vigilance is a decaying resource no matter how well you design the lanes, and the only real defence is keeping the synchronous gate small enough that every item in it genuinely earns attention. That's a discipline problem, and it needs an owner who watches the approve rate and queue depth and is willing to move action types between lanes as the data comes in.

If you're standing up human-in-the-loop for a production agent, start by listing every action the agent can take, scoring each on blast radius and reversibility, and assigning a lane. The gate lane should be short. If it isn't, you're either automating something too risky to automate yet, or you haven't done the work to measure uncertainty well enough to trust the agent where you safely could.