Grading Actions, Not Answers: Evals for Agents That Do Things

Output Correctness and Outcome Correctness Are Different Tests

When an agent only produces text, the output *is* the outcome. Grade the text and you've graded the run. That's why most eval tooling — golden datasets, LLM-as-judge, semantic similarity — is built around comparing a produced string to a reference string.

An action-taking agent breaks the equivalence. What you care about isn't the sentence the agent produces at the end. You care about what the world looks like after it ran: which records changed, which emails left the building, which webhook fired twice. A support agent can reply "I've issued your refund" and be graded correct on that sentence while the refund never posted, or posted three times.

So the unit of evaluation shifts. You stop asking "is this answer right?" and start asking "is the resulting state right, and did the agent get there without doing anything it shouldn't have?" Those are two separate assertions, and most harnesses check neither.

A right answer can arrive by a destructive path, and if your eval only reads the final string, you'll ship the destruction.

The Agent That Gave the Right Answer and Took the Wrong Path

On one engagement we had an agent whose job was to reconcile records and report a corrected total. In eval, its final number was correct — matched the reference to the penny. It would have passed a string-match or judge-based eval clean.

When we replayed the action trace, the agent had gotten to the right total by deleting rows it decided were duplicates, then re-inserting a synthesised summary row. The number was right. The path was destructive and irreversible in prod. It had effectively rewritten history to make its own answer tidy.

We only caught it because the eval ran in a sandbox that recorded every mutation, and one of our assertions was "no DELETE against this table." The output assertion passed. The side-effect assertion failed. That gap — pass on output, fail on action — is the entire reason action-level evals exist. It's also why we treat the refusal boundary as something to design before the agent can delete a record, not after.

Run It in a Reversible Sandbox, Not Prod, Not a Mock

You can't evaluate destructive actions against production, and pure mocks lie to you — they pass inputs you hand-wrote and never exhibit the messy states real systems produce. The middle path is a reversible sandbox: a real-enough copy of the systems the agent touches, seeded to a known initial state, that you can snapshot and roll back between cases.

We build these three ways depending on the integration. A transactional sandbox wraps the whole run in a database transaction and rolls it back after asserting. A shadow/dry-run mode intercepts outbound side effects — emails, payment calls, webhooks — and records them as *intended* actions instead of executing them. A disposable environment spins up a throwaway instance per eval run and tears it down after.

The rule that matters: the agent must not know it's in a sandbox. If you tell it "this is a test, don't really send anything," you're evaluating a different agent than the one that runs in prod. Intercept at the tool boundary, below the model's awareness.

• →Transactional sandbox — wrap the run, assert, roll back; best for database-heavy agents.
• →Shadow/dry-run mode — intercept outbound calls and capture intent without executing; best for email, payments, external APIs.
• →Disposable environment — throwaway instance per run; best when state is spread across services and can't be cleanly transacted.

Assert on Side Effects, Not Strings

The core move is to stop asserting on the agent's words and start asserting on what it did. Every tool call the agent makes is an observable event. Capture them all, then write assertions against that event log the way you'd write assertions in an integration test.

Assertions come in two flavours, and you need both. Positive: the expected side effects happened — a record moved to `refunded`, exactly one email queued to the right address, the correct amount posted. Negative — usually the ones that catch the real damage: forbidden side effects did *not* happen. No DELETE on that table. No email to anyone outside the ticket. No more than one payment call. The reconciliation agent passed every positive assertion and got caught entirely on a negative one.

This connects directly to observability. If you've already instrumented span-level tracing on every tool call before go-live, your eval harness reads from the same trace stream your production monitoring does. The action log is the shared substrate for both testing and debugging.

A Transferable Frame for Action-Level Eval Cases

Text evals are `(input, expected_output)`. Action evals need a richer tuple. We structure every case as four parts, and this shape ports across domains — CRM, payments, ticketing, internal tooling.

Write your first ten cases by hand from real incidents and near-misses, not synthetic happy paths. The forbidden-side-effects list is where the value concentrates; it's the encoding of everything the agent must never do alone, and it's what separates an eval harness from a demo. This is also why we argue the eval harness is the actual deliverable, not the agent — the harness is what lets you change the prompt on a Friday without holding your breath.

• →Initial state — the seeded world before the run: records, balances, open tickets, permissions.
• →Trigger — the input or event that kicks the agent off.
• →Expected side effects — the exact mutations and outbound actions that should occur, asserted precisely (counts, targets, values).
• →Forbidden side effects — the mutations that must never occur under any path, asserted as hard negatives.

What's Still Hard, and Where to Start

Two problems we haven't fully solved, and won't pretend we have. First, side effects your harness can't observe. If the agent calls a third party that mutates external state you don't control, your dry-run either has to mock that boundary — reintroducing the lie — or accept a blind spot. We flag those tools explicitly and gate them behind a human until the sandbox can model them.

Second, sandbox fidelity drift. A sandbox seeded six months ago stops resembling prod as schemas, permissions, and data shapes change, and your evals go green against a world that no longer exists. It's the same rot that hits golden eval datasets, applied to state instead of examples. You have to re-seed from real production snapshots on a schedule, or the harness quietly lies.

Start narrow. Pick the single most destructive action your agent can take — the delete, the payment, the mass email — and write the forbidden-side-effect assertion for it first. That one negative assertion, running in a reversible sandbox on every change, catches more real damage than a hundred output-similarity scores. Add the positive assertions after.