← Field Notes
NOV 11 · Clipped · via Martin Fowler Eval Set AuthoringHuman HandoffReview

Vaccari treats tests as a trust layer between humans and coding agents

Vaccari splits the agent's output in two: the code, and the proof the code works. That split is what lets a developer judge without reading everything. Design your coding agent to produce both, visibly apart.

Read the source ↗ attached to Eval Set Authoring →
watching

watching → emerging → established

A candidate behavior, not yet a published pattern. Early signal — worth tracking, not yet worth standardizing on.

12 documented sightings since Jul 2024 — every one traces to a dated source.

See the evidence →
FIG. 1 · Where this sits in Eval Set Authoring
    3 NOTES ON EVAL SET AUTHORING IN 26 WEEKS · SCRUB THE WEEKS, CLICK ONE TO PIN IT
    Published, annotated field notes attached to Eval Set Authoring, by week observed · last 26 weeks · n = 3. Rail markers are lifecycle events from the pattern changelog.
    Data table
    Week ofNotes
    OCT 19Annotrace gives agent trace labelling a repeatable standard
    OCT 26EnvOS finds that refreshing the screen beats reminding the agent
    NOV 9Vaccari treats tests as a trust layer between humans and coding agents

    Machine summary of the source

    Matteo Vaccari proposes a design shift in how developers work with AI-generated code. He frames the developer as a customer of the agent's output, not a co-author. That reframing matters: it changes what the developer needs to see and when.

    Vaccari's answer is to treat tests as data, a separate record that the agent writes but that you can read, audit, and run independently. The tests sit outside the agent's own judgment. They let you check whether the agent did what you asked, without reading every line of code it wrote. That is a different kind of oversight from code review.

    For teams building coding agents, this surfaces a real design question. The agent produces two things: the work and the evidence that the work is correct. Those two things need to stay legible and separate. If the agent controls both, the developer cannot tell whether the evidence is honest. Vaccari's pattern keeps the developer in a position to judge, even when the code itself is too dense to inspect directly.

    The summary above is generated; the note at the top is the editorial judgment. Primary source ↗