← Field Notes
NOV 13 · Paper · via arXiv Human HandoffObservabilityReview

Rasoi shows 12 developers what a coding agent changed and why that matters

The developers who trusted the agent caught more mistakes. The ones who trusted themselves caught fewer. That is the design brief for every review view: earn trust with clarity, not with warnings.

FIG. 1 · Where this sits in Human Handoff
    26 NOTES ON HUMAN HANDOFF IN 26 WEEKS · SCRUB THE WEEKS, CLICK ONE TO PIN IT
    Published, annotated field notes attached to Human Handoff, by week observed · last 26 weeks · n = 26. Rail markers are lifecycle events from the pattern changelog.
    Data table
    Week ofNotes
    AUG 10A designer ran app maintenance through Claude for weeks from Slack
    AUG 24Karpathy's autoresearch runs experiments start to finish unattended · Linus Torvalds kept debugging the kernel after the AI gave up · Maggie Appleton says design, not the model, now sets the pace
    AUG 31CrabOS proposes an operating system that agents and workers share
    SEP 14GitHub used Copilot to rewrite a core part of Copilot itself · 20 workers show how reviewing finished agent work opens a reuse loop · 535 humans and capable AI agents still left gains on the table · OpenAI engineers hand Codex their code reviews and move on
    SEP 21Enterprise workers renegotiate their roles when agents join the team
    SEP 28NYT's Epstein Files agent cut 26,000 documents to 2,500 for reporters · 366-person study finds AI lifts confidence faster than skill
    OCT 12Kolab measured 115 hours of real work and found agents need humans · Hidden ads in search results steered what a shopping AI bought · Expertise shapes how far workers trust and delegate to AI · Researchers find AI interaction leaves measurable marks on the human
    OCT 19DoorDash ran 35,000 code changes through AI agents for $4.79 each · Atlas gives designers a map of what a long agent run did · HCI researchers map how shared workspaces change agent work · RCT finds pausing to reflect lifts AI-assisted writing quality · Model strength sets who leads, not screen design
    OCT 26Clinical AI panels reached agreement while hiding racial bias
    NOV 2arXiv paper maps the paths an agent considered but did not take · 420 papers on human-AI collaboration distilled into one live finding
    NOV 9Rasoi shows 12 developers what a coding agent changed and why that matters · Vaccari treats tests as a trust layer between humans and coding agents

    Machine summary of the source

    Researchers built Rasoi, a visual review tool that shows developers exactly what a coding agent changed in a codebase, using side-by-side before-and-after views of structure rather than raw code. A 12-person study found that the tool measurably improved how well developers judged the agent's work. That is the expected result. The surprising one is what drove it.

    Developers who trusted the agent more were better at catching its mistakes, not worse. And developers who felt confident in their own coding skills were worse at catching mistakes, not better. The tool seemed to let trust work in the right direction: a developer who trusted the agent paid closer attention to what it did, while a developer who relied on personal skill tended to skim.

    For teams building oversight into coding agents, this inverts a common assumption. The goal is not to make developers skeptical of the agent. The goal is to give them a view that is worth trusting, so their attention goes to the right places. Visual structure, not text line counts, appears to be that view.

    The summary above is generated; the note at the top is the editorial judgment. Primary source ↗