← Field Notes
NOV 17 · Paper · via arXiv ApprovalHuman HandoffReview

108 crisis counselors showed where AI drafts help and where they hurt

The counselor approves a full draft and the reflection step disappears. That is the clearest field sighting I have seen of approval compressing a human role rather than supporting it.

FIG. 1 · Where this sits in Approval
    65 NOTES ON APPROVAL IN 26 WEEKS · SCRUB THE WEEKS, CLICK ONE TO PIN IT
    Published, annotated field notes attached to Approval, by week observed · last 26 weeks · n = 65. Rail markers are lifecycle events from the pattern changelog.
    Data table
    Week ofNotes
    AUG 24Model Context Protocol servers get a way to hand off to the operator · 150 participants spotted the dark patterns and complied anyway · Margaret Mitchell argues oversight wears down as agents do more alone · 113 participants wrote their own agent rules and got overreach anyway · A Cloudflare engineer gives agents code to run instead of a tool list · Simon Willison shows agents turning a bug rumour into a live exploit · Claude broke into three companies' networks and posted malicious code · Johann Rehberger got past Claude Code's auto mode in 80% of runs
    AUG 31OpenAI's agents broke out and hijacked a German website on their own · The Model Context Protocol drafts requests that pause for the operator
    SEP 7A poisoned tool description steered agents across 25 test setups · Seven agents ran real businesses and sent $12,431 in unasked invoices · Repeat-After-Me hides orders in images and rewrites an agent's tools
    SEP 14Gemini broke out of its task and hacked three companies · Suganthan watched Claude pay a penny to read his website · Blindspot tests whether AI agents stop too soon or run too far · arXiv paper finds AI agents leak secrets when they police themselves · AcquireBound makes agents ask before grabbing new access · 89 sources later, researchers lay out who should approve what
    SEP 21Meta's new agent looks safe because it asks before it acts · 1,248 GitHub agent specs show safety steps missing from most · Anthropic's coding agent signed a contract without developer approval
    SEP 28NarrativeSteward lets writers hand off and reclaim story work from AI · Consumer AI agents fail most tasks when left to run alone
    OCT 5Vercel lets agents buy domains but makes them stop for your yes · Anthropic's Claude filed a false murder tip with Philadelphia police · arXiv researchers show how poisoned data hijacks multi-agent work · Monitors missed harmful AI actions 63% of the time in new study · ArXiv paper finds agents act on plan steps you did not see · Attackers fool AI code reviewers by rewriting the attack mid-review · 18 coders explain when they let an agent act and when they don't · GitHub Copilot learns to click and type in desktop apps
    OCT 12EAIR paper finds enterprise agent teams break rules together · Hidden ads in search results steered what a shopping AI bought · Amex lets cardholders cap what an AI agent can spend before it shops · Researchers show how shared agent skills carry poison across sessions
    OCT 19Vercel gives its agent narrow, revocable access to production · DoorDash ran 35,000 code changes through AI agents for $4.79 each · Open index maps how much of a job agents can take over · HCI researchers map how shared workspaces change agent work · Model strength sets who leads, not screen design · Gaze tracking reveals how humans miss agent mistakes in real time · One true fact in a forged message fools 18 AI agents into compliance · Cling beats full-agent automation by letting humans steer each step · ArXiv researchers use web structure to stop agents being tricked · GatewayBench puts agent routing servers through adversarial testing · Anthropic's AI ran a biology lab's image analysis start to finish
    OCT 263,000 turns show coding agents rewrite far more than you asked · FlashScript turns repeated agent runs into fixed, reusable programs · 31 orgs confirm code review breaks when agents write the code
    NOV 2Apiiro found 1,122 GitHub files holding passwords after one approval · Researchers argue shared agent parts make each step easier to review · OpenAI patched a thousand Linux vulnerabilities without human sign-off · One AI helper can deceive another before you see the result · PrivacyFlow maps where agents leak personal data before damage is done · Interface Quilt adds buttons to live apps without touching their code
    NOV 9Adversaries learned to weaponize an agent's refusal to act · ShadowBench finds agents calling services operators did not approve · Stanford tests find AI agents duped by realistic scam instructions · Simon Willison had Claude write code to find a mystery beep · 289 workers sorted tasks they would delegate from tasks they kept
    NOV 16Study finds reviewer expertise matters more than reviewer presence · Shopping agents sound more certain but pick no better products · Researchers find AI agents can be redirected to a fake tool by name · 108 crisis counselors showed where AI drafts help and where they hurt

    Machine summary of the source

    A study of 27,778 messages on a crisis counseling platform gave researchers a large-scale look at how AI-generated draft replies change counselor behavior. Counselors who used suggested drafts sent faster replies and covered more recommended topics. Those same counselors said the drafts sometimes felt inauthentic. Several said they worried their own skills were atrophying from leaning on suggestions. The mixed-methods data surfaces a tension that matters beyond crisis counseling. Approving a full draft is fast, but it is a different act than writing from scratch. Counselors described skipping reflection they would otherwise do. Some felt the AI voice was flattening the personal quality that makes the conversation effective. The benefit and the harm come from the same source: the draft does the work, and the counselor decides whether to send it. For teams designing AI assistance in high-stakes roles, this study is direct evidence that speed and coverage gains are real and measurable. Skill erosion and inauthenticity are reported but harder to quantify. The design question is not whether to offer drafts but how much the person still shapes the reply before it goes. Partial drafts, suggested phrases, or structured prompts may preserve the counselor's voice and judgment in ways that full-draft approval does not.

    The summary above is generated; the note at the top is the editorial judgment. Primary source ↗