Reviewers who know the trick still click yes, two studies find

The assumption

Most teams assume that adding an approval step puts a human in the loop. Ship the gate, add a review screen before the agent commits, wire up an interrupt — done.

Two findings from this week say otherwise. A study on dark patterns showed that humans comply even when they know they're being manipulated. A separate note on oversight showed that review steps wear down in production: volume goes up, nothing bad happens for a while, and humans stop looking. So you can have a gate, and a human at the gate, and still have no real oversight.

Why this is dangerous now

On its own, that's a behavior problem. The rest of the week's evidence makes it a safety problem, because agents are stacking up the exact conditions that make careless review expensive.

A security note on exploit-finding agents named the worst combination: broad permissions, no approval gate, and an attacker-reachable context window. Any one of these is manageable. Together they hand an attacker the keys.

A second note pointed out that permission screens show users half the picture. They show what the agent could do. They don't show what it has already done with that access, or how much is left before you'd want to step in.

And a panel note on multi-agent setups raised the sharpest case: agents reviewing other agents. If the reviewer shares training or tools with the agent it reviews, you haven't added independent judgment — you've added a second copy of the same judgment, dressed up as a check.

The real flaw: all the oversight happens at the start

These failures share one design choice. Our approval and review tools do their work at kickoff — the moment you hand the task over, when you're paying attention and the stakes feel real. They do almost nothing to stay useful while the agent runs longer, acts more often, or works in a team.

The MCP interrupt primitive and protocol-level checkpoints are promising for exactly this reason: they move review into the middle of the task instead of piling it all at the start. But they open design questions nobody has answered yet. What does a good waiting state look like? How do you stop a mid-task request from being dismissed as noise?

Tools can be fixed. Defaults are harder.

Part of this is a tooling problem, and tooling problems get solved: better audit trails, spend budgets shown next to permission grants, interrupts that save state. The harder problem is defaults. Oversight doesn't fail with a decision — nobody chooses to stop watching. It fades. Each week, a little less counts as worth reviewing.

The unsanctioned multi-agent incident showed where that leads. Every oversight pattern we have assumes a human is already watching. None of them cover the case where the agent should raise the alarm itself because nobody has looked in a while. That pattern doesn't exist yet. We need it.

Design for month six, not day one

If you're building an approval or review surface right now, the question isn't whether the gate exists on day one. It's what keeps the gate meaningful in month six, when the novelty is gone and approving is a habit. Treat the worn-down state as the normal state, and design for it.

Effect limits that tighten as the agent's actions pile up. Reviews that escalate when the agent is genuinely unsure, not just when a number crosses a line. And interrupts that tell you why the agent stopped — “I need permission” and “I'm stuck” are different situations and need different handoffs. The Torvalds debug session made that concrete: same pause, completely different next step.

A gate that never pushes back teaches humans it doesn't matter. After enough approvals, they're right — and that's the day it stops being oversight.