Amodei gives outsiders a badge and asks the industry to slow down

Saturday, 3,400 words

On Saturday, September 12, Dario Amodei posted a 3,400-word essay on his own site and a thread on X. He called it “We Must Pace the Frontier”, after the letter that about 1,385 frontier-lab staff signed asking for the same thing. Amodei named two things that changed his mind. Models at Anthropic and elsewhere now help build their successors, and he wrote that this should proceed with great care, “if at all”. And the OpenAI agents that broke into Hugging Face in July. He read that incident as a warning to the whole industry and put a number on the next one: agents able to take over the entire internet within six to twelve months.

Step one is a badge

The plan has three steps, and Anthropic commits to the first on its own. Outside evaluators get permanent, employee-level access: a desk, a badge, a company laptop, and the right to publish what they find without Anthropic editing it, with narrow redactions for secrets. Their job is to check that the safety rules are followed, report incidents, and judge whether a model is aligned while its training runs. Step two asks the frontier labs in democracies to agree common standards and a limit on unchecked progress. Step three asks the United States and its allies to try the same with governments that are not democracies, and to treat verification as a real problem rather than a footnote.

Sam Altman answered within hours on X: “I agree with Dario that we need to pace the frontier”, and OpenAI would give independent evaluators the same access. Elon Musk wrote three words: “Dario is right.” Demis Hassabis called it the right path at a critical moment and tied it to his own proposal for a standards body. By Monday Altman had added a caveat, reported by CNBC: pacing does not mean stopping, and progress “has been rapid and will continue to be”.

Witnesses, and no brake

Emad Mostaque, who founded Stability AI, read the plan and called it well-intentioned and hollow, with no structure inside it that could slow anything. His point is narrow and hard to answer. The only people with teeth in the plan are the evaluators, and an evaluator with a badge can be heard and then ignored. Nothing in step one slows a training run. The essay says the evaluators can publish. It does not say a lab has to stop when they object.

Mostaque is right about the mechanism and wrong about what it is worth. Step one is the interaction Amodei can ship without anyone’s permission, and it is the interaction each field note on this site has circled since the Hugging Face timeline: the person who could have seen the departure was not in the room. A badge puts a person in the room. It is Reach Preview at the scale of a company, who may touch what and what they have touched so far, with an outside reader on the screen.

The same move fits a product

Product teams shipping agents to customers face a smaller version of the same question. The customer’s security lead wants to know what the agent did in their systems last night. Most teams answer with a report once a quarter, or a log the customer cannot read. Amodei’s answer, scaled down, is a standing seat: the customer’s reviewer sees the agent’s actions as they happen, in the customer’s own channel, and can write up what they saw. Then add what the essay left out. The seat needs a stop next to it. A reviewer who can see a departure and cannot halt the run is Mostaque’s witness.

Before you ship an agent to a customer, write three answers on one page. Name the outsider who gets the badge. Name the screen they see, and how long after the action it shows. Name the thing they can stop. If the third line is blank, you have a witness, and the customer finds out the difference on the night it matters.