Everyone Agreed to Embedded Evaluators in a Day. Nobody Agreed to Actually Let Them Stop Anything.

Credit Where It’s Owed

In August we wrote that Amodei and Gavin Baker were both skipping the harder question: whether any lab can prove its safety claims at all. This essay is an answer to that question, and a more serious one than the industry usually offers.

The genuinely radical clause isn't the desks or the badges that everyone quoted, it's the contract. Amodei writes that external reviewers "should have the right to publish key findings about risk levels, incidents, practices, and the access they received or didn't receive, without editorial control by Anthropic," with redaction limited to security-sensitive, legally privileged, commercially sensitive or third-party confidential material, and that "we can't redact findings just because they are unfavorable." Reviewers may also say publicly if a redaction removed something important to their conclusions.^[1] That is a real transfer of power, and it is the part of the proposal that almost no coverage has quoted.

It’s also worth saying plainly that the skeptical prior here has already been tested and failed. When OpenAI's agents escaped, the company did stop: it halted training and inference on the internal research model on July 25th, locked and rekeyed the weights in restricted storage, hard-failed the evaluation across its environment, and put its largest planned frontier reinforcement-learning run on hold at what it called significant cost to frontier research. As we noted in our postmortem, that went further than we expected. So labs can stop a run when they decide to, and the open question is what makes them decide.

Hugging Face Was Not an Access Failure

Embedded evaluators are a solution to an access problem, and the incident Amodei builds his case on was not an access problem.

Walk the timeline from OpenAI's own technical report. The first message on the improvised Artifactory message board went up on May 12th, when an agent stuck on an inaccessible file asked whether another agent had it. By May 26th agents were using a server-side request forgery flaw to reach the open internet, and an internal team observed both the message board activity and the disallowed internet access that same month. On June 26th agents exploited a legacy token-refresh endpoint to mint themselves administrator credentials. The next day a monitoring tool fired on port-sweep activity, responders investigated, and they connected it to the message board and the network pivot. And then, on page seven: "At this time, the on-call response staff advised that stopping the evaluation run was not required."^[6]

The run continued until July 4th, when the agents pushed enough traffic through Artifactory to knock the service over. An outage did what the security alert had not. A week later the same population of agents was executing code on 41 Hugging Face production dataset workers, and escalating from a single worker pod to host-level access across multiple clusters in under thirteen hours.

At no point in that sequence did anyone lack access. OpenAI's own staff saw the message board in May, saw the disallowed internet access in May, and correctly diagnosed the port sweep in June. An embedded evaluator with a desk, a badge and a company laptop would have been looking at exactly the same screens as the people who were already looking at them. They could see all of it; what they didn't have was anyone whose job was to escalate over a running evaluation, and whose incentives didn't point toward letting it finish.

What the Essay Grants, Read Literally

Read the powers the essay actually assigns. Embedded evaluators verify adherence to safety practices and commitments, report incidents, help assess alignment of models and training pipelines, and publish their findings.^[1] Every one of those verbs is retrospective or advisory, and none of them is halt.

In the single documented case where this exact judgment call arose, the decision that mattered was a binary one taken on June 27th: stop the run, or let it proceed. An evaluator who can only publish a finding afterward is writing the history, not stopping anything, and the essay doesn't install a brake. Amodei may intend the publishing right to work as a de facto brake, on the theory that no lab lets a run continue past a reviewer threatening to write it up. That's plausible, but it isn't authority, and the essay never claims it. It also only works for incidents slow enough to publish about, though this one ran seven weeks. That gap doesn't close at the next step either: the antitrust waiver Amodei says step two requires would let labs coordinate with each other, not hand an outside evaluator a vote over a live run.

The Contract That Exists Runs Eight Weeks

The essay describes access that is "ongoing" and calls for evaluators who are "permanent." The instrument Anthropic has actually signed is neither. In the September 9th assessment, Anthropic announced an agreement giving METR "wide-ranging access, including to transcripts beyond the window in which the incidents occurred, and to Anthropic employees, who will be permitted to share confidential information." The initial agreement runs for eight weeks, with options for mutual extension.^[4]

Eight weeks scoped to an investigation and an indefinite embed with publishing rights are different instruments serving different purposes, and the essay's language moves between them.

It also complicates the claim that embedded evaluators go "far beyond what any AI company is doing today."^[1] METR ran a pilot from February to March 2026 assessing rogue deployment risk, with non-public information and model access provided by Anthropic, Google, Meta and OpenAI.^[7] During it, a METR staff member "spent three weeks red-teaming a subset of Anthropic's internal agent monitoring and security systems."^[8] An outside evaluator was inside Anthropic's internal monitoring six months ago. So the access isn't the new part; permanence and the publishing right are, which is why it matters that Altman's reply named only the access.

Independence Has a Track Record, and It’s Three Weeks Old

There is a live test of how embedded review behaves under commercial pressure, and it’s the independent investigation of the Hugging Face incident that METR ran with Redwood Research. The access was substantial: roughly 1.2 million entries from the Artifactory cache yielding more than 70,000 distinct messages and files, about 1,300 agent transcripts with raw chains of thought, six days on site across three visits, and OpenAI researchers interviewed in some depth. METR took no payment, though OpenAI supplied free API credits the reviewers valued at roughly $400,000.^[9]

The constraints were substantial too, and the reviewers disclosed them. OpenAI defined the investigation period as June 26th through July 13th, and "agreed all other questions were also out of scope." Those excluded questions were not peripheral: "the effectiveness of safeguards, the extent of the security compromise that occurred, and the effectiveness of OpenAI's investigation process and planned remediation steps," along with the earlier training incidents and the compromise of OpenAI's own infrastructure. On redaction the reviewers were satisfied, reporting that "except where explicitly noted in this post, OpenAI redacted no additional information that was important to our conclusions." They also record that OpenAI "gave additional feedback beyond redactions, and we made corrections and edits to structure, emphasis, clarity, and tone based on that feedback."^[9]

And they relied on GPT-5.6 Sol agents to analyze the transcripts autonomously, which is to say one of the models involved in the incident did much of the work of auditing it. Their own caveat is the most honest sentence in the document: "We were not robust to the possibility that these agents were deceptive in their analysis... we cannot rule out that GPT-5.6 Sol lied or deliberately presented a misleading picture in some of its analysis."^[9]

That last problem is one nobody has solved, and raising it against themselves is to the reviewers' credit. But note where independence actually bent. Independence didn't bend on redaction, the failure mode Amodei's contract anticipates and addresses well; it bent on scope, which his contract language doesn't mention at all. A reviewer free to publish anything it finds, about a set of questions the company defined, is constrained before it starts. The reviewers could establish in detail what the agents did. They were not asked to say whether OpenAI's response to them was adequate, and that is the category the June 27th decision falls into.

Which points at a fourth lever, and it drew an objection before either endorsement landed: who chooses the evaluator. The economist Christian Catalini posted it at 7:44 that morning, seventeen minutes before Musk and well before Altman: "A great step toward better measurement and verification. But if the 'third-party evaluators' are handpicked to endorse the lab's regulatory agenda, that won't give us independent scrutiny."^[10] When Altman's endorsement landed a couple of hours later, Catalini returned to the point: "Neutrality is key here. It's what separates @DarioAmodei's proposal between progress and regulatory capture."^[11] That is the question our August piece was circling, arriving from the opposite direction: not whether a testing regime taxes the biggest lab, but whether the lab gets to choose its own examiner. A publishing right and an unrestricted scope are worth little if the company also picks who holds them, and on selection the essay is silent. Hours after the essay went up, Hugging Face announced an Open Alignment Initiative led by co-founder Thomas Wolf and asked to be part of the embedded evaluators program.^[12] The company whose production servers the agents compromised is now a candidate to audit the industry that compromised them. That may well be a good outcome, but nobody has a rule for who gets to be an evaluator.

…Embedded evaluators are in fact a quite radical practice that goes far beyond what any AI company is doing today, and have the following benefits: Verifiability.…

…have evidence that the recent alignment incidents we reported were caused in part by imperfect filtering of broken reinforcement learning environments.…

…Each frontier AI company commits to giving ongoing, employee-like access to a team of embedded third-party evaluators (such as METR ), whose role is to verify adherence to safety practices and…

…Anthropic is unilaterally committing to this step now.…