AI experts call for independent safety evaluators to get protected access to frontier models

The letter is titled "Minimum Conditions for Embedding Evaluators." Signatories include AI researcher Geoffrey Hinton and members of Johns Hopkins University, Stanford University and the nonprofit evaluator METR. The signers asked foundation model providers to ensure that embedded third-party evaluations have "scientific objectivity, transparency, independence, and robust protections," arguing that those conditions are necessary for the work to be effective and credible. They also want evaluator work conducted independently from the businesses, more transparency about the technologies, and protection "from retaliation from the companies they embed with."

Conrad Stosz, chair of the AI Evaluator Forum, said in an interview that the coalition is not advocating one particular way to ensure safe AI development but wants basic principles and greater standardization for evaluators and others working independently of major labs. He said the effort seeks to hold foundation model companies to recent pledges to support more thorough third-party AI safety testing.

The letter follows Anthropic CEO Dario Amodei's proposal over the weekend to give some evaluators "employee-like access" to inspect and audit bleeding-edge foundation models and their development processes. Stosz said Amodei's proposal appears to involve significantly more access than evaluators have previously had, potentially including access to company computers, candid conversations with employees and sensitive internal data and unreleased systems. Such access, he said, would give greater confidence and certainty about actual risk, particularly for systems used internally and not released. He cited the unreleased OpenAI model used in the Hugging Face attack.

OpenAI CEO Sam Altman, SpaceX's Elon Musk and Microsoft CEO Satya Nadella have publicly supported Amodei's proposal, but they have yet to address logistical issues such as which AI evaluators would be selected and how deeply they would inspect closely guarded technologies. Some industry leaders have called for government regulation of AI development, while President Donald Trump and his former AI czar, David Sacks, have adamantly opposed such efforts.

Vinh Nguyen, a Council on Foreign Relations senior fellow for AI and former chief AI officer of the National Security Agency, said independent evaluators are needed to uncover information that could help mitigate potential security failures and economic calamities. "When a few powerful labs control capabilities that can endanger the cybersecurity, critical infrastructure, and the systems our national security and economy run on, the government and the public cannot be dependent on those labs' own account of what's secure and safe," Nguyen, who signed the letter, said in a statement.

Stosz said third-party evaluators are not intended to "be a replacement for any internal efforts to evaluate." He acknowledged that foundation model companies could ignore the public letter and its call to action, but said their credibility is at stake. "There's a very small number of groups that are actually sufficiently technically credible and have the scale and the ability" to perform the kind of work, he said.

The letter states that all frontier AI companies should embed evaluators to independently assess AI risks, including the systems themselves and significant incidents of real-world harm, as well as the companies' training, deployment, oversight, operational and safeguard practices. To be credible, it says, embedded third-party evaluations must have scientific objectivity, transparency, independence and robust protections.