Anthropic gives six METR researchers employee-level access to assess its models

Anthropic said in a blog post on Thursday that it is giving employee-level access to six researchers from METR, a nonprofit that tests frontier AI models. Their job is to independently assess Anthropic's models, safety practices and risk management.

The announcement lands at a tense moment. Incidents of AI agents acting outside their instructions have been disclosed every week or two since late August, and the incidents have sharpened calls for independent checks on AI companies. Anthropic wrote: "We want to be publicly accountable for the decisions we’re making," and added that it aims to close the gap between what evaluators understand about a frontier AI company's systems and what the company itself understands.

In practice, the METR team has Anthropic-issued company laptops, office space, and access to internal tools roughly on par with what Anthropic's own risk assessment teams use. Anthropic says its employees are encouraged to share information with the METR researchers just as they would with a fellow employee working on alignment risk. Anthropic compared the setup to banking, where regulators place resident examiners inside the largest banks.

The key term is publication. METR will publish its own assessments of Anthropic's risks, incidents and practices, and the company has given up the right to block them. Anthropic can, however, require redactions if information falls into narrow, pre-defined categories, such as customer data, anything attorney-client privileged, or information about its model architectures. METR is free to characterize the redactions publicly and say whether it thinks they were justified, as long as it does not reveal the redacted information.

METR sits at the top of a three-tier system Anthropic is building, and it is the only evaluator there so far. Bottom-tier evaluators, such as the biosecurity nonprofit SecureBio, run narrow reviews and need Anthropic's approval to publish. The middle tier gets more information and can publish without approval. The top tier adds the employee-level access METR now has.

The move makes good on a promise Anthropic CEO Dario Amodei made in a September essay, where he proposed embedding outside evaluators inside AI labs as a first step toward slowing the technology's development. OpenAI quickly pledged the same, and both companies, along with Google, Meta, Nvidia and xAI, have since signed a White House accord committing to independent review.

Key facts

Anthropic is giving employee-level access to six METR researchers, with laptops, office space and internal tools roughly on par with its own risk assessment teams.
METR will publish its own assessments of Anthropic's risks, incidents and practices; Anthropic cannot veto publication.
Anthropic can require redactions only for narrow, pre-defined categories such as customer data, attorney-client privileged material or model architecture details, and METR may publicly characterize them.
METR is the only evaluator so far at the top of a three-tier system; bottom-tier evaluators like SecureBio need Anthropic's approval to publish.
The step follows a September essay by CEO Dario Amodei; OpenAI pledged the same, and Anthropic, OpenAI, Google, Meta, Nvidia and xAI have signed a White House accord on independent review.

Why it matters

Outside evaluators of AI models usually see only what a company chooses to show. Anthropic says it wants to close the gap between what evaluators understand about a frontier company's systems and what the company itself understands. Here the evaluators get the same kind of access as staff, and the right to publish without a veto. Anthropic frames it as the AI equivalent of resident examiners inside the largest banks. It also arrives while incidents of AI agents acting outside their instructions have been disclosed every week or two since late August, which has sharpened calls for independent checks. And it fulfils the embedding idea Amodei floated in a September essay.

Who it affects

Most directly, the six METR researchers and Anthropic's own staff, who are encouraged to share information with the METR team as they would with a colleague working on alignment risk. Other outside evaluators are affected through the tier system: SecureBio is named as a bottom-tier example, and the middle tier is described only by its terms. OpenAI pledged the same, and Google, Meta, Nvidia and xAI have signed the White House accord on independent review alongside Anthropic and OpenAI, so the same pressure applies to them.

How to use it

This is an access arrangement, not a product, so there is nothing to adopt. For readers who follow AI safety, the practical thing to watch is METR's own publications about Anthropic's risks, incidents and practices, and any public comment from METR on whether Anthropic's redactions were justified. Other labs and evaluators can read the three-tier structure as a template: narrow reviews with approval to publish at the bottom, more information and free publication in the middle, employee-level access at the top.

How solid is it

The facts come from Anthropic's own blog post as relayed in the article, so the terms are the company's description of its own arrangement. The core details are specific and consistent: six researchers, a named nonprofit, a no-veto publication rule, defined redaction categories and a three-tier structure. No METR statement or reaction is quoted, and the six researchers are not named.

Risks and caveats

The no-veto promise has limits. Anthropic can still require redactions, and one of the permitted categories is information about its model architectures, so the boundary of what METR can say will matter in practice. METR may characterize redactions but cannot reveal what was redacted. No duration or end date of the arrangement is given, and no financial terms are stated. METR is the only evaluator at the top tier so far. The source does not say whether OpenAI has actually granted any access, only that it pledged the same, and it does not describe the specific agent incidents or say whether any involved Anthropic.

“Anthropic can’t veto publication, but we can require them to redact information if it falls into narrow, pre-defined categories”

— Anthropic, blog post