Anthropic, Accenture to invest $1 billion each in embedded AI evaluation
Anthropic and Accenture will each invest at least $1 billion over five years to embed independent evaluators within the AI company.
Artificial intelligence research company Anthropic is partnering with Accenture to independently evaluate frontier AI systems, with each side expecting to invest at least USD 1 billion in building capacity over the next five years.
The arrangement implements a commitment set out in an essay by Anthropic's chief executive, "We Must Pace the Frontier," which argued for placing outside evaluators directly inside the organisation. Accenture's specialist AI business, Faculty, will lead the joint effort.
The work covers testing model safeguards, red-teaming frontier systems and carrying out detailed alignment assessments. Accenture brings its experience of deploying technology for corporate and public sector clients to the safety evaluations.
What sets the framework apart from conventional external audits is the level of access it grants. Embedded evaluators will work inside the AI company with rights comparable to those of an employee, rather than reviewing systems from outside.
That position, according to Anthropic, lets evaluators observe models as they take shape during training, follow the decisions governing how they are built and deployed, and speak directly with staff. From there, they can assess how the company operates, verify that it is meeting its safety commitments and identify blind spots. They can also report incidents and give the public a better-informed account of the benefits and risks involved.
Anthropic maintained that hosting external monitors does not dilute its own responsibility. Independent embedded evaluators do not reduce its accountability but make it more verifiable, the company said, adding that the safety of its models remains its responsibility.
Formal industry standards for access levels and reporting frameworks for embedded evaluators are yet to emerge, and no settled funding model exists. Anthropic wants long-term financing to come from state or pooled funds, as proposed in its June Advanced AI Framework. In the interim, it is funding Accenture's work directly while holding talks with non-profit groups, including METR, to pilot similar embedded evaluations using their own resources.
The partnership is non-exclusive, leaving Accenture free to work with other developers and allowing Anthropic to bring on additional evaluators in the coming weeks.