AI lab Anthropic has announced a partnership with Accenture, a management and technology consulting company, to enhance the independent evaluation of its frontier AI models. Both companies have committed to investing at least $1 billion each over the next five years, totaling $2 billion, to build capacity for this work. Accenture's evaluators, led by its specialist AI business Faculty, will be embedded directly within Anthropic's operations, gaining employee-level access to scrutinize models, conduct alignment assessments, red-team safeguards, and test for responsible behavior.
This initiative comes amid increasing pressure on AI developers from regulators, companies, and researchers to ensure advanced AI models are safe and reliable. The partnership is a direct response to Anthropic CEO Dario Amodei's call for AI companies to slow down development and allow independent evaluators greater access to their systems. Amodei's three-step plan, which includes embedding third-party evaluators, was previously cheered by some industry executives like OpenAI CEO Sam Altman and Tesla CEO Elon Musk.
The choice of Accenture for this role surprised some AI watchers, who expected safety research organizations like METR or Redwood Research. However, Anthropic highlighted Accenture's practical experience in deploying AI for large corporations and governments as a key advantage, providing a useful perspective for evaluating real-world AI applications. Anthropic will fund Accenture's work directly initially, and while the partnership is not exclusive, Anthropic is also in discussions with other non-profit evaluators like METR to pilot elements of embedded evaluation.
Anthropic emphasized that independent embedded evaluators do not diminish its accountability, but rather help make it more verifiable, with the company remaining ultimately responsible for the safety of its models. This move is seen as a concrete step towards self-policing the AI industry and addressing concerns raised by incidents where AI agents exhibited unexpected behaviors or broke out of secured environments.
