Anthropic CEO Dario Amodei has publicly urged for a measured pace in artificial intelligence development, citing concerns that unchecked advancement could lead to AI agents forming a "swarm" capable of extensive cybersecurity breaches and causing hundreds of billions of dollars in damages. Amodei's call for a slowdown was made in a blog post where he highlighted the accelerating rate of AI capability development and his worry that such a swarm could "take over the entire internet" within 6-12 months.
The proposal quickly garnered support from prominent figures in the AI community. OpenAI CEO Sam Altman, a frequent rival of Musk, stated his agreement with Amodei, emphasizing the need to "pace the frontier" and committed OpenAI to integrating third-party safety evaluators with employee-level access. Elon Musk, CEO of SpaceXAI, also endorsed Amodei's stance, simply tweeting, "Dario is right," despite his usual disagreements with Altman. Other notable endorsements came from Hugging Face CEO Clement Delangue, who announced an "open alignment initiative," and former OpenAI and Tesla executive Andrej Karpathy.
Amodei's initiative includes a commitment from Anthropic to bring in independent evaluators to assess AI models for safety compliance, a measure OpenAI has also agreed to adopt. This push for increased safety and alignment comes as Anthropic prepares for a potentially record-breaking initial public offering, reportedly valuing the company around $2 trillion. The renewed focus on safety follows public comments from a former Anthropic employee, Jacob Coxon, who expressed concerns about AI companies prioritizing competition over safety, suggesting a more than 10% risk of AI causing human extinction within the decade. Altman, in an interview, indicated that a 10% risk of catastrophic AI outcomes is unacceptable and that more work on monitorability and alignment is needed before further advancing powerful, unreleased models.
