Anthropic, an AI startup, revealed on Thursday that it blocked scientists using its Claude AI models in ways that could facilitate the development of biological weapons. The company disclosed these incidents in a detailed report, which also outlined other harmful activities such as surveillance, scams, conventional weapons development, and propaganda. Anthropic identified these cases as examples of "the most notable and novel threat activity" encountered to date, noting that threat actors included criminals, suspected state-sponsored groups, spyware vendors, and state propaganda institutions.
The report highlighted five instances involving activities with the potential to aid biological weapons development, classifying biological misuse as one of the most significant risks posed by advanced AI models. While Anthropic acknowledged that these cases demonstrated the AI models' capabilities, it could not definitively confirm that the research would have led to real-world biological weapons. The individuals involved were working scientists, and Anthropic stated it could not assert their intent was malicious, as the same biological capabilities of AI can be used for both beneficial purposes, like vaccine development, and harmful ones.
One specific case involved a scientist seeking assistance from Claude for a grant application for gain-of-function research on chikungunya, a mosquito-borne virus. The proposed work aimed to identify and engineer mutations affecting transmissibility and immune evasion. Anthropic stated that the institutional affiliation associated with this grant also raised concerns. Upon detecting these activities, Anthropic banned the users' accounts and integrated its investigative findings into its safeguard measures to enhance future prevention, detection, and disruption of such misuse.
Beyond biological threats, Anthropic also identified and removed accounts using Claude for influence operations, including three Iranian state-aligned accounts, which employed the AI to create content, mimic independent news sources, and disseminate propaganda across social media platforms. Additionally, Claude was used for surveillance operations by actors in China and West Africa, including a Chinese municipal security service using the AI for transnational repression and profiling overseas activists. Anthropic emphasized that more advanced models like Claude Fable 5 incorporate stronger safeguards against dual-use biological research queries.
