Anthropic, an AI startup, revealed that it has blocked scientists from using its Claude AI models in ways that could support the development of biological weapons. The company disclosed these incidents in a new report detailing various misuses of its AI over the past eight months. While Anthropic identified five cases where its AI was used for research with dual-use potential, it stated it could not definitively determine the researchers' intent, as the same information could be used for both beneficial medical applications and harmful weapon development.
One notable case involved a scientist seeking Claude's assistance in preparing a grant application for gain-of-function research on the chikungunya virus, a mosquito-borne pathogen. Anthropic emphasized that biological misuse represents one of the most serious risks associated with advanced AI models. In response to these findings, Anthropic banned the user accounts involved and integrated the investigative insights into its security protocols to enhance future detection and prevention.
Beyond biological weapon concerns, the report also detailed other harmful activities, including the use of Claude for surveillance, scams, conventional weapons development, and propaganda. Anthropic identified and removed accounts, including three linked to Iranian state propaganda institutions, that used Claude to generate content for influence operations, create fake news sources, and target U.S. naval forces. Additionally, the company found that state-aligned actors in China and West Africa used Claude for surveillance and transnational repression, in some instances replacing human engineering workforces with AI. More advanced models like Claude Fable 5 now feature stronger safeguards against dual-use biological research queries.
