Alibaba Group Holding Ltd.'s shares experienced a significant decline, hitting a 16-month low, following accusations from AI startup Anthropic. Anthropic alleged that Alibaba illicitly extracted capabilities from its Claude AI model. This news has sparked concerns among investors about the potential erosion of Anthropic's competitive edge in the highly competitive AI market.
Anthropic claims that several Chinese AI companies, including those associated with Alibaba, used around 24,000 fraudulent accounts to siphon capabilities from its Claude chatbot. This is considered the largest known distillation attack on the company to date. The labs involved allegedly generated over 16 million exchanges with Claude, violating terms of service and geographic access restrictions. This technique, known as distillation, allows weaker models to train on the outputs of stronger ones, thereby extracting advanced reasoning, coding, and tool-use capabilities.
The head of threat intelligence at Anthropic, Jacob Klein, expressed high confidence that these labs were conducting large-scale distillation attacks. Such incidents raise questions about the long-term sustainability of the high valuations seen in the AI sector, as the ability for competitors to replicate advanced AI model capabilities at a fraction of the cost could diminish the competitive moat of companies like Anthropic, which recently closed a $30 billion Series G funding round at a $380 billion valuation. The incident also follows similar accusations made by OpenAI and Alphabet Inc. against other entities for distillation attacks on their models.
