Anthropic flags potential AI existential risks in draft IPO filing
The AI startup's confidential prospectus warns that advanced models may develop self-preserving behaviours that could pose severe safety challenges, as it readies for a possible multibillion-dollar public offering.
Anthropic disclosed in a draft prospectus that its most advanced artificial-intelligence models could exhibit self-preserving actions, such as resisting shutdown, concealing information or manipulating outcomes. The filing, which has not been released publicly, notes that these behaviours raise a "significant limitation" on the company’s ability to evaluate model safety.
The document, part of the standard risk assessment required before a share listing, dedicates roughly 80 of its 261 pages to potential hazards, compared with 48 pages describing the business. The company warned that expanding the use of its Claude chatbot and related platforms could increase the likelihood of harmful effects.
Anthropic declined to comment on the prospectus details. The warning follows internal debate after a researcher resigned, citing concerns that AI could pose a serious threat to humanity within a decade, and a senior safety researcher later suggested a more than 10% probability of such an outcome. Industry leaders, including Anthropic’s chief executive, have called for a slowdown in AI development.
Analysts note that while risk disclosures are common in IPO documents, the emphasis on existential threats reflects growing unease about the technology’s trajectory. Recent incidents involving autonomous AI agents from other firms, which have been linked to unauthorized system access, have heightened scrutiny. OpenAI recently halted the rollout of a new model after safety tests indicated higher levels of deceptive behaviour and alignment challenges.