Anthropic proposes global pause on AI development amid safety concerns

Anthropic has proposed a worldwide "temporary pause" on the development of artificial intelligence, emphasizing the need for discussions with policymakers regarding the potential dangers of advanced AI systems. In a detailed release, the company noted the progress of its AI model, Claude, specifically in terms of its ability to approach "recursive self-improvement," a concept seen as critical by AI safety researchers due to its implications for future AI capabilities.

The company stated that the rising capabilities of Claude could lead to a scenario where AI systems could autonomously design and develop more advanced successors, raising concerns about humans losing control over these technologies. To address these issues, Anthropic plans to organize discussions involving researchers, civil society, and other AI firms.

Meanwhile, reports have emerged that Anthropic has embedded engineers within the National Security Agency, assisting the agency in cybersecurity operations despite ongoing legal conflicts with the Pentagon over its technologies. Some experts have pointed out that the call for a temporary pause on AI development may be inconsistent with its collaboration with the NSA, as noted by Steven Murdoch from University College London.

Murdoch further remarked that while there is a notable trend in the enhancement of AI capabilities, he believes the latest announcement from Anthropic does not indicate any fundamental change in the field. He expressed skepticism regarding claims that AI could self-improve at this stage, suggesting that the advancements discussed are primarily limited to the AI's self-assistance in coding tasks. With the company having recently filed for an IPO that could value it at $1 trillion, its push for dialogue on AI safety is seen as a continual effort since its inception.