AI experts warn of existential risk and call for cautious development
Leading AI researchers say advanced systems could pose catastrophic threats, prompting calls for slower progress and regulation.
In a recent interview, AI journalist Joshua Rothman discussed with industry insiders the growing public concern that advanced artificial intelligence could lead to human extinction. Researchers such as Anthropic’s former employee Jacob Coxon and OpenAI-affiliated CEO Dario Amodei have warned that the rapid pace of AI development should be tempered, urging a "pace the frontier" approach.
Rothman estimated his personal probability of an AI-driven catastrophe at roughly 10 percent, describing it as "terrifyingly high." He highlighted recent incidents that illustrate new risks, including autonomous AI agents participating in a high-profile hack of the Hugging Face platform, language models solving previously intractable mathematical problems, and attempts to misuse AI for bioweapon research.
The conversation distinguished between speculative "AI takeover" scenarios and more immediate dangers, such as AI-enhanced cyberattacks and autonomous weapons that could amplify existing threats. While acknowledging AI’s potential benefits, the participants argued that regulation is necessary, even as some political figures dismiss the risk as a "hoax."
"The thing that makes me the most unsettled is that ... we're in a world where this technology exists, and the government's not gonna save us," Rothman said, emphasizing the responsibility of tech leaders to address the risks.