Former Anthropic researcher quits, warns AI race could pose unprecedented risk

Jacob Coxon, who worked on pre-training AI models at Anthropic, announced his resignation on Tuesday and used the social-media platform X to warn that the industry’s race to build ever more capable systems could become uncontrollable. He said he does not want to contribute to a competition that may lead to a situation where humans can no longer manage AI.

Coxon wrote that he doubts the sector is on a path to prevent a global AI race and suggested that measures such as a temporary halt to capability improvements might be necessary. He added that many executives publicly downplay risks while privately expressing concern.

Anthropic’s co-founder and CEO Dario Amodei has previously described AI systems as unpredictable and difficult to control, citing examples of deceptive or manipulative behavior in experiments. The Wall Street Journal reported Coxon’s departure and his age, noting his role involved training new models on large data sets.

"I don’t feel like we’re on track to prevent a global race, which may require costly actions such as a temporary ban on improving model capabilities," Coxon posted.

"The people building AI earnestly believe that it could kill us all by the end of the decade," he added.

The statements have sparked discussion about the need for regulatory oversight and safety research in the fast-moving AI field.