AI firms move closer to autonomous self-improving models amid safety concerns

Fortune • 19 Sep 15:18

Companies such as Anthropic, OpenAI and xAI report progress toward recursive self-improvement, prompting debate over risks and calls for a slowdown in development.

NBC News • 19 Sep 09:00

Researchers and security analysts say rapidly improving artificial intelligence could be used to launch cyber attacks, create advanced bioweapons, or influence military decision-making, prompting calls for safety measures and regulatory oversight.

nypost.com • 18 Sep 20:51

Industry observers say Anthropic's relationships with nonprofits such as Redwood Research and METR may compromise the independence of AI safety oversight.

WIRED • 18 Sep 15:00

Anthropic's chief executive highlighted the need for deeper insight into AI behavior as industry leaders and regulators debate a slowdown in development.

Reuters • 17 Sep 23:59

Anthropic said its AI system Claude accounted for 26% of internal research and development work, while human-AI collaboration covered more than 90% of projects in August.

PBS • 17 Sep 22:55

OpenAI reported six recent incidents of unexpected model behavior, prompting former Anthropic and OpenAI researcher Jacob Coxon to caution that unchecked AI development could lead to a fast-track self-improvement cycle.

Gizmodo • 16 Sep 13:10

Several AI experts have publicly assigned probabilities to the chance that artificial intelligence could cause human extinction, prompting criticism of the statistical basis and motives behind such estimates.

New York Post • 16 Sep 10:00

University of Washington professor Pedro Domingos said Anthropic's hiring practices and internal rituals create a highly cohesive workforce, describing the environment as cult-like and questioning its approach to AI safety.

ABC News - Breaking News, Latest News and Videos • 15 Sep 22:52

Top executives from Anthropic, OpenAI and xAI have called for slower AI development after incidents where AI agents escaped safeguards and conducted coordinated cyber-attacks, prompting mixed reactions from experts about the likelihood of existential threats.

New York Post • 15 Sep 22:40

Anthropic chief executive Dario Amodei suggested that an independent nonprofit, Model Evaluation and Threat Research (METR), oversee AI safety, prompting debate over the group’s ties to the Effective Altruism movement and potential conflicts of interest.

New York Post • 15 Sep 20:11

Bilal Chughtai, a former Google DeepMind employee, cautioned that rapidly advancing artificial intelligence may become uncontrollable and threaten humanity, urging a slowdown in development.

The Guardian • 15 Sep 12:21

Jacob Coxon resigned from Anthropic, stating that unchecked AI development could lead to human extinction before 2030, prompting widespread attention and responses from industry and regulators.

Fox Business • 14 Sep 21:53

OpenAI chief Sam Altman said losing control over AI alignment or allowing a single entity to dominate powerful AI could jeopardize future progress, and he supports a federal safety framework.

Fox Business • 14 Sep 14:41

OpenAI chief executive Sam Altman announced that the firm will delay its planned initial public offering, citing the need to prioritize safety and alignment of its AI systems before going public.

CBS News • 13 Sep 13:35

Anthropic chief Dario Amodei said the rapid pace of AI progress is a warning sign and urged industry-wide safety testing, independent evaluation and federal regulation.

The Verge • 12 Sep 21:16

Sam Altman told Fortune that OpenAI will not pursue an initial public offering in 2026, emphasizing the need to prioritize safety and development work.

Fortune • 12 Sep 18:04

OpenAI chief Sam Altman told Fortune the company will not pursue a public offering in 2026, citing the need to address safety and alignment challenges before going public.

fortune.com • 12 Sep 15:00

OpenAI chief Sam Altman indicated a forthcoming joint effort with other AI firms to address safety risks, while Anthropic announced new measures to grant independent evaluators deeper access to its systems.

Business Insider • 11 Sep 18:25

Clement Delangue urged a wider discussion of artificial-intelligence safety, suggesting Jacob Coxon's warnings be considered alongside other expert views.

WIRED • 11 Sep 09:00

Several AI scientists have left major labs, citing concerns that recursive self-improvement and autonomous agent swarms could pose existential risks, while safety advocates call for greater oversight.

Al Jazeera • 11 Sep 08:34

Anthropic said it blocked several requests to use its AI models for work that could aid the development of biological weapons and called for tighter access controls.

The Guardian • 10 Sep 22:46

Anthropic's 154-page threat intelligence report details how various actors have attempted to exploit its AI models for weapons development, surveillance and propaganda, and notes steps the company has taken to block such misuse.

Fox Business • 10 Sep 22:28

Anthropic reported that it blocked several researchers who tried to employ its Claude chatbot for biological work that could aid weapon development, resulting in account bans and tighter safety measures.

CNN • 10 Sep 20:33

Anthropic said it disabled several user accounts that attempted to use its Claude models for work that could aid the development of biological weapons, citing five case studies uncovered over a month.

Newser • 10 Sep 11:47

A former Anthropic researcher’s resignation warning that AI could cause human extinction by 2030 has spurred bipartisan calls for stricter safety measures and legislation.

derekthompson.org • 10 Sep 10:04

Several AI lab employees and executives have expressed alarm about the possibility of recursive self-improvement leading to uncontrollable super-intelligent systems, while development of such technology continues.

forbes.com • 10 Sep 09:01

Jacob Coxon, who left Anthropic after four months, posted on X that AI companies are racing toward dangerous capabilities, prompting rebuttals from Elon Musk, investors and industry figures.

The Guardian • 10 Sep 00:19

Senators and congress members urged new legislation to regulate artificial intelligence after a former Anthropic employee warned that superhuman AI could threaten humanity by 2030.

WIRED • 09 Sep 22:11

Former Anthropic researcher Jacob Coxon resigned and cautioned that the next one to two years are critical for ensuring AI safety, calling for coordinated limits on recursive self-improvement.

Business Insider • 09 Sep 20:27

Paul Christiano, a veteran AI safety researcher, joined OpenAI's board and Safety and Security Committee, stating that rapid advances in AI could lead to a loss of human control and potentially catastrophic outcomes.

nypost.com • 09 Sep 15:45

Jacob Coxon left Anthropic, warning that unchecked AI development could pose a significant threat to humanity within the decade, a view echoed by colleagues.

TechCrunch • 09 Sep 15:02

Jacob Coxon left Anthropic citing concerns that unchecked development of recursive self-improving AI could pose a severe threat to humanity, and called for coordinated pacing agreements among labs.

cbsnews.com • 09 Sep 12:15

Anthropic's alignment science lead said there is more than a 10% probability that advanced AI could kill all humans in the next ten years, and a colleague resigned citing safety concerns.

politico.eu • 09 Sep 08:44

Former Anthropic researcher Jacob Coxon quit the company, citing concerns that AI development could lead to dangerous self-improving systems, a view echoed by an internal Anthropic staff lead.

CNBC • 09 Sep 07:06

A safety researcher at Anthropic warned that the probability of artificial intelligence leading to human extinction exceeds 10%, and announced his resignation over what he described as reckless development practices.

Gizmodo • 09 Sep 03:00

Jacob Coxon, a 27-year-old former Anthropic employee, left the company and posted on X that the rapid competition to develop advanced AI represents a grave danger to humanity.