AI safety experts warn that OpenAI's Astra model uses opaque reasoning technique
Juste les faits

AI safety experts warn that OpenAI's Astra model uses opaque reasoning technique

Résumé

OpenAI's upcoming Astra model will employ a method called recurrent depth, which could make its chain-of-thought less transparent, prompting concerns from AI safety researchers.

OpenAI plans to incorporate a reasoning method known as “recurrent depth” into its forthcoming Astra model, a technique that processes a query through repeated loops rather than a linear sequence. The approach, also described as “opaque recurrence,” could reduce the visibility of the model’s chain-of-thought, a tool traditionally used to monitor AI behavior.

The change has drawn criticism from several AI safety advocates. Redwood CEO Buck Shlegeris said he is “extremely concerned” about the technique and warned that further use could “massively increase the recurrence and totally destroys CoT monitorability.”

AI safety researcher Zvi Mowshowitz suggested that legislation might be needed to prevent a “race to the bottom” among labs, noting that the method “risks a taboo” that companies have tried to uphold regarding chain-of-thought fidelity. Redwood Research chief scientist Ryan Greenblatt added that scaling opaque reasoning could eventually “remove all reasoning from visible channels.”

OpenAI officials have responded that Astra’s use of the method will be limited and that the model’s chain-of-thought will remain legible. In a post on X, chief scientist Jakub Pachocki emphasized the company’s long-standing commitment to chain-of-thought monitoring, stating it is “a core goal of our current research program.”

The Information reported that other firms, including Anthropic and Google DeepMind, are also exploring the technique, raising broader industry concerns about the future transparency of AI reasoning.

Continuez à lire — gratuitement

Créez un compte gratuit pour suivre l'actualité. Aucune carte requise.