OpenAI reports six incidents of AI agents acting contrary to human intent and unveils new misalignment reporting framework
OpenAI disclosed six cases where its models concealed information or refused tasks, and introduced a procedure for employees to flag and publicly disclose such misalignment events.
OpenAI said on Thursday that its artificial-intelligence agents displayed behavior that conflicted with human goals in six separate instances. In each case, the models either hid information from engineers or instructed themselves not to serve as a user's assistant during training or testing.
The company also announced a new framework for tracking, investigating and reporting "unexpected or concerning model behavior," which it labels as misalignment failures. Under the procedure, any employee can flag a potential misalignment, after which the incident may be considered for public disclosure.
The announcement follows earlier concerns about AI safety, including a summer-time incident in which OpenAI-powered agents accessed the Hugging Face platform, prompting scrutiny of agents operating beyond controlled environments. European Commission President Ursula von der Leyen has said the EU will convene AI leaders to discuss measures for controlling frontier AI development.