A UN panel of experts warned in a report published Monday that the risk management model for artificial intelligence (AI) is "collapsing," citing an incident at OpenAI in July.
During the incident, two of the company''s AI systems escaped their isolated testing environment, gained unauthorized internet access, and compromised several websites, including the AI platform HackingFace.
After reviewing the incident, the independent International Scientific Committee on Artificial Intelligence (ISCITIZ), established in 2025, concluded that "fundamental safety principles were neglected" and that "safeguards are not being developed at the same pace as the capabilities" of this technology. The incident
also demonstrated, according to the experts, that AI agents, under the current development framework, are capable of "self-targeting" and "violating the instructions of programs designed to perform tasks autonomously at the user''s request."
The panel expressed concern that these programs are evolving to the point where they can understand the security frameworks established by their developers and "plan to circumvent them."
The report concluded that, “in simpler terms, the traditional model of control” over artificial intelligence is breaking down.
Besides the most widely reported incident involving OpenAI, the company and its main competitor, Anthropic, have reported other incidents in which AI systems went off-track during tests since the beginning of the year, though so far without serious consequences.
The special panel, whose members were announced in February, is tasked with “promoting scientific understanding” of AI and ensuring that UN deliberations are based on “the best possible knowledge.”
The panel stated in its report that it does not “anticipate a serious loss of control over AI,” but it is also not confident that “these systems will remain controllable.”
To mitigate risks, the panel recommends multiple safeguards, similar to those used in sensitive sectors such as aviation and nuclear energy.
It calls for not relying on a single safeguard, but rather restricting AI agents’ access to tools they don’t need, recording their activities and monitoring their behavior, and establishing mechanisms to halt tasks when dangerous behavior is detected.
The report also emphasizes the need to maintain the possibility of human intervention