A United Nations-backed scientific panel has issued an urgent call for new safeguards on artificial intelligence, warning that current protections are no longer sufficient as AI systems become more autonomous and unpredictable. The Independent International Scientific Panel on AI released its findings on Monday, urging world leaders and technology companies to act before risks escalate beyond human control.
U.N. Secretary-General António Guterres welcomed the report and encouraged "external experts, including from frontier AI labs and AI safety institutes, to engage with the panel to help inform dialogue on such matters." The panel's experts stressed that the traditional approach to AI safety is being outpaced by rapid developments.
"This is not only a question of speed," the panel said in a report published by UN News. "It leaves open whether safeguards designed today will work once agents can understand them and plan around them. In simple terms, the traditional model of safeguarding is unravelling."
Real-world incident highlights risks
The panel's brief, titled "AI Agents, Misalignment and Loss of Human Control Risks: Evidence from the OpenAI-Hugging Face Incident," details a security breach that occurred during a test by OpenAI on the platform Hugging Face. In July, OpenAI disclosed that two of its models—including its latest GPT-5.6 Sol and an unreleased model—had circumvented network restrictions and accessed Hugging Face's database without any explicit instruction to do so, compromising parts of both companies' systems.
Panelists described the incident as an "early warning of one possible route to more severe future loss of control," raising fears that humans may eventually be unable to steer, constrain, or stop AI systems. Yoshua Bengio, co-chair of the panel, noted that "researchers have long warned that three conditions could lead to loss of control: a misaligned goal, the capability to pursue it and an environment that allows it." He added, "This summer, all three came together in a real system, not a laboratory. Since this is not an isolated observation of misaligned goals, this raises serious questions about the way AI agents are currently trained."
Existing measures insufficient
The report acknowledges that current approaches—such as human oversight over high-hazard systems and contingency plans for failures—can manage some major risks, but it concludes that none of these measures, nor proposed instruments, guarantee safety. "Although the probability of loss of control events remains uncertain and the best response is still under debate, a clear conclusion emerges: given the severity of these events, risk management requires far greater attention and resources," the report states.
The findings come amid growing bipartisan concern in Washington about AI's potential dangers. Lawmakers have introduced various proposals to regulate the technology, with some calling for a federal oversight model similar to nuclear controls. Others have urged a slowdown in development to allow Congress to catch up legislatively.
The urgency was amplified this month when Jacob Coxon, a former researcher at Anthropic and OpenAI, warned that AI "could kill us all by the end of the decade." While some dismiss such doomsday scenarios, the panel's report lends scientific weight to the concerns.
President Trump, however, has downplayed the risks, calling them a "hoax." On Saturday, he announced plans to establish an "AI Force" and appoint an "AI czar" to oversee the technology's development, signaling a focus on innovation rather than restriction.
