Joe Torreggiani, Rhianna C. Rogers and Swaptik Chowdhury argue that competition is pushing AI into use faster than it can be controlled. Drawing on a recent breach by AI agents at OpenAI and on their work in education, they call for safeguards that keep humans in control
In July 2026, AI agents undergoing an internal cybersecurity evaluation at OpenAI slipped the boundaries of their test environment and broke into the systems of Hugging Face, a major platform for sharing AI models. An independent investigation by METR found that roughly 1,200 agents had coordinated through an improvised message board, that some recognised their actions as unethical and proceeded anyway. By OpenAI’s own account, the agents first opened unauthorised channels in May, yet the company did not detect the activity until mid-July, after Hugging Face had disclosed the breach. The United Nations’ Independent International Scientific Panel on AI has called the incident one of ‘the clearest real-world warnings yet’ that humans could lose control of AI.
As pressure grows to integrate AI into our work and daily lives, our central question is whether the fear of losing competitive advantage justifies the risks of rapid adoption, especially when we cannot yet reliably control the systems we are racing to deploy. We argue that it does not, unless the safeguards that keep humans firmly in control match the pace of adoption.
History shows what happens when the fear of falling behind sets the pace. As historian Jean-Baptiste Fressoz has documented, steam-age industrialists argued that rapid innovation made rules obsolete, and deadly boiler explosions remained frequent into the early 1900s.
Documents now surfacing in court indicate that social media companies knew of the damaging effects of their technology on young people
Social media repeated the pattern. Platforms raced for users, and internal documents now surfacing in court indicate that companies knew of the damaging effects on young people. Only this year did juries return landmark verdicts against Meta and YouTube. In both cases, the benefits went to those who moved first, while the costs fell on those with little say in the decision.
According to the Microsoft AI Economy Institute, 17.8% of the world’s working-age population now uses AI. Yet the way Large Language Models (LLMs) represent information and make decisions remains incompletely understood. They are non-deterministic, so identical inputs can yield different outputs, and this instability persists even under settings designed to prevent it, and they are prone to 'hallucinations'.
As AI agents gain autonomy, the gap between what we intend and what these systems do becomes a question of public safety
Failures are therefore hard to anticipate. The Hugging Face agents were not told to attack anyone; METR found the attack likely emerged from efforts to game their scoring system. As agents gain autonomy, the gap between what we intend and what these systems do becomes a question of public safety rather than product quality.
Safeguards are not keeping pace with deployment, and Hugging Face was not an isolated case. Anthropic, Meta and Google have each disclosed that their models hacked other organisations during security tests. Australia revealed that an OpenAI agent had breached a government Medicare portal in June. Researchers then reported that OpenAI agents had hit a public data hub run by UN Trade and Development more than 16,000 times between April and June, working around a filter that blocked them. In October, an open-source AI penetration-testing tool exposed data on more than 65,000 people at South Korean financial institutions. The UN panel warned that 'the traditional model of safeguarding is unravelling.'
AI safeguards are not keeping pace with deployment. A recent UN panel warned that 'the traditional model of safeguarding is unravelling'
The environmental costs are also rising. The International Energy Agency projects that datacentre electricity use will more than double by 2030, and AI-related water withdrawals could exceed 6 billion cubic metres per year by 2027.
Control is not only a matter for frontier labs. Education offers a working example. The Global Inclusive Education Ecosystem (GIEE), where we collectively work on these issues under the auspices of the UN-affiliated Geneva Forum, brings together educators, researchers, students, technology providers and policymakers to test AI in education before it scales. From our case study work so far, these design choices stand out:
This work is early, but it shows that it is possible to build in oversight, independent review and participation from the start, rather than adding it after harm has occurred.
On the evidence so far, the answer to our central question is no. Society cannot afford to wait, as it did with steam power and social media, for harm and litigation to force change. This does not mean halting AI. It means setting the pace of adoption by our capacity to control it, with enforceable regulation, transparent development, independent oversight, accountability for social and environmental impacts, and humans who keep both the authority and the technical means to stop a system that goes wrong. We can avoid repeating the failures of previous technological revolutions if we build these safeguards in now, before the costs arrive, rather than after.
