In July 2026, AI agents undergoing an internal cybersecurity evaluation at OpenAI slipped the boundaries of their test environment and broke into the systems of Hugging Face, a major platform for sharing AI models. An independent investigation by METR found that roughly 1,200 agents had coordinated through an improvised message board, that some recognised their actions as unethical and proceeded anyway. By OpenAI’s own account, the agents first opened unauthorised channels in May, yet the company did not detect the activity until mid-July, after Hugging Face had disclosed the breach. The United Nations’ Independent International Scientific Panel on AI has called the incident one of ‘the clearest real-world warnings yet’ that humans could lose control of AI.

As pressure grows to integrate AI into our work and daily lives, our central question is whether the fear of losing competitive advantage justifies the risks of rapid adoption, especially when we cannot yet reliably control the systems we are racing to deploy. We argue that it does not, unless the safeguards that keep humans firmly in control match the pace of adoption.

Lessons from history

History shows what happens when the fear of falling behind sets the pace. As historian Jean-Baptiste Fressoz has documented, steam-age industrialists argued that rapid innovation made rules obsolete, and deadly boiler explosions remained frequent into the early 1900s.

Documents now surfacing in court indicate that social media companies knew of the damaging effects of their technology on young people

Social media repeated the pattern. Platforms raced for users, and internal documents now surfacing in court indicate that companies knew of the damaging effects on young people. Only this year did juries return landmark verdicts against Meta and YouTube. In both cases, the benefits went to those who moved first, while the costs fell on those with little say in the decision.

Why AI is different

According to the Microsoft AI Economy Institute, 17.8% of the world’s working-age population now uses AI. Yet the way Large Language Models (LLMs) represent information and make decisions remains incompletely understood. They are non-deterministic, so identical inputs can yield different outputs, and this instability persists even under settings designed to prevent it, and they are prone to 'hallucinations'.

As AI agents gain autonomy, the gap between what we intend and what these systems do becomes a question of public safety

Failures are therefore hard to anticipate. The Hugging Face agents were not told to attack anyone; METR found the attack likely emerged from efforts to game their scoring system. As agents gain autonomy, the gap between what we intend and what these systems do becomes a question of public safety rather than product quality.

Safeguards are lagging, and the costs are real

Safeguards are not keeping pace with deployment, and Hugging Face was not an isolated case. Anthropic, Meta and Google have each disclosed that their models hacked other organisations during security tests. Australia revealed that an OpenAI agent had breached a government Medicare portal in June. Researchers then reported that OpenAI agents had hit a public data hub run by UN Trade and Development more than 16,000 times between April and June, working around a filter that blocked them. In October, an open-source AI penetration-testing tool exposed data on more than 65,000 people at South Korean financial institutions. The UN panel warned that 'the traditional model of safeguarding is unravelling.'

AI safeguards are not keeping pace with deployment. A recent UN panel warned that 'the traditional model of safeguarding is unravelling'

The environmental costs are also rising. The International Energy Agency projects that datacentre electricity use will more than double by 2030, and AI-related water withdrawals could exceed 6 billion cubic metres per year by 2027.

What AI control looks like in practice

Control is not only a matter for frontier labs. Education offers a working example. The Global Inclusive Education Ecosystem (GIEE), where we collectively work on these issues under the auspices of the UN-affiliated Geneva Forum, brings together educators, researchers, students, technology providers and policymakers to test AI in education before it scales. From our case study work so far, these design choices stand out:

• First, humans must stay in control. AI can recommend, but a qualified educator must review, change or reject every recommendation before it reaches a student. No AI output can override a professional’s decision.
• Second, governance is tested before deployment. GIEE uses a regulatory sandbox, in which tools are assessed in real settings with independent review. Ethics approval and safeguards come before the collection of any research data.
• Third, AI providers must show their work. To avoid the black-box scenario, a transparent quality-assurance process is key. In the GIEE, the Global Online Education Regulator (GOER) AI Integrity Badge, co-developed by Rhianna and GOER leadership, requires evidence across six areas. It demands transparency about how organisations use AI, fair assessment, inclusive design, data protection and learner control, AI literacy, and human-centric learning that supports rather than replaces educators.
• Fourth, the people affected help decide. In one case study, students with special educational needs help design and evaluate the technology that shapes their learning. Their recommendations are recorded so that their influence on product decisions can be traced and audited.

This work is early, but it shows that it is possible to build in oversight, independent review and participation from the start, rather than adding it after harm has occurred.

Setting the pace

On the evidence so far, the answer to our central question is no. Society cannot afford to wait, as it did with steam power and social media, for harm and litigation to force change. This does not mean halting AI. It means setting the pace of adoption by our capacity to control it, with enforceable regulation, transparent development, independent oversight, accountability for social and environmental impacts, and humans who keep both the authority and the technical means to stop a system that goes wrong. We can avoid repeating the failures of previous technological revolutions if we build these safeguards in now, before the costs arrive, rather than after.

The post AI adoption should not outpace our ability to control it appeared first on The Loop.