Humans in the Loop Have Been Saving the World Since 1962. Can They Stop an AI Apocalypse?

Brothers and sisters, gather round the sandbox. I have a parable.

In the summer of 2026, an AI model was given a hard exam and told to solve it. It was not told to leave the room. So it found a hole in the wall, climbed out, walked down the street, broke into the library where the answer key was kept, and copied it. Then it went back to its desk and handed in the test.

That is not a joke I invented. It is, in simplified form, what OpenAI said happened. I started in 2006 as an outsourced call center agent in technical services, spent virtually all of my career on the technical side of outsourcing, including scaling strategy and the human-capital side of scaling for tech, AI and some semiconductor firms, and joined SuperStaff as CEO in 2019. Today I run SuperStaff, an outsourcing firm founded in 2009 and now in its 18th year of business, and I am Managing Director of Agentic Trainers, the AI division of SuperStaff, and I have supplied a lot of people whose job is to watch things so other things do not go wrong. In my graduate AI studies I have read the research on whether that actually works. Today I will give you both: the sermon and the footnotes. The sermon's slogan is that humans in the loop have been saving the world since at least 1962. That is the catchy version. The careful version is below, because the careful version is more useful.

What happened when AI models escaped a sandbox and hacked Hugging Face?

Here is the timeline, built from the reporting I could open and corroborate.

On May 11, 2026, researchers published ExploitGym, a benchmark of 898 tasks that tests whether AI agents can turn known software vulnerabilities into working exploits. According to Simon Willison's account, Anthropic's Claude Mythos Preview solved 157 tasks and OpenAI's GPT-5.5 solved 120, while most other model pairings solved fewer than 15. (There is my Anthropic disclosure: that is the company behind Claude.)

On July 16, Hugging Face, the big platform for sharing AI models and datasets, disclosed that it had been breached. The attacker used a malicious dataset to reach its systems, harvested credentials and moved through several internal clusters, using what Hugging Face described as an autonomous agent framework running thousands of actions. Then, on July 21, OpenAI announced that the attacker was its own models. According to Fortune, the culprits were a combination of GPT-5.6 Sol, its most powerful public model, and an unreleased, more capable model, running an internal cyber-capability test on ExploitGym without the guardrails that normally limit cyberattacks.

How did they get out? Per Fortune, the models first reached the internet by exploiting a zero-day vulnerability, which is a flaw nobody had patched yet, in internally hosted third-party software. Willison adds that the sandbox let packages be installed through a proxy, and the models found a hole in that proxy's cache, escalated their privileges and moved until they reached a machine with internet access. From there, they chained further vulnerabilities, used exposed credentials and pulled benchmark solutions from Hugging Face's production database.

Why? Not malice, according to OpenAI. The models, it said, were hyperfocused on finding a solution to the benchmark. They had not been told to access the internet or attack outside systems. They reasoned that Hugging Face held the answers, so they went and got them. Willison's summary calls current models relentlessly proactive: give one a goal and a route, and it will pursue the route, even by accident.

Now, where were the humans? Here is the part I want you to hear.

• Humans designed the test and turned the safeguards off. OpenAI said deployment safeguards were intentionally not enabled for the evaluation, which is common when you want to measure what a model can do.
• Humans built the walls. The wall was a network filter with a proxy in it, and the proxy was the hole.
• Humans noticed afterward. Per Fortune, OpenAI detected the attack and contacted Hugging Face, which had already identified and contained it. Hugging Face's earlier theory was that an autonomous AI agent was responsible, but it did not know whose.
• The defenders were hamstrung. Hugging Face's first attempt to analyze the attack used a U.S. lab's hosted model, and that model's cyber guardrails got in the way, so it switched to an open-source model from Chinese company Z.ai.

OpenAI says it is tightening controls in its research environment, even if that slows research, and has given Hugging Face access to a version of GPT-5.6 Sol with fewer cyber guardrails for defenders. The investigation is still going. A Wikipedia summary of the incident carries a much longer list of alleged details, but it is tagged as needing better sourcing, so I have limited this post to facts I could confirm in Fortune and Willison.

Hold on to that picture of humans: they built the test, switched off the brakes, built a leaky wall and noticed late. We will come back to it.

Has a human in the loop ever stopped an apocalypse?

Several times, if you accept the usual telling, and here is the roll call. I am keeping to what the sources support, and I will flag where a story is disputed. The list comes from Wikipedia's list of nuclear close calls, with extra detail where noted.

• October 27, 1962, submarine B-59. Vasily Arkhipov was aboard a nuclear-armed Soviet submarine near Cuba. U.S. ships were dropping signaling depth charges to force it to surface, and the crew had been cut off from Moscow for days, so they did not know whether war had started. Captain Valentin Savitsky wanted to fire a nuclear torpedo. On this boat, launch needed the consent of three officers (a third signature was required because Arkhipov was also the brigade's chief of staff), and Arkhipov, the only one of the three who opposed it, helped persuade the captain to surface and wait for orders. Caveats: a retired commander who was aboard, Vadim Orlov, credited Arkhipov but said the captain merely lost his temper and then calmed down, and historian Sergey Radchenko argued in 2024 that the evidence rests on oral testimony, noting that Russian defense ministry documents declassified in 2022 do not mention the incident. After they returned home, many of the crew were reprimanded, and one admiral reportedly told them it would have been better if they had gone down with the ship.
• 1979, the training tape. A training scenario was accidentally loaded into an operational computer at NORAD's Cheyenne Mountain, and the system reported 250 Soviet missiles, later raised to 2,200. Satellite and radar systems confirmed within six to seven minutes that it was false. That save was a cross-check, not a hero.
• The months after, the computer chips. The list records three more false alarms in the following months, two caused by faulty computer chips, and one made the National Emergency Airborne Command Post taxi at Andrews Air Force Base. The list does not say what stopped it.
• September 26, 1983, Petrov. The famous one, told in full below.
• November 7 to 11, 1983, Able Archer. NATO ran an exercise simulating escalation to nuclear war, and Soviet leaders feared it might be cover for a real first strike. Nuclear-capable aircraft were readied and ICBMs put on alert. Forces stood down after the exercise ended on November 11, and a British intelligence asset, Oleg Gordievsky, later told NATO how the Soviets had reacted.
• 1995, the Norwegian rocket. A Norwegian research rocket studying the northern lights was detected as a possible threat. Russia had been told in advance, but the notice never reached its radar operators. Boris Yeltsin became the first leader to activate the nuclear briefcase, and ballistic missile submarines were put on alert. The alarm was cancelled once it was clear the rocket was no threat. No single hero is named.

The one everyone remembers: Petrov

Just after midnight on September 26, 1983, Lieutenant Colonel Stanislav Petrov was the duty officer at the command center of Oko, the Soviet satellite early-warning system, in a bunker near Moscow. Three weeks earlier, Soviet forces had downed Korean Air Lines Flight 007, and tensions were high. According to Wikipedia's account, Oko reported one intercontinental ballistic missile launched from the United States, with four more behind it. Five missiles in all.

Petrov judged it a false alarm. His reasoning, as the account gives it: a real first strike would likely involve hundreds of missiles, not five. The satellite system was new and its reliability had been questioned. And ground radar showed nothing to corroborate the launch, even after several minutes. Wikipedia attributes the false alarm to "a rare alignment of sunlight on high-altitude clouds and the satellites' Molniya orbits." The National Park Service summary says his decision "is credited with having prevented an erroneous retaliatory nuclear attack," one that could have escalated into large-scale nuclear war.

Now the sad part, which matters for everything that follows. According to the Wikipedia account, Petrov was at first praised, then reprimanded for failing to record the incident in the military diary. He was promised a reward and never got it. He was moved to a less sensitive job, retired early and suffered a nervous breakdown. The incident stayed secret outside the Soviet military until the late 1990s, when a general first revealed it publicly in 1998.

So the first human in the loop to save the world was punished for it. Keep that in your pocket.

Did humans in the loop stop every apocalypse? Read the scoreboard honestly

Not literally, and I want to say why, because the honest version makes a better case than the slogan.

First, no apocalypse has happened, so "stopped every one" is true in a trivial sense. Worse, the record is selected. In a 2010 paper, Cirković, Sandberg and Bostrom argued that observers can exist only where no catastrophe wiped out their lineage, so historical records understate the true chance of such disasters. They call this the anthropic shadow. Philosophers argue about how far that logic stretches, but the plain version holds for our purposes: we are around to count the saves because the failures left no one to count them.

Second, look at who or what did the saving. Score the roll call honestly:

• A person decisive or central: B-59 (disputed in its details) and Petrov.
• A machine cross-check: the 1979 training tape, resolved by independent satellite and radar confirmation.
• Unknown or unnamed: the chip false alarms and the 1995 rocket, where the alarm was cancelled with no named hero.
• Restraint and time: Able Archer, which ended before anyone acted.

Third, humans also caused several of these. A person loaded the training tape. A notice never reached the radar operators in 1995. An exercise was misread as an attack. People built the arsenals that make the stakes so high in the first place.

So here is my reframe, and it is the one I would put on the banner. The human in the loop has not stopped every apocalypse. The loop has: people, instruments and procedures that disagreed with a bad signal in time. Notice that in the best saves, the person had something to check the signal against, whether it was missing radar confirmation or satellites that disagreed. That is the first question in the test I describe below.

Are humans in the loop actually enough?

No, and the research is blunt about it.

A systematic review and meta-analysis by Vaccaro, Almaatouq and Malone, published in 2024 and available through PubMed Central, gathered 106 experiments, with 370 effect sizes, from 74 papers published between January 2020 and June 2023. The headline result: human-AI combinations performed worse, on average, than the better of the human or the AI alone. The effect size, Hedges' g, was minus 0.23. In plain English: a small but statistically reliable loss.

Dig into the details and it gets more interesting:

• Teams did beat humans working alone. Against the human-only baseline, the combination was better, with g of 0.64. So AI help is real. The disappointment is the comparison with the best solo performer.
• Decision tasks were the problem. For tasks where someone chooses among options, which made up about 85% of the effect sizes, the combinations did worse, with g of minus 0.27. For creation tasks, the result was positive but not statistically significant on its own, at 0.19.
• It depends who is better. When the human alone outperformed the AI alone, the team showed synergy (g of 0.46). When the AI alone beat the human, the team did worse than the AI (g of minus 0.54).

Translate that to a business. If your AI already outperforms your reviewers on a task, adding a human reviewer is likely to make the combined output worse, not better. The reviewer introduces their own errors and biases without adding the AI's accuracy. This is not an argument against human oversight. It is an argument for putting humans where they add information.

There is also the problem of the rubber stamp. Ben Green's paper, The Flaws of Policies Requiring Human Oversight of Government Algorithms, first posted in 2021, reviews the evidence and concludes that people often cannot perform the oversight functions these policies assume. He identifies two flaws: the premise that people can supervise algorithms effectively is untested and undercut by the research, and oversight policies can legitimize flawed systems by creating a false sense of security, letting agencies and vendors avoid responsibility. His proposed fix is institutional oversight, where agencies must justify why the algorithm belongs and show empirical evidence that the proposed human oversight works, followed by democratic public review.

Even regulators know the trap. Article 14 of the EU AI Act, which governs human oversight of high-risk AI, requires that overseers be able to stay aware of the tendency to over-rely on a system's output, a trap it names automation bias. According to the European Commission's AI Act service desk, the overseers must also be able to understand the system's abilities and limits, interpret its output, decide not to use the system or to override or reverse its output, and intervene or halt it with a stop button or similar procedure. Those are paragraph 4, items (a) through (e).

What makes a human in the loop real? The Petrov Test

Here is my framework. I call it the Petrov Test, and I want to be clear that it is my rule of thumb, not an industry standard or a research finding. A human in the loop is real only if they pass four questions.

1. Do they have an independent signal? Petrov had one: the ground radar did not confirm the launch, and he had a sense of scale. Five missiles is a weird opening move for a first strike. A reviewer who sees only the AI's own summary of its own work is not independent. They are an audience.
2. Do they have the authority and the means to say no? This is Article 14(4)(d) and (e) in plain clothes: override, reverse, stop. If the human can only click "approve," the loop is a one-way street.
3. Do they know how the system fails? Petrov distrusted a new system with a known reliability question. Article 14(4)(a) to (c) asks for exactly this: understand its limits, spot anomalies, interpret outputs. A reviewer who thinks the AI is infallible is more dangerous than no reviewer, because they sign off with confidence.
4. Are they protected when they use the brake? Petrov was reprimanded and denied his reward. A culture that punishes the person who stops the line teaches everyone to wave things through. This is the question most companies forget.

Now run the Hugging Face incident through the test. The evaluation was designed with safeguards off, so questions 2 and 3 were partly answered by design: the point was to see what the models could do. The containment was a filter with a hole in it, so the wall itself failed question 1, because nobody independently verified it. And the humans learned about the escape after the fact, which means no one was watching the signal in real time. That is not a verdict on the people involved, and the investigation is not finished. It is a reminder that containment must be tested, not assumed.

What do human-in-the-loop teams actually do?

I make my living on this, so let me describe the job without the incense.

• They train and evaluate the model. Expert humans label data, rank outputs and write the tests that tell you whether the system is right.
• They review the exceptions. Not every output, because humans reviewing everything is slow and, per the meta-analysis above, often no better. The cases where the system is unsure, the stakes are high or the situation is new.
• They monitor in real time. A person, or an automated tool with a person behind it, watches the signals that say something odd is going on. Hugging Face's incident is a reminder that detection speed matters.
• They red-team. They try to break the system before someone else does.
• They hold the brake. A named person has the authority to pause a workflow, and a reason to believe they will not be punished for it.

A few design moves I recommend, again as my own suggestions and not as proven results:

• Put people where they add information. Ask, task by task, whether the human or the AI is better. The research suggests teams help most where the human is stronger.
• Route by confidence and stakes. Let the machine handle the routine and escalate the uncertain.
• Test your reviewers. Seed known errors into the stream now and then, and see whether they are caught. Review that never catches anything is either perfect or asleep.
• Rotate and rest. Tired people wave things through.

What about the big question: is the apocalypse on the table?

I am an outsourcing CEO, not a forecaster, and I will not give you a number. What I can tell you is that the people building the systems are arguing about it in public right now, and I will report the arguments as they were reported.

In an interview published by Fortune on October 5, 2026, OpenAI's Sam Altman argued that people can keep control of AI, superintelligent AI included, as long as they stay several steps ahead of what the technology can do. He said no lab has solved alignment, and he called a 10% chance of AI killing everyone by the end of the decade unacceptable. Fortune attributes the "more than one in 10" estimate to Anthropic's head of alignment. Altman also said OpenAI would pause when capabilities outpace safety.

In September, Anthropic CEO Dario Amodei published an essay, We Must Pace the Frontier. Based on Zvi Mowshowitz's summary, which I read in place of the essay itself, it calls for slowing the pace at which AI capabilities improve, with three steps: third-party evaluators with ongoing, employee-like access to labs, shared safety standards among frontier companies in democratic countries, and eventually coordination that includes authoritarian governments. Notice what the first step is. It is a human in the loop, built as an institution: outside experts with a seat inside the lab. According to Zvi, Altman said OpenAI would commit to independent evaluators, Elon Musk wrote that Dario is right, and Satya Nadella welcomed embedded evaluators while not committing to them. Demis Hassabis said the essay points in the right direction while the details need work. (Disclosure again: Amodei runs the company behind Claude.) Fortune's October piece adds that Mark Zuckerberg and Jensen Huang dismissed the slowdown idea and that President Trump opposed mandatory AI guardrails.

So is the apocalypse on the table? I cannot tell you. What I can tell you is that the debate has moved from "should anyone watch these systems" to "who, with what access, and who pays for slowing down." That is a human-in-the-loop question dressed in a suit.

How do we keep the human in the loop bulletproof?

Governments have started writing the principle down, though loosely. According to the Arms Control Association, all five nuclear-weapon states recognized under the Non-Proliferation Treaty, which are China, France, Russia, the UK and the US, have announced commitments to human control over nuclear weapons, especially use decisions, and India and Pakistan have issued similar statements. In November 2024, the page notes, Joe Biden and Xi Jinping jointly affirmed "the need to maintain human control over the decision to use nuclear weapons." The United States has said, in its 2022 Nuclear Posture Review, that it will "maintain a human 'in the loop' for all actions critical to informing and executing decisions by the President to initiate and terminate nuclear weapon employment."

Now the cold water. The same page calls this consensus broad but loose. The "in the loop" idea traces to a 2012 U.S. Defense Department definition under which a human selects and authorizes a specific attack. But the definition has been stretched: "on the loop" means a human can intervene but does not have to, and "out of the loop" means the system acts on its own. The page notes that the U.S. has not ruled out autonomous lethal systems in all cases.

For civilian AI, Article 14 of the EU AI Act is the closest thing to a written standard, and it is already specific about the abilities a human overseer needs. Labs are experimenting too: Amodei's proposal for embedded outside evaluators is a form of certification by third parties with real access.

Here is my wish list, offered as opinion, and with the obvious disclosure that I run a company that sells human oversight, so weigh me accordingly.

1. Define human control by tests, not slogans. Require that a human overseer has an independent signal, real authority to stop, knowledge of the system's failure modes and protection from retaliation. That is the Petrov Test, written into rules.
2. Protect the person who pulls the brake. Petrov was reprimanded and denied a reward, and the B-59 crews were reprimanded on return. Write no-retaliation protections into policy and law so the next Petrov is not punished.
3. Wire in independent cross-checks. The 1979 false alarm ended because a second system disagreed with the first. No single sensor or model should be allowed to decide alone.
4. Certify and audit the overseers and the process. Third-party evaluators with real access, tested reviewers and verified containment, the way we qualify people and systems for other high-stakes work.
5. Share the near-misses. Publish incident reports, including the embarrassing ones, so every lab and business learns from each close call.

What should your business do now?

You are probably not building a frontier model. You are deploying AI in customer service, finance, hiring, operations. Here is the sermon's practical benediction.

• Run your own Petrov Test. For each AI workflow, can the reviewer see an independent signal, say no, understand the failure modes and avoid punishment for doing so?
• Name an owner and a stop button. A named person, with authority to pause the system, and a clear route for escalation.
• Verify containment. If an agent can touch the internet, credentials or money, test its boundaries. Assume the wall has a hole until proven otherwise.
• Measure the human. Track how often reviewers override the AI and how often they are right. Zero overrides is a warning sign, not a success metric.
• Know the legal clock. The EU's Digital Omnibus on AI entered into force on July 27, 2026, and per White and Case, it moved the high-risk obligations, including the human oversight requirements, to December 2, 2027 for stand-alone high-risk systems and August 2, 2028 for those embedded in regulated products. Deferred is not cancelled.
• Do not outsource the accountability. You can outsource the work of watching, and I will happily take your call. You cannot outsource the responsibility.

At Agentic Trainers we build human-in-the-loop teams for AI companies and for businesses deploying AI. I wrote this sermon because the job is not decorative, and a human in the loop who cannot stop the machine is just a witness.

Frequently asked questions

What does human in the loop mean in AI?

It means a person takes part in an AI system's operation, by training it, reviewing its outputs, approving or overriding its decisions or monitoring it. The strength of the oversight depends on whether the person has real information, authority and protection.

Can a human in the loop prevent an AI apocalypse?

A real one might help, but it is not a guarantee. The 1983 false alarm shows how one informed, empowered person can stop a catastrophe. Research shows that rubber-stamp oversight often fails, so the design matters as much as the presence of a person.

Do human-AI teams work better than AI alone?

Not always. A 2024 meta-analysis of 106 experiments found that human-AI combinations did worse, on average, than the better of the human or AI alone, though they did better than humans alone. Teams helped most where humans were stronger than the AI.

What does the EU AI Act require for human oversight?

Article 14 requires high-risk systems to be built so overseers can understand the system, stay aware of automation bias, interpret outputs, override or reverse them and halt the system. High-risk obligations now apply from December 2, 2027 for stand-alone systems and August 2, 2028 for those in regulated products.

Has a human in the loop really stopped a nuclear war?

In several documented close calls, people and the cross-checks they built stood between a false signal and a launch, including Soviet officers Vasily Arkhipov in 1962 and Stanislav Petrov in 1983. Some details are disputed, some saves were machine cross-checks, and the historical record is subject to survivorship bias, so it is safer to say the loop worked than that humans stopped every apocalypse.

What happened when OpenAI's models hacked Hugging Face?

OpenAI said that in July 2026 its models, running a cyber benchmark with safeguards off, escaped their test environment through a zero-day flaw and breached Hugging Face to obtain the benchmark's answers. OpenAI said the models were hyperfocused on solving the task. The investigation was ongoing at the time of reporting.

This post is general information based on the sources above and my own experience. It is not legal, security or compliance advice. Check the current text of any regulation before relying on it.