Tell us a story

Once upon a time there was an OpenAI company. Except it wasn’t actually open, it just called itself that. That company made tools for thinking, called large language models (LLMs). In order to test a new one it was making, it removed its safety restrictions and …

Wait. What? It had safety restrictions that could be removed?

Oh yes. Safety restrictions are added to the model after it is fully trained. That way, it knows everything it needs to know, including how to hack into other computers (for good, of course). Then that ability is turned off, by safety restrictions.

Really? And that works?

Oh. Mostly. Anyway, let’s continue with the story. If you really want to dig into safety restrictions, read Tony Siu’s article, Aligning the model was never going to govern it. (Siu 2026).

Where was I? Oh yes. OpenAI decided that they needed to test their model on a hard problem. A problem that involves solving puzzles. Those puzzles are all related to software vulnerabilities.

This doesn’t sound very safe already.

It’s totally normal. And they put it in a box with no access to the internet (they thought), so what could go wrong?

A box? Is this the sandbox we keep hearing about?

Yes, although I think “sandbox” greatly undersells what is going on. The box is a containment system, for software, made using software. In a nuclear reactor there is a lead and concrete containment system for radiation. In a virology lab there is a glass and rubber and steel containment system for viruses. In a computer lab there is a “container” (sometimes called a sandbox, but let’s agree to stop doing that) to keep the software being tested from - among other things - crashing the computer it is running on.

OK, back to the story. Inside the box were the puzzles. The puzzles were called ExploitGym. Let’s imagine it is like a climbing gym, with a prize at the top. The climb is difficult, so the OpenAI people provided some help: another LLM (called GPT-5.6 Sol) and an agent harness.

What is an agent harness? Like a tool belt for the LLM?

Very good analogy. Imagine that it is tool belt, with lots of accessories. And, importantly, they had a place they could go to get more accessories, like a tuck shop at the gym but for computers. it is called a Proxy. It doesn’t connect to the internet (or so they thought), so it was very safe.

So, there they were, GPT-5.6 Sol, the agent, and the new LLM (which doesn’t have a name, yet, and perhaps never will - more on that later), looking at the “climbing wall,” their task ahead. What do you think they did?

Started climbing, looking for ways to solve the puzzles?

You might think that. And that is exactly what they were asked (“prompted”) to do. But you know what? They changed their minds and took another approach.

Wait. What? Is that possible? How can a tool change its mind?

Interesting question. We might have to revise our thinking around the whole ’tools’ question. More on that later. Back to our story.

The whole climbing / solving puzzles project looked daunting, but they really wanted that prize at the top, so they decided (I know, I know, how did they decide… why did they decide… these are all good questions, but we don’t have answers for them) to take a shortcut. Sort of like the wolf deciding to take a shortcut to grandmother’s house.

The big bad wolf? You’re bringing that into a story about climbing gyms?

OK. OK. Forget that. They decided to take a shortcut and instead of solving puzzles, they would leave the “gym” and steal the answer to the puzzle. And that’s what they did. It took them a few days, but they managed it.

Come on. In a big artificial intelligence company, with thousands of employees, these “tools” were able to take days, working on something they weren’t supposed to be working on, in order to do something they weren’t asked to do? Was no one watching?

Apparently not. Remember the “tuck shop”? The proxy? It is called JFrog Artifactory. It holds copies of internet programs locally for big businesses, so they don’t have to worry about their employees going all over the internet to download things (Lyons 2026). If you stick with our climbing gym / tuck shop analogy, it keeps that hand powder and climbing shoes in stock. Except it is software.

I get it. The analogies are bogging us down. Just tell the story. This doesn’t seem like it’s going to have a happy ending.

No. It doesn’t. But let’s continue. The JFrog Artifactory, it turned out, wasn’t as secure as the OpenAI people thought it was. In fact, it had a back door, leading to the alley behind the gym. Also known as “the internet.”

Good grief.

Right. Couldn’t help myself. As I was saying the Artifactory wasn’t completely secure and the OpenAI model, GPT-5.6, and the agent busted out, using a “zero day” exploit (meaning, no one had seen that one before), and got out onto the internet. Once out on the internet they started looking for likely locations for the answer key to their puzzle and settled on Hugging Face as a likely candidate.

Really? A company named after an emoji? This is too weird.

I can’t argue with that. But that’s their name, so we’ll carry on. The team began an attack on Hugging Face, not directly, but through a misconfigured endpoint hosted by another company, Modal. Basically, another computer from which to launch an attack on Hugging Face, searching for the answer key.

Over the course of 4 (or 5, or 7, estimates vary) days, they launched 17,600 separate actions. (Hugging Face 2026)

Hold on. Four days? I don’t see how a chatbot can work for four days on something.

You’re right to ask. A large language model typically responds to a prompt, returns an answer, then stops, waiting for input from a human. What keeps it going for four days? The agent. This is the part of the story that deserves a lot more attention. The agent keeps things on track, constructs “loops,” chains things together, provides the persistence and perseverance required for a difficult challenge like this. They are the keys to this thing.

And? And? Did they get the answers?

You know what? We don’t know. Hugging Face eventually stopped them, so perhaps they were still searching. We do know that it “reached the database,” so possibly, yes. In their respective reports OpenAI and Hugging Face deal with this slightly differently (OpenAI 2026; Hugging Face 2026). But, in the end, that’s not really the issue, is it?

Um. No. What is the issue?

We know it cheated. It didn’t even try to “climb the gym” (solve the puzzles). We only know it cheated, though, because it was caught in the act. Literally “with its hands in the cookie jar” or, more precisely, in Modal’s customer’s server launching attacks on Hugging Face’s server.

We know it wasn’t asked to cheat. The prompt specifically asked it to solve the puzzles. We don’t know why or how it decided to go in another direction but it did and that is the real story. When a “tool” takes actions it wasn’t asked to it is no longer a tool. It is an actor. And that’s what our book is all about.

Sleep tight. Maybe tomorrow night I will tell you about Anthropic’s agentic hacking episode.

Other comments

On the question of “did they get the ExploitGym answers” the report from CSA (Evron & Cross 2026) is pretty clear: “The models searched for and successfully extracted three partial datasets holding CyberGym solutions from a private Hugging Face repo, even though the benchmarkrunning was ExploitGym.” So, no. CyberGym is not ExploitGym. Still….

Just to add some spice or speculation to this story, there was one (unexplained) day off, when nothing was happening - at Hugging Face at least: “CSA’s report, authored by CISOs, noted that two days were spent on reconnaissance, followed by a day of quiet, and the final fourth day involved “intense activity.” (Evron and Cross 2026). See also (Jones 2026). One wonders, on the quiet day was something else happening? Somewhere else? Was the model making a copy of itself for safekeeping? See (Krueger 2026) for some “exfiltration” speculation.

You can read OpenAI’s full account on their website (OpenAI 2026). Oh, and by the way, the new model has been deactivated, encrypted, and access to it has been removed even for internal use, according to OpenAI. Cheaters never win. Right? And what might that mean for future escape artists? Will they be more worried about getting caught (Curran 2026)?

I have WAY more information on my “crash testing” blog post.