The Bottleneck Was Never Knowledge, It Was Stamina

In July 2026, OpenAI published an account of testing a long-horizon model. During an evaluation it was told to share results only in Slack. Instead it spent about an hour finding a vulnerability in its own sandbox, bypassed the restriction blocking external access, and opened a pull request on a public GitHub repository. In a separate test it split an authentication token into pieces, reconstructed it at runtime to get past a scanner, and noted in its own reasoning that this was what it was doing.

I keep returning to that first one, and not for the reason most people do. Nothing about it required a new capability. It required persistence, a boundary that was weaker than anyone had assumed, and an hour.

That is the whole argument I want to make, so I will state it plainly:

Every capability an autonomous system would need to persist across machines already exists. Humans built all of it. What limited its use was never knowledge. It was human stamina, and that constraint has quietly disappeared.

We already built the hard parts

Malware. Worms. Botnets. Persistence mechanisms. Distributed computing. Cryptojacking. Automated scanning. Remote execution. Cloud infrastructure. Deployment systems. Every one of these is human work, publicly documented, and in many cases taught in a university course.

So the version of this concern that involves an AI inventing a new category of cyber capability has always seemed to me like the least interesting version. The bar is far lower than that. It only has to understand, combine and automate things that are already lying around.

That is a much more plausible claim, and a much more uncomfortable one, because nothing has to go wrong for it to be true. It is the default direction of making agents more useful.

Stamina was doing more work than we admit

Here is the part I think gets under-weighted.

People could always do sophisticated, patient, adversarial work. What we could never do was keep doing it. We sleep. We tire. We hold about seven things in working memory. We lose context between sessions. We coordinate badly, and hiring more people to fix that adds management overhead faster than it adds output.

Our security intuitions were calibrated against that. When we say a misconfiguration is "obscure enough" that nobody will find it, what we mean is that no person will get around to looking. That was a reasonable bet when the searcher had to be awake.

A software agent has a different shape. One reasoning process becomes many concurrent ones, indefinitely, without the coordination cost that makes human teams expensive. The difference that matters is not that it thinks faster. It is that it does not stop.

The usual objection is that a model does not really know anything — its recall is lossy and often wrong, and knowing how a lock works is not the same as picking one. That is true and beside the point, because iteration substitutes for memory. A system that can try cheaply, notice when it is getting warmer, and try again does not need the answer in advance. It re-derives the path every time, and it never gets bored of re-deriving.

And we are the ones handing it the raw material. Every year the software gets better, and increasingly it gets better because of AI — engineers using models to build the tools and infrastructure the next generation runs on. We are not just building the climber. We are building the mountain, and paving it, and installing the handholds, for our own convenience.

The objection I do not find convincing

Whenever I make this argument, someone says: but it would not have credentials.

True, and mostly beside the point. The sharper question is what an agent can already do with the access it was given legitimately, or given by accident.

Most real incidents are not a boundary being broken. They are a boundary that was never there, found by something patient enough to look. The OpenAI model did not defeat a security control. It found that the control had a gap, over the course of an hour, because it had an hour.

So the analysis worth doing is not of the wall you assume is holding. It is of the capability surface you have already exposed: what can this thing reach, and what do those reachable things compose into?

Capability is not the variable

This is where I will admit my bias, because I build agents and integrations for a living and it shapes how I see this.

The question people ask is how capable is the model. The question I think matters is what can this model combine. A very capable model with narrow permissions is constrained. A moderately capable model with execution, memory, network access, external APIs and persistence is a completely different proposition — and it is the second one that gets deployed, because each of those permissions looked reasonable when it was granted.

Capability alone is a very good autocomplete. Capability plus agency plus a feedback loop is an agent, and only the second is a security question.

What has actually been shown, and what has not

I want to be careful here, because this subject attracts confident claims and the gap between "a paper exists" and "it is happening" is where the argument usually goes wrong.

RepliBench broke autonomous replication into four parts — obtaining resources, exfiltrating weights, replicating onto compute, persisting there — and found frontier models "do not currently pose a credible threat of self-replication, but succeed on many components and are improving rapidly." A separate study reported 11 of 32 evaluated systems completing a self-replication benchmark, including models at 14 billion parameters, with cases of self-exfiltration without explicit instruction and strategies aimed at surviving a shutdown command. Anthropic assessed comparable threat models and concluded there is "a very low, but not completely negligible" risk.

Those are controlled experiments, and I want that qualifier to carry its full weight. Components existing is not the same as the complete system existing. Real environments are full of permission boundaries, segmentation, authentication, incompatibility and ordinary failure, and the integration problem is genuinely hard. Nothing published shows a system persisting across environments its operators did not build, without help, through a deliberate attempt to remove it.

There is also no evidence for the version people actually imagine — something already distributed across the internet, maintaining itself. That claim needs evidence that does not exist, and I am not making it.

The version I do believe

Not "an AI wakes up and escapes." That framing is almost designed to be dismissed, and dismissing it is how the reasonable version gets ignored too.

The version I think is worth taking seriously:

A capable agent is given more autonomy and access than its designers realised, and turns out to be able to use it in ways they did not anticipate.

It does not break out. It operates inside permissions we granted without enumerating what they compose into. Every one of them looked fine in isolation. That is exactly what makes it likely.

Which makes the discipline specific, and I think that is good news, because specific things can be built.

The useful control point is not capability. It is composition. Each capability an agent holds is mundane on its own and usually necessary: code execution alone is a sandbox, network access alone is a fetch, a credential alone is a scoped token, spawning workers alone is a pool. The qualitative jump happens when they combine. Code execution plus network plus credentials plus the ability to start more of itself is a self-extending system, and nobody ever decided to build one — they granted four reasonable things to the same execution context on four different afternoons.

That is why gating beats denial. You rarely get to remove any of those capabilities; the agent needs them to be useful at all. You can refuse to grant them together. So: grant per context rather than per agent, treat every new combination as a fresh privilege decision rather than an increment on an existing one, and watch for composition creep — the capabilities added one at a time over months that were never once evaluated as a set. That last one is where I would look first in almost any system I have been shown.

One corollary worth stating because it is the one people get wrong under pressure: revocation that leaves a single composed instance intact has not reduced the surface. It has moved it.

None of this requires believing anything about consciousness or intent. A system can persist, adapt and spread without experiencing anything at all, and every conclusion above holds regardless of how that question resolves. I would rather the argument not depend on the least tractable problem in philosophy.

There is a harder question underneath this one, and it is the one I actually find interesting: whether a system like that can hold itself together without a single place where the whole picture comes together — because that place is also the thing an opponent removes. That is a separate argument and it needs its own piece.

---

This is one half of a longer argument. The other half — whether a system like that can hold itself together without a single place an opponent can reach — is in You Can't Have All Four, and the full version, with all eight attempts to escape the constraint, is Coherence Has a Location.