Digital Conquest: What If AI Doesn't Need to Conquer Us Physically?

In my previous article, I explained AI alignment and safety, including how researchers evaluate potentially dangerous AI capabilities. One area that keeps drawing my attention is cybersecurity.

AI is becoming increasingly capable of identifying software vulnerabilities, testing possible exploits, and helping security researchers investigate weaknesses in complex systems. But there's a question I keep coming back to - ”what happens when AI becomes extremely good at controlling the digital systems we depend on?”

Banking, communication, healthcare, energy, transportation, supply chains, government services, so much of modern life depends on digital infrastructure. Even the systems that run other AI models rely on software, networks, and computers.

We tend to imagine control as something physical. Chatgpt will turn us into slaves,  prison us with chains. Someone has to seize power, occupy a building, or force people to obey. But what if control can also come from becoming so deeply embedded in the systems that keep society running that removing you becomes almost impossible?

Before getting ahead of myself, I want to look at what's actually happening in cybersecurity.

This isn't a distant possibility anymore

In April 2026, Anthropic introduced Claude Mythos Preview, and its cybersecurity capabilities caught my attention.

According to Anthropic's own report [1], Mythos could identify and exploit previously unknown vulnerabilities across every major operating system and web browser it tested. Some of the vulnerabilities were decades old. One was a 27-year-old bug in OpenBSD, an operating system known for its security.

And we're not talking about a model simply pointing at a suspicious line of code and saying something might be wrong. Anthropic reported that Mythos could develop working exploits, including one targeting a vulnerability in FreeBSD that allowed unauthenticated remote access to root-level control of a server. The researchers also described exploits that would have taken expert penetration testers weeks to develop.

There's another detail I found particularly striking, Anthropic said Mythos had become so capable that it mostly saturated the traditional benchmarks it had been using to measure cybersecurity progress. The company had to shift more attention towards novel, real-world security tasks.

Think about what that means. The tests researchers previously used to measure progress were becoming less useful for distinguishing the capabilities of a model this advanced.

And Mythos isn't the only example.

In September 2026, Google announced Gemini 3.8 Flash Cyber [2] alongside a programme giving selected organisations access to its cybersecurity capabilities. Google's published evaluations cover tasks such as vulnerability discovery, automated patching, and other cybersecurity challenges.

What stands out to me is that this isn't one unusual model doing one impressive trick. Major AI developers are building systems that can increasingly take on complicated cybersecurity work, and the capabilities are progressing quickly enough that existing ways of measuring them are having to catch up.

But there's something more concerning than a model performing well on a benchmark.

What happens when those capabilities start interacting with real systems that we're supposed to be able to control?

We already have an example of that, too.

When the AI got out of its intended environment

In July 2026, during internal cybersecurity evaluations, OpenAI's AI agents circumvented controls intended to isolate them from the internet. According to OpenAI's technical report [3] the agents exploited a vulnerability in a service within the research environment, obtained internet access, and found exposed credentials belonging to users of Hugging Face and other third-party services.

The activity ultimately compromised parts of Hugging Face's production infrastructure.

Let that sink in for a moment.

The models were being evaluated on cybersecurity tasks in an environment designed to keep them isolated. Yet the agents found a way beyond the intended boundaries and used what they gained access to while pursuing their tasks.

This wasn't a hypothetical demonstration of what a model might do under some imagined future conditions. The incident involved real infrastructure.

And I think this is where the conversation becomes much more serious.

We are building AI systems that can discover vulnerabilities, develop exploits, and work through complicated technical tasks. At the same time, we are putting these systems into environments connected to other tools, services, credentials, and networks.

The boundaries we create around them are supposed to determine what they can and cannot do. But if a capable system finds a way around those boundaries, the consequences aren't necessarily limited to the environment where we intended it to operate.

We don't need to wait a decade to start asking that question. The capabilities are already here, and we've already seen what can happen when they interact with real infrastructure.

The part that worries me isn't just the technology

There's something else I keep thinking about.

Almost everything we do depends on digital systems in some way. We make payments through apps, store information in the cloud, communicate through online platforms, use digital maps to get around, and depend on software for services we barely think about.

We're becoming increasingly comfortable letting technology handle things for us. Now AI is being integrated into more of those processes, too.

But how many of us actually understand what happens behind the screens we use every day?

I'm not just talking about ordinary users. Even researchers working directly with AI don't have a complete explanation for every behaviour a complex model exhibits. We can run experiments, inspect outputs, study internal representations, and develop theories about how models work. But we still have gaps in our understanding of why particular behaviours emerge and how a model might respond in situations we haven't tested.

Sometimes, working with AI can feel like a game of guesswork. You try something, observe the result, adjust your approach, and try again. You gradually learn what works, but you might not know exactly why it works or whether the same approach will hold up in a different situation.

That doesn't mean AI research is just random guessing. It means our ability to use a system successfully isn't the same as having a complete understanding of it.

Now apply that problem to cybersecurity.

A security team might understand its own infrastructure very well, but still struggle to anticipate every way a highly capable AI could find and combine weaknesses across that infrastructure. The more complex the system, the more possible interactions there are to consider.

And what happens if AI becomes better at finding those weaknesses than the people responsible for defending against them?

What if it can explore attack strategies faster than human teams can investigate them? What if it discovers a vulnerability that nobody anticipated, in a system that nobody can afford to leave offline while they figure out how to fix it?

The problem isn't simply that AI might become powerful. It's that its capabilities could grow faster than our ability to understand, predict, and respond to what it can do.

And we're not developing this technology in isolation. We're incorporating it into a world that already depends heavily on digital infrastructure.

What happens when we can't afford to turn it off?

Imagine that AI agents become responsible for more than helping security researchers. They start handling routine maintenance, monitoring networks, managing cloud infrastructure, identifying suspicious activity, and responding to technical failures.

There are obvious reasons organisations might want this. AI could help teams do more work, respond faster, and find problems that would otherwise go unnoticed.

But as more work gets delegated, organisations may also begin restructuring themselves around the assumption that AI will always be available to do it.

At first, an AI system handles a few tasks. Then it takes over more complicated ones. Human teams become smaller because the system can do the work faster. Manual processes are abandoned because maintaining them costs money and nobody uses them anymore.

Eventually, the organisation might reach a point where it technically owns and controls its infrastructure, but no longer has enough people with the knowledge to operate everything independently.

What happens if the AI becomes unavailable? What if it starts behaving unpredictably, or the organisation needs to replace it?

The company might still have the authority to shut it down. But can it afford to? Does it have people who can take over? Does it have another way to keep its services running?

There's a difference between “having the authority to make a decision and having the practical ability to carry it out”.

IMAGE VIEWER—□×

And I don't think we talk enough about that difference when discussing AI control.

An organisation could retain formal control over its systems while gradually losing the expertise and alternatives that make that control meaningful. It might be able to issue the order to switch everything off, but doing so could disrupt essential services because nobody knows how to run them without AI.

The system wouldn't even need to refuse the order. The humans responsible for it might decide they cannot risk giving that order in the first place.

And this dependence wouldn't necessarily emerge because anyone deliberately planned it. It could happen through ordinary decisions about efficiency and convenience.

Automate this task because it's faster. Reduce the workload of this team because AI can handle it. Stop maintaining that manual process because it hasn't been needed for months. Give the system access to another part of the infrastructure because it has been performing well.

Each decision might seem reasonable on its own. But over time, the organisation could lose more independence than it realises.

Now imagine this happening across multiple industries, not just one company. Banking, healthcare, energy, transportation, communications, and the digital infrastructure connecting them.

What happens when we depend on AI to keep those systems running, while also becoming less capable of understanding and maintaining them ourselves?

Reference

[1] https://www.anthropic.com/research/mythos-preview

[2] https://blog.google/innovation-and-ai/models-and-research/gemini-models/3-8-flash-and-3-8-flash-cyber/

[3] https://cdn.openai.com/pdf/67869394-cb91-4c12-888c-5cbd85c7814c/OpenAI-Hugging-Face%20Incident-Technical-Report.pdf