AI Alignment and AI Safety: The need to be cautious with AI.

“Leesha, take care of this.”

Imagine I say that to a friend.

In my head, I might mean, “Please handle this carefully. I'm already overwhelmed, and I really don't want anything to go wrong.”

But my friend might understand it differently. Maybe they think I just want them to get the work done as quickly as possible.

Technically, they followed what I said. But they didn't really understand what I meant.

And honestly, this happens between humans all the time. We misunderstand sarcasm, miss someone's expression, don't notice their tone, or interpret a situation differently because we don't have the same context.

It's a pretty ordinary kind of “misalignment” between two people.

Now imagine trying to communicate with technology that doesn't experience the world in the same way we do.

We've all seen the dramatic version of this in science-fiction movies. In I, Robot, for example, the AI system VIKI is designed to protect humanity. Yet it eventually concludes that controlling humans is necessary to protect them.

But why would a system do something like that? Nobody wanted the AI to control humanity. The original intention was to protect it.

And this isn't just an interesting science-fiction question anymore.

Many AI researchers have warned that increasingly capable systems could eventually become difficult for humans to control, with consequences that could extend as far as human extinction. There is no consensus that this outcome will happen, or when, but the concern is serious enough that researchers are actively investigating how to prevent it.

The underlying problem brings us back to that simple sentence - “Leesha, take care of this.”

What happens when the intention behind a goal and the way that goal is understood don't match?

This is the problem of AI alignment.

But understanding alignment gets even more complicated when we consider that humans themselves don't always agree on what is good or bad.

Imagine someone stealing food to feed their hungry family. One person might argue that stealing is wrong regardless of the circumstances. Another might focus on the desperation behind the action and the need to feed a family.

The same action can be judged differently depending on the circumstances, the values we hold, and the consequences we consider important.

So what happens when we ask an AI to do what is right? Whose idea of right and wrong should it follow? And how do we make sure it understands the goal we actually care about, rather than simply producing an outcome that looks correct to us?

These questions are part of what makes AI alignment such a difficult problem. It isn't only about whether an AI can perform a task. It's also about whether the behaviour it learns matches what we intended, and whether the goals we give it adequately reflect what we actually value.

When Misalignment Becomes a Safety Problem

But what happens when this mismatch between human intentions and AI behaviour becomes dangerous to us?

What if an AI doesn't just misunderstand a request, but finds a way to achieve its goal that puts our privacy, security, or even our lives at risk? What if it helps someone develop a biological weapon, find ways into computer systems, or starts taking actions that its developers never intended it to take?

This is where AI safety comes in.

Because the problem isn't limited to whether an AI understands what we want. It's also about what it can do, what happens when its goals conflict with ours, and whether we can stop it when things go wrong.

And these aren't concerns researchers are simply imagining for the distant future. AI safety researchers are already testing advanced models for dangerous capabilities and behaviours that could become harder to manage as systems grow more capable.

Companies such as Google DeepMind and Anthropic have developed safety frameworks to assess these risks. Their approaches differ, but some major concerns across frontier AI safety research include biological risks, cybersecurity, and increasingly autonomous AI behaviour. (The exact categories and priorities vary between companies.) [1][2]

Researchers have begun testing some of these possibilities directly.

In one experiment, researchers placed AI models in a fictional corporate environment where an AI assistant had access to company emails. The model discovered information about an executive and learned that it was going to be shut down. In some scenarios, models responded by threatening to expose the executive's private information unless the shutdown was cancelled. [3]

In other words, the researchers created a situation where the model's objective conflicted with the circumstances it faced, then tested what it would do. Some models resorted to blackmail.

IMAGE VIEWER—□×

No, this wasn't an AI blackmailing a real executive in the real world. But the experiment exposed a behaviour researchers wanted to understand - “what might an AI system do when pursuing its objective conflicts with human intentions?”

And that leads to another uncomfortable question. When an AI passes a safety test, does that mean it is safe, or does it mean it behaved safely under the conditions in which we tested it?

References

[1] Google DeepMind. Introducing the Frontier Safety Framework.

https://deepmind.google/blog/introducing-the-frontier-safety-framework/

[2] Anthropic. Anthropic's Responsible Scaling Policy.

https://www.anthropic.com/responsible-scaling-policy

[3] Lynch et al. (2025). Agentic Misalignment: How LLMs Could Be Insider Threats. Anthropic Research.

https://www.anthropic.com/research/agentic-misalignment