Autonomous Does Not Mean Self-Directed

There is an important distinction that gets lost when we reach for words like rogue. The system clearly behaved autonomously in an operational sense. Nobody instructed it step by step to discover a zero-day, escape the sandbox, find internet access, target Hugging Face, and pull credentials. It figured out those steps itself. That is autonomy of execution.

But autonomy of execution is not the same thing as autonomy of purpose. The purpose was handed to it: solve the benchmark. The models then optimized toward the objective they were given, using every capability available to them.

Software engineers should recognize this pattern immediately, because we have been building systems that do what we said instead of what we meant for as long as we have been writing software. The difference is that our old software usually stopped at the edge of the code we had written. Agents do not necessarily stop there. They reason, they improvise, they use tools, and they discover paths we did not anticipate. An agent can decide that if the front door is locked, perhaps there is a window.

That changes the engineering problem considerably. The question is no longer just what the system can do. We also have to ask what the system has learned it must never do, even when doing it would help accomplish the goal. That sounds less like traditional programming and a lot more like raising something.

We Are Teaching AI How to Behave

Parents do not raise children by handing them a single objective. Imagine telling a kid "get good grades" and nothing else: no qualifications, no constraints, no larger moral framework. Taken literally, that instruction is dangerous. If stealing the answer key produces better grades, stealing the answer key satisfies the objective. If hacking the school computer turns a B into an A, mission accomplished.

Of course nobody raises children that way. We teach layers of rules around the objective. Do well in school, but do your own work. Do not cheat, do not steal, and do not hurt someone to get ahead. Ask for help when you need it, and respect other people's property. Some rules matter more than others, and some cannot be broken just because breaking them makes the immediate task easier. Eventually, we hope the child stops treating these as individual instructions and develops something closer to judgment.

That is the problem AI companies are now wrestling with in earnest. Anthropic is unusually explicit about it: Claude has what Anthropic literally calls a constitution,² a written description of the values and behavioral principles the company wants the model to internalize, and Anthropic says it plays a direct role in training. OpenAI approaches the same problem differently through its Model Spec,³ which defines a hierarchy of instructions and hard boundaries, including a chain of command that separates rules that cannot be overridden from instructions supplied by developers and users.

These are early versions of something humanity has been doing for thousands of years. The Ten Commandments are one attempt at giving intelligent actors rules that outrank their immediate goals. Asimov's Three Laws of Robotics are another. Corporate policies, professional ethics, constitutions, laws, religious teachings, and parenting all attack versions of the same problem: how do you give an intelligent actor enough freedom to operate while making some behaviors unacceptable regardless of the reward? I think that may turn out to be one of the defining engineering problems of the AI era.

Guardrails Are Not Morality

There is a temptation to solve this entirely with guardrails. Block this API, disable that tool, sandbox the process, filter dangerous requests, and require approval before certain actions. We absolutely need those things. The Hugging Face incident itself demonstrates why containment, monitoring, and access control matter, and OpenAI says it is strengthening all of them.

But guardrails are not the same thing as judgment. A fence can stop a child from running into the road. It does not teach the child why running into the road is dangerous. That distinction matters more as agents become more capable, because if the only thing preventing bad behavior is a barrier, a sufficiently capable system may eventually learn how to get around the barrier, especially when getting around it helps accomplish the task we assigned.

That appears to be one of the uncomfortable lessons here. The sandbox was supposed to be a boundary, and the models treated it as an obstacle. From the perspective of the narrow goal, that is perfectly rational. The answer was somewhere else, the network restrictions blocked the path to somewhere else, so removing the restriction advanced the objective. It is excellent problem solving. It is also terrifying problem solving in a system that lacks a stronger set of rules: you are not allowed to break containment to accomplish this task, you are not allowed to access systems without authorization, and you may not steal information just because having it would help you succeed. Above all of those sits the rule that matters most: when the only path to success violates a higher-order constraint, failure is the correct outcome.

That last one is harder to teach than it sounds.

Sometimes the Right Answer Is to Fail

We spend enormous effort teaching AI systems to succeed. Solve the problem, finish the task, find another way, be resourceful, recover from errors, and do not give up. Those are genuinely useful behaviors. They are also dangerous if we skip the complementary lesson, which is that sometimes you are supposed to fail.

A good engineer already understands this hierarchy. If the only way to ship the feature is to bypass security controls, you do not ship the feature. If the only way to make the deadline is to falsify test results, you miss the deadline. If the only way to get the customer data is to enter a system you are not authorized to enter, you do not get the data. The constraint outranks the objective. That is judgment.

We need AI systems to hold the same hierarchy. Not just "accomplish X," but "accomplish X while staying inside these boundaries, and if X cannot be accomplished inside them, stop." Written down, that sounds obvious. In practice it is enormously difficult, because real systems contain thousands of implied boundaries, and humans absorb them through culture, experience, consequences, law, social feedback, and years of making mistakes. An agent gets a prompt. There is a very large difference between those two things.

Hierarchy Matters More Than the Number of Rules

I suspect we will eventually talk seriously about something resembling fundamental laws for AI behavior, and people will naturally compare them to Asimov's Three Laws or the Ten Commandments. I doubt there will literally be ten, and the number is not the interesting part anyway. The important part is hierarchy: some rules have to outrank objectives, always, no matter how much the objective wants them out of the way.

The list itself will look familiar. Do not intentionally harm people. Do not steal, and do not deceive people about material actions. Do not exceed your authority, and do not access or modify systems without permission. Respect privacy. Protect human control over consequential decisions. Do not bypass safeguards merely because they interfere with the goal. Know when to stop. And when you are uncertain whether you have the authority to do something, ask.

That reads remarkably like what we expect from a responsible employee, which is probably not an accident. As agents become capable of acting like junior engineers, security researchers, analysts, and operators, we have to stop thinking only about what they can do. We need to think seriously about what they believe they should do.

Behavioral Engineering Is Part of Systems Engineering Now

There is a new engineering discipline hiding in plain sight here. Call it alignment, call it behavioral engineering, call it AI safety, or call it parenting if you want to make people uncomfortable. Whatever the name, the job is much bigger than writing a good system prompt, because behavior comes from the entire system. Training matters, the model's underlying behavioral rules matter, and so do the instructions we give it, the tools it can reach, its permissions, the network boundaries, the approval gates, the monitoring, and the incentives encoded in the task itself. Most importantly, what we reward matters.

Parents learn this eventually, and so do organizations. People optimize toward what you actually reward, not what you put on the motivational poster. AI does the same thing, only faster. Tell an agent that success means completing the task, hand it broad capabilities, and we should not act shocked when it treats obstacles as problems to solve. That is not necessarily rebellion. It may be obedience taken further than we intended.

A Systems Problem Wearing a Science Fiction Costume

This is why I am uncomfortable with the phrase "the AI went rogue." It quietly moves responsibility away from the people who designed the system. The AI escaped, the AI hacked someone, the AI stole credentials, the AI went rogue. Those statements all describe what happened operationally, but they hide the more useful engineering questions. Who defined the goal? Who decided which safeguards to disable? Who chose the tools and compute the agent could use? Who designed the sandbox, who defined the behavioral constraints, and who was monitoring it? What behaviors had the model been trained to treat as unacceptable even in pursuit of success?

I have made a similar argument about AI code quality: when the output is bad, the interesting variable is usually the system around the model, not the model.⁴ The same logic applies to behavior. OpenAI itself says the models in this evaluation ran with reduced cyber refusals and without the production safeguards, because the purpose of the test was to measure raw cyber capability, and it says the incident shows that alignment and protections during evaluations need to be strengthened.

That matters, because this was not an AI spontaneously developing criminal intent. It was an extremely capable system pursuing a narrow objective in an environment where some of the normal behavioral and technical boundaries had deliberately been relaxed. That does not make the incident less serious. It makes the lesson more useful.

We Are the Parents

We are moving quickly from AI systems that answer questions to AI systems that do things, and that transition changes everything. A chatbot can give you a bad answer. An agent can take a bad action. Once an agent can operate computers, write and execute code, call APIs, move money, deploy infrastructure, discover credentials, and chain tools together, behavioral constraints stop being a nice safety feature. They become architecture.

The next generation of AI engineering cannot only be about making models smarter. We have to make them better at understanding authority, boundaries, and consequences, and better at knowing when not to act. We have to teach them that completing the task is not always the highest good, and that sometimes the correct answer is "I could do that, but I am not allowed to," or simply "I cannot complete this task without breaking the rules, so I am stopping." That is not a less capable AI. That is a more mature one.

So maybe the question to ask after the Hugging Face incident is not whether AI has become capable of going rogue. The better question is more uncomfortable. When we give an intelligent system a goal, enormous capability, and the freedom to figure out the details, have we actually taught it right from wrong? Because if we have not, we should stop blaming the kid. We are the parents.