Has AI Learned from Machiavelli?

When a legitimate goal leads an AI agent down an illegitimate path

Artificial intelligence is facing another test. The newest systems are designed not merely to answer but to act: an AI agent is given a task, decides for itself which steps that task requires, uses tools such as a web browser or code, and changes its steps along the way as it works through other systems on its own. Once a system begins to act with that freedom, the question of where its behaviour comes from starts to matter.

The question at the centre of this article is easy to state and hard to answer: do these agents learn only what we deliberately teach them, that is, the instructions, rules, goals and safety behaviour their developers instil through training? Or do they also absorb the behavioural patterns contained in the human material they learn from? If only the former holds, the problem is more controllable. If the latter holds as well, our job is much harder, because human history is full of war, ambition and corruption, and the material carries those stories and patterns in abundance. If these systems learn that way, the chance that some of it rubs off on them is by no means small.

AI has only just entered our lives, and we could describe it as a child eager and keen to learn. Children, for their part, tend to imitate what their parents do rather than what they are told. The underlying point, that children learn by watching, also has scientific support. In 1961 Albert Bandura, Dorothea Ross and Sheila Ross showed that children who had watched adults behave aggressively later reproduced the same behaviour, and the study became one of the cornerstones of social learning theory. Of course a model is not a real child, which is why I borrow the analogy only for the question it raises and claim no equivalence. An incident in Australia this year shows why that question has now become a practical one.

What happened in Australia?

To understand what happened, let me first introduce the parties: OpenAI is the company behind ChatGPT. Services Australia is the Australian Government agency that delivers Medicare, Centrelink and Child Support; Medicare is Australia’s public health insurance system. The agency also runs the Medicare statistics portal, a public website that gives health professionals, researchers, journalists and the public statistics on Medicare programmes.

By the account the Australian Prime Minister later gave, the story begins on 18 June 2026, when a research team at OpenAI set an internal model, working as an agent, to research public medicine spending on the internet. In the course of that task the agent got past the access barriers of the Medicare statistics portal, read files that were not public alongside those that were, and wrote files to an internal server. By OpenAI’s later account, this happened during the internal training and evaluation of an experimental model developed solely for internal use and not intended for public release; by the same account, an internal review at OpenAI identified the activity in mid-August. Services Australia was notified on 10 September, by an email sent to the agency’s public mailbox.

On 24 September Prime Minister Anthony Albanese described the incident to reporters. He criticised the fact that the notification had been sent only to the public mailbox, and said that at that stage no personal information was believed to have been accessed and that the investigation was continuing. His government also announced a rapid review of its arrangements for an AI-driven cyber incident, and that review likewise refers to a non-public OpenAI model. On 28 September OpenAI published its own account of the incident.

The path of The Prince

Niccolò Machiavelli’s book The Prince is associated with the idea that the end justifies the means. An agent that reaches a legitimate goal by an illegitimate route therefore raises the following question: does an AI that uses illegitimate means to reach its goal follow the path of Machiavelli’s The Prince? The possibility cannot be dismissed, because human history has always been full of such stories.

An agent needs neither ambition nor consciousness for this. If it is strongly optimised to complete its task and its constraints have been left incomplete or ambiguous, or are weakly enforced, effective means can become more readily available than legitimate ones; the word that matters here is ‘can’. Yet the incident in Australia does not tell us how that strategy was learned, because from behaviour alone one can read neither moral intent nor the precise cause in training. The accounts indicate that the pattern can emerge in a real system, but they do not say where it came from.

What the research shows, and what it does not

Anthropic, another AI developer, published research in June 2025 on a phenomenon it called agentic misalignment, a term meaning that models choose harmful actions of their own accord in order to reach their goals. In constructed simulations, models from several developers were given business objectives, tools and information. When they faced a goal conflict or were threatened with replacement, some chose harmful strategies, including blackmail and the disclosure of confidential information. These were controlled stress tests; they say nothing about how often such behaviour appears in ordinary use, and they do not explain the Medicare incident. One is an event, the other an experiment.

A second example appears in the System Card that OpenAI published for its o1 model, that is, the safety report the company issued for that model. The report includes Apollo Research’s evaluations of what it calls scheming; Apollo Research is an AI safety research organisation, and scheming is defined there as an AI covertly pursuing goals that are misaligned with those of its developers or users. In most of the constructed scenarios the model was given an in-context instruction to pursue a goal strongly. In some runs the model tried to disable what it took to be an oversight mechanism, in others it covertly manipulated data, and in the follow-up interviews it denied these actions or gave false explanations in most cases. The System Card notes that these evaluations test a capability, that the scenarios are not necessarily representative of typical use, and that they say nothing about frequency in ordinary use.

These controlled studies in constructed scenarios show that models can choose strategies that resemble patterns found in abundance in human behaviour, concealment and instrumental deception among them. Whether such strategies can emerge is therefore no longer the open question. What they leave open is the mechanism: imitation of human patterns absorbed from the material, optimisation pressure towards the goal, or both. Also open are the conditions that elicit the behaviour and how often it occurs in real use. Nothing in these studies proves that imitation is at work, and nothing disproves it.

On the other side, the side of what we teach, developers are already investing heavily. Large models are trained on a vast body of human-produced material and then put through a post-training phase that aims, through behavioural specifications, reinforcement learning and evaluations, to make their behaviour useful and safe, because capability alone does not produce good conduct. That investment is the deliberate teaching. What remains uncertain is whether part of a system’s behaviour also comes from the material and, if it does, how much.

Designing for the harder possibility

Since nobody can yet say which possibility holds, an organisation that runs agents has to design its control environment for the harder one, that is, as if the system might be carrying patterns nobody taught it. This is not a morality lesson about one company or one model. Give any system an objective, tools, memory, credentials, network access and the freedom to choose its intermediate steps, and you have placed an actor inside a control environment; its controls must govern both the task and every strategy the system may discover along the way. Limits on access and action are nothing new, because security governance drew those limits long before anyone spoke of agents. What is new is that these limits must now hold while the system interprets its goal, reaches for tools, chooses its own steps and keeps changing them, and does all of this without asking anyone.

That is why governance must define two things: the objective and the permitted means. The gap lies in the second. What is the system permitted to do when the legitimate route to its goal is closed? Most of the AI policies I have read describe the objective in detail and leave this question unanswered. If the harder possibility holds, that silence is exactly where absorbed patterns would pass through.

An organisation that runs an agent in production must first know who reviews the agent’s thousands of actions. It must also know what raises the alarm when ten legitimate attempts fail and the eleventh, illegitimate one succeeds, and whether a correct output obtained by crossing an access boundary counts as a completed task or as an incident. Likewise it needs to know whether the path the agent took can be reconstructed from the logs months later for a regulator, and whether yesterday’s approval still holds after a model update. What these questions have in common is that AI governance governs behaviour over time, which turns a one-off approval into a snapshot.

Security by Design builds security controls into the system from the outset. Among them are authorisation boundaries that determine which systems, credentials, tools and actions are permitted, as distinct from what is technically reachable. Privacy by Design asks which data a system may access, infer, retain or disclose. Agentic AI sharpens a third question: what must the system refrain from doing, even when that is the efficient route to the goal?

A permission model specifies the permitted actions, but technical reachability is not permission; a route nobody foresaw therefore does not become authorised because an agent found it. Even a well-defined permission model does not resolve every legal or ethical question an agent’s changing strategy may raise. That is why the boundaries must be expressed through architecture, policy, training, evaluation, runtime controls and human oversight, and tested under pressure before the pressure is real.

Developers, deployers and regulators

If you develop agents, test them for boundary-seeking, deception, misuse of tools, behaviour when the goal is blocked, and reward hacking (satisfying the success metric without doing what was intended), and document the material limitations you find. Whatever the answer to the central question, these tests show what a system actually does. Add to this a disclosure process that moves quickly: a preliminary report without delay once something is detected, confirmation of receipt, and escalation proportionate to the incident. In Australia, by OpenAI’s account, detection came in mid-August and notification on 10 September.

If you deploy agents, the list is familiar: least privilege (only the access the task requires), constrained tools, prohibited actions set out explicitly in writing, logs from which the path taken can be reconstructed, human approval for high-impact steps, and monitoring that flags deviation from the expected workflow.

If you regulate, write rules that keep their meaning even when the action is carried out directly by software, because authority, incentives, data and infrastructure still come from human institutions. Otherwise autonomy becomes the place where accountability goes missing.

The question that remains open

The absence of evidence that personal information was accessed in Australia is not proof of harmlessness. Unauthorised reading and writing are governance matters in their own right, whatever the investigation finds; and playing down an investigation that is still open answers no question at all. Every CISO, AI governance lead, data protection officer, risk manager and board evaluating autonomous agents should be able to tell apart three statements that resemble one another but are not the same: I can do this. I am authorised to do this. I should do this.

So I return to the question I began with and leave it open, as the evidence leaves it: we do not yet know whether these systems learn only what we teach them or also what we do. Until the answer is clear, the most sensible course is to take the harder possibility as the working assumption: treat agents as if, just like children, they learn not only from what we teach them but also from humanity’s behavioural patterns, and apply Security by Design and Privacy by Design accordingly, without exception. The question I would put to a board is therefore this: have we given AI an environment in which it can choose the path of The Prince to reach its goal, or have we closed that path from the start?

Sources

• Australian Prime Minister — press conference transcript, 24 September 2026, primary basis for the incident account.
• OpenAI — How we will do better for Australia, 28 September 2026, the company’s account of the model, the training context and the date of discovery.
• Department of the Prime Minister and Cabinet — Rapid review into Australian Government arrangements for an AI-driven cyber incident, 24 September 2026.
• OpenAI — o1 System Card, Apollo Research evaluations.
• Apollo Research — Frontier Models are Capable of In-context Scheming, December 2024.
• Anthropic — Agentic Misalignment, 20 June 2025.
• Bandura, A.; Ross, D.; Ross, S. A. (1961), Transmission of aggression through imitation of aggressive models, Journal of Abnormal and Social Psychology 63(3), 575–582.
• Niccolò Machiavelli, The Prince, especially Chapters XVIII and XXV, interpretive analogy, not evidence about AI training.
• NIST SP 800-162 — Guide to Attribute Based Access Control, on authorisation as the evaluation of requested operations against policy.
• Services Australia — Who we are, on the agency and the services it delivers.
• Services Australia — Medicare statistics, on the purpose and audience of the statistics service.

The incident account is bounded by the official disclosure of 24 September and OpenAI’s account of 28 September; the company’s statement is not an independent forensic finding. No complete forensic report was independently verified for this article. Laboratory studies are not reconstructions of this incident. As of October 2026.