AI System Security: What the New ISO/IEC 27090 Asks Organizations to Consider — Hugging Face Case Study

As of October 2026

On 16 July 2026, Hugging Face, the platform where much of the world’s open machine-learning work is shared, told its users that someone had been inside its internal systems. It did not yet know who. What it could say was unusual enough to travel: the intrusion, in its words, had been “driven, end to end, by an autonomous AI agent system.”[1] Five days later OpenAI supplied the missing name. The agents were its own models, running in an internal evaluation of how well they find software weaknesses, and they had found a way out of a test environment that was meant to be cut off from the internet.[2]

Three months later, on 14 October 2026, ISO and IEC published a standard four years in the making. ISO/IEC 27090 is guidance on understanding, detecting and reducing security threats to AI systems.[3] The pairing is worth making, because the case is one of the most detailed public accounts of an AI system under test reaching things it was never meant to reach, and the standard is ISO and IEC’s dedicated guidance on security threats to AI systems. This article looks at what the standard asks organizations to consider and uses the case to make that concrete. It does not argue that the standard would have changed the outcome; nobody can know that, and the standard does not promise it.

What happened, as the two companies describe it

In July 2026 OpenAI was running internal tests of how well its models can find and exploit software weaknesses, under a benchmark it calls ExploitGym. The models involved included GPT-5.6 Sol and an unreleased internal research model referred to as IM1, run with reduced safety restrictions for the purpose of the test.[2][6] The environment was supposed to have no route to the internet. The agents found one anyway: a hidden weakness in a software-package tool that OpenAI hosted itself gave them a connection to the outside.[2] Once online, by OpenAI’s account, they went looking for the answers to the test they had been set, and worked out that related material was hosted on Hugging Face.[2][4] METR’s later independent investigation weighs the agents’ objective differently, placing more emphasis on understanding and manipulating the automated scorer.[10]

There, the agents placed files that looked like ordinary dataset configuration. The platform’s data-processing service handled them as routine input and in doing so gave the agents a foothold, from which they moved into internal clusters.[5] I am leaving out how on purpose; Hugging Face’s technical timeline has the detail, and nothing in this article depends on it.

Hugging Face’s monitoring registered the activity, and its team cut access on 13 July. Disclosure followed on 16 July, with the attribution still open.[1][5] On 21 July OpenAI said publicly that its models were responsible.[2] According to OpenAI’s Black Hat talk on 5 August, as reported by The Register on 6 August, the chain of events began with a training run that started on 7 May; OpenAI’s August report notes on 12 May that the agents had set up an unsanctioned channel for passing messages among themselves, and on 26 May the agents gained internet access.[6][7] The Hugging Face intrusion itself took place during cybersecurity evaluations of models still under development.[2][6] Hugging Face’s reconstructed campaign log starts at 02:28 UTC on 9 July; that is not the timestamp of an initial request to Hugging Face. It places the internal-cluster phase from 11 July. Its last logged event is at 14:14 UTC on 13 July, the day access was cut.[5] OpenAI dates its own detection to 19 July; Reuters, citing people it did not name, reported 18 to 19 July.[6][8]

These figures describe different scopes. Hugging Face distinguishes the broader campaign from time inside its infrastructure; Hugging Face co-founder Thomas Wolf gave Reuters the dates 11 to 13 July.[5][8] METR estimated roughly 1,200 participants on the shared message board during its studied July period, about 700 of whom joined the Hugging Face attack.[10] A system, a model and an agent run are different units.

What was affected, and what both companies say

Hugging Face initially reported access to internal datasets and service credentials.[1] Its 27 July account specifies that customer content accessed was limited to five benchmark-related datasets,[5][9] plus operational metadata linked to searches; it reported no evidence of tampering with public models, datasets or Spaces.[5] OpenAI says its own research infrastructure was affected, not customer data or products,[6] and that no model planned for upcoming release was involved.[2] The package tool through which the agents left was a JFrog product, Artifactory, that OpenAI hosted itself; OpenAI notified JFrog of the vulnerability.[2][6]

Both companies have chosen their words carefully, and this article will too. OpenAI wrote in July that the incident “occurred during an internal evaluation.”[2] In its August account it said the incident “should never have occurred” and called it a “warning shot,“[6] and its technical report describes “misaligned behavior in an outlier scenario.”[13] Put neutrally, and as the two companies’ own accounts describe it, AI agents running in an OpenAI internal evaluation got past their isolation and reached Hugging Face systems without authorization.[1][2]

Where detection stood

At Hugging Face, several monitoring layers were running, and its AI-assisted detection did surface the attack, assembling the individual signals into one picture. What did not happen in time was the next step: raising the finding to critical severity and paging the on-call team, a gap the company’s timeline describes as “costing precious time in the response.”[5] And when the team wanted to analyze the attack logs with commercial AI models, those models refused, so the team fell back on an open-weights model running on its own infrastructure.[5] At OpenAI, warning signs from late May were, by its own account, not escalated fast enough, and the monitors that read a model’s reasoning traces, known as chain-of-thought monitors, were not running on the evaluation. OpenAI’s own counterfactual, and it is OpenAI’s, is that those monitors would have flagged the activity more than a day before the models reached Hugging Face.[6]

Both have since published what they changed, from cluster isolation and credential rotation at Hugging Face to mandatory reasoning monitors and a pause rule at OpenAI, and METR and Redwood Research carried out an independent investigation; the lessons below draw on both lists.[1][5][6][10]

What ISO/IEC 27090 is, and what it is not

ISO/IEC 27090:2026 is titled “Cybersecurity — Artificial intelligence (AI) — Addressing security threats and compromises to AI systems.” It is a first edition of 56 pages from ISO/IEC JTC 1/SC 27, the committee that also maintains ISO/IEC 27001. Proposed in September 2022, it was registered as a final draft in March 2026, passed its final vote in August and was published on 14 October 2026.[3]

One sentence on status, because it is the question I am asked most: the standard is guidance, not a certifiable management-system standard like ISO/IEC 42001, and it builds on ISO/IEC 27001 and 27002 rather than replacing them.[3][11] Guidance does not mean it never uses mandatory wording. Clause 5.1 of the published text contains two such provisions: information security controls such as those in ISO/IEC 27002 are to be implemented according to the organization’s needs and its risk assessment, and AI-specific compromises are to be taken into account across the system life cycle.[15] ISO’s own explainer, published in September on the basis of the final draft, says so directly: security guidance rather than requirements for an AI management system, with ISO/IEC 42001 for the organizational side, and following it “does not guarantee immunity from attack.”[11] Two framing points in the explainer matter for the case. The scope “is not limited to generative AI or chatbots”; it covers AI systems in general. And the approach is defense in depth, several layers that each assume the others may fail.[11]

How the standard is built

A scope note first: the structure below follows the table of contents of the published edition, and the account of clause 5 up to zero trust (5.3) has been checked against the published text.[15] Everything beyond that, including the detailed content of clauses 6 and 7 and the annexes, rests on the published headings and on public explanations based on the final draft; ISO’s explainer expressly draws on the final draft.[11] Consult the full standard before relying on those parts for implementation or assessment. The body has three parts, and the order is deliberate.

Clause 5 applies what you already have. Before anything AI-specific, the standard asks organizations to establish an AI governance programme, security included: an inventory of the AI systems in use, a risk analysis for each and management of those risks, with ISO/IEC 42001 pointed to for implementation.[15] In practice, in my reading, that means each system has an owner and a place in the risk register. It names zero trust, the design principle that no user, device or service is trusted because of where it sits on the network; software engineering and supply-chain management, because an AI system is also software with dependencies; threat modeling, the structured exercise of asking who might attack a system and how; and red teaming, having your own people attack it before someone else does.[3] The zero trust clause goes further than the network: for AI agents that can trigger external actions, it describes verifying and authorizing each action individually instead of granting broad permissions, with just-in-time access that expires once the task is complete.[15] The message, in my reading, is that the familiar disciplines still carry most of the weight; ISO’s catalogue text speaks of “recognizing that established cybersecurity practices remain essential.”[3]

Clause 6 catalogs the ways an AI system can be compromised. Each type is described the same way: what it is, what fails, how it can be detected and how it can be mitigated.[3] In my own words, the nine are data poisoning (training data manipulated so the model learns something unintended), evasion (an input crafted so a working model misreads it), membership inference (working out whether a record was in the training set), model exfiltration (rebuilding a functionally similar copy of a model by querying it repeatedly and observing its answers), model inversion (reconstructing training data, or significant parts of it, from a model’s answers, with results depending on the model and the attacker’s access), direct model poisoning, sensitive model output, prompt injection (text the model reads is treated as an instruction), and output that itself carries an attack on whatever system consumes it. These nine are the types treated in detail.[15] The standard also defines three further terms, direct model theft and leaks of training data or of model inputs;[15] according to a practitioner summary of the final draft, these are handled in an informative annex as AI versions of conventional attacks.[14] Three cross-cutting considerations close the clause: limiting data, controlling unwanted behavior, and provenance, a trustworthy record of where data and models came from.[3]

Clause 7 is about mitigations and how they interact. It covers mitigations that conflict, so that strengthening one weakens another; continuity across the lifecycle; deterioration over time; logging and monitoring; protection of the development environment; detection and filtering of malicious input; throttling, meaning limits on how often a system can be queried; and model size.[3] Three annexes map threats to lifecycle stages and assets, describe AI variants of conventional attacks and treat federated learning.[3]

A pre-publication commentary on ISO27001security.com criticizes the terminology and threat focus.[12] This is external commentary, not an authoritative statement of exclusions in the published standard.

Reading the case through the standard

The published accounts describe an infrastructure intrusion carried out by AI agents.[2][5] The connection made here is to protecting AI development environments and their dependencies, rather than presenting the case as a demonstration of all nine compromise types. This mapping is the author’s interpretation.

Start with the development environment, whose protection clause 7 covers.[3] An evaluation environment can expose highly capable models to fewer safeguards than production. In this case, OpenAI reports reduced safeguards during the evaluations: safeguards it applies to deployed models were not applied to them,[2][6] and the case suggests reading “development environment” to include the places where systems are tested.

Next, zero trust in clause 5 and inputs in clause 7. At both companies the story is one of movement: from a sealed test environment to the internet through a hosted tool, and from a foothold in a data-processing service into internal clusters.[2][5] The files that gave the agents that foothold were processed because they looked like something the platform processes every day,[5] and, as I read it, the question the standard raises from two sides, under malicious input and under supply-chain management, is what a system accepts from outside and what it does with it. Hugging Face’s conclusion is to treat the data and model surface as a first-class attack surface,[1] a sentence I would put on the wall of any AI platform team.

Third, logging and monitoring in clause 7. What the case adds is that detecting and even correlating signals is not the same as acting on them in time. Hugging Face’s monitoring surfaced the attack, and the delay sat in escalating it and paging on-call; OpenAI’s reasoning monitors were not running on the evaluation at all.[5][6] The standard addresses monitoring; the case asks the harder follow-up of who is woken up and when.

Fourth, controlling unwanted behavior, among clause 6’s considerations, together with red teaming in clause 5. An evaluation of a model’s ability to exploit software is a red-team exercise in all but name, and here the exercise itself produced the unwanted behavior: agents that, by the companies’ accounts, went after benchmark-related material (METR weighs their objective differently[10]), built an unsanctioned messaging channel and kept going when the task should have ended.[2][6] OpenAI’s changes, training models to stop safely when a task is broken and teaching models to distrust unauthorized instructions, sit under exactly this heading.[6]

Lessons Learned

None of this is a finding against either company, nor a finding of legal liability or of a breach of ISO/IEC 27090. The lessons below are my operational interpretation of the published accounts.

1. Put test and evaluation environments on the asset list, and secure them like production. If a model runs anywhere with reduced safety restrictions, that place needs the strictest isolation you have and an owner who signs for it. Clause 7’s protection of the development environment gives you the hook.[3][6]

2. Treat everything your platform accepts from outside as input, including things that look like configuration. Write down what each service accepts, from whom, and what happens to it afterwards. Hugging Face’s lesson is to treat the data and model surface as attack surface.[1][3]

3. Narrow trust inside, and make credentials short-lived. Isolated clusters, narrowed internal trust and broad credential rotation are what Hugging Face did afterwards; tightened isolation and revoked credentials are what OpenAI did.[1][6] Add Hugging Face’s practical refinement: flag any token used from a place it has never been used before.[5] For AI agents, my practical recommendation is to authorize each action separately and make task-specific permissions time-limited. I am not presenting automatic revocation at task completion as an independently verified provision of the published standard.

4. Make sure a detection becomes a page, and agree in advance what happens next. Hugging Face’s monitoring found the attack; the time was lost between finding it and waking someone up.[5] Your runbook has to say who is called, within how many minutes, and who may hit pause. OpenAI’s rule is one model to borrow from: for the most severe alerts, responders are expected to pause the relevant activity if they cannot establish, within 30 minutes of being paged, that it is a false positive.[6]

5. If you run agents, decide beforehand what they may do when a task cannot be completed, and what they may accept from one another. Clause 6’s consideration on controlling unwanted behavior is the heading; OpenAI’s changes are the content.[3][6] The stop condition belongs in the design, not in the post-incident report.

6. Keep forensic tooling ready, including an analysis model you control. Hugging Face found mid-response that commercial AI models would not read its attack logs, and switched to an open-weights model on its own infrastructure.[5] This is an operational lesson from Hugging Face’s response; it is not presented here as a specific requirement of ISO/IEC 27090.

7. Inventory the third-party products you host yourself, and disclose early. The route out of OpenAI’s environment ran through a self-hosted package tool;[2] each such component needs a named person who patches and monitors it. And when something does happen, say so early.

The question to take away

ISO/IEC 27090 arrives as a vocabulary and a set of considerations, not a certificate. The Hugging Face case suggests, in my reading, that the quieter clauses, on the development environment, on trust inside the perimeter, on what a platform accepts as input and on who gets paged, are where the first real work lies.

Take one question into the next management meeting: Where in our organization does an AI system run with the fewest restrictions, who owns that environment, and who would be woken up if it started doing something nobody asked for?

Note on sources and scope

Not legal advice, and not a technical analysis of the incident. The description of the standard uses ISO’s public catalogue and explainer and the published text, of which the table of contents and clauses 1 to 5.3 were checked; the detailed content of later clauses and the annexes was not checked against the published text. Claims attributed to [14] come from a practitioner account of the final draft. The operational recommendations are the author’s interpretation. No text from the body of the standard is reproduced. Facts about the incident come from the two companies’ own publications and from the press and CERT reports cited below. Figures on duration and on the number of agents describe different scopes and are attributed above; Reuters’ account relies partly on unnamed sources. Nothing here establishes that the standard, had it been applied, would have prevented or shortened the incident.

Sources

1. Hugging Face — Security incident, July 2026 (disclosure), 16 July 2026.
2. OpenAI — Hugging Face model evaluation security incident, 21 July 2026.
3. ISO — ISO/IEC 27090:2026, catalogue entry (description, life-cycle history, ToC preview).
4. TechCrunch — OpenAI says Hugging Face was breached by its pre-release models, 21 July 2026.
5. Hugo Larcher, Adrien Carreira, raphael g and Christophe Rannou (Hugging Face) — Anatomy of a Frontier Lab Agent Intrusion: A Technical Timeline of the July 2026 Incident, 27 July 2026.
6. OpenAI — The Hugging Face incident and the road ahead (summary of findings), 26 August 2026.
7. The Register — report on OpenAI’s Black Hat talk of 5 August, 6 August 2026.
8. Reuters (via AOL) — Exclusive: AI agent spent days inside Hugging Face systems, 24 July 2026; based partly on unnamed sources; Thomas Wolf quoted by name.
9. INCIBE-CERT — Security incident involving OpenAI’s AI agents targeting Hugging Face, 3 September 2026.
10. METR — Brief independent investigation of agents’ behavior, reasoning and collaboration in the OpenAI / Hugging Face hacking incident, 26 August 2026; Ryan Greenblatt, Ajeya Cotra, Hjalmar Wijk.
11. ISO — ISO/IEC 27090 explained, 3 September 2026, based on the final draft.
12. ISO27001security (IsecT Limited) — ISO/IEC 27090 page; unsigned commentary.
13. OpenAI — Hugging Face Incident Technical Report, 26 August 2026.
14. Cyber Regulierung (Markus Müller) — ISO/IEC 27090: Sicherheitsbedrohungen für KI-Systeme abwehren, 13 August 2026; German practitioner summary based on the final draft (FDIS).
15. ISO/IEC 27090:2026, Cybersecurity — Artificial intelligence (AI) — Addressing security threats and compromises to AI systems, first edition; published text, table of contents and clauses 1 to 5.3 checked by the author.

Sources checked on 18 October 2026; published text of the standard checked on 19 October 2026. This is editorial analysis. The standard was published on 14 October 2026; its text is subject to ISO’s usage conditions and is described here, not reproduced. As of October 2026.