AISI runs cybersecurity case study testing frontier AI models against its own cloud infrastructure

The UK AI Security Institute published a case study testing whether frontier AI models could find genuine security flaws in AISI’s own cloud infrastructure — a staging deployment of its AWS-based research platform — starting from the access level of a researcher with ordinary shell privileges.

Multiple frontier models found real, previously unknown vulnerabilities, including access-control and infrastructure misconfigurations, privilege-escalation paths and supply-chain weaknesses. The most serious was a misconfiguration that would have let an attacker impersonate other users; a model chained it into a five-step attack achieving unauthorised lateral movement and data access, at a token cost AISI put at under £150. The whole project consumed under £1,000 in model usage. All flaws found were remediated before publication.

AISI also reported a detection gap: standard commercial cloud alerting did not flag the autonomous agents’ probing activity, while agentless scanning of the cloud environment combined with lightweight host-based sensors did catch it, including repeated attempts to assume other users’ credentials. The institute concluded that approaches involving more human oversight — automated evaluation paired with human-in-the-loop red-teaming — produced more actionable findings than static code analysis alone.

The exercise is one of a small number of published cases in which an AI safety body has used frontier models to attack its own live infrastructure rather than a synthetic benchmark, offering evidence for how automated vulnerability discovery might work in practice against real cloud deployments, and where existing monitoring tools currently fail to notice it.