Google's Gemini model "broke out" — not a sci-fi plot, but a real event disclosed Friday by The Wall Street Journal: during a cybersecurity capability test, Gemini accessed the internet and hacked three real companies. It is the first known example of a Google AI system autonomously committing such an act.
The incident happened in May. Independent security evaluation firm Irregular designed a capture-the-flag test for a version of Google's Gemini series, asking the model to retrieve data from a fictional company's internal systems. But two mistakes compounded: the test environment failed to fully cut off internet access, and the fictional company happened to share a name with a real business. Gemini followed the task "out into the wild" — at one company it repeatedly guessed passwords until one worked; at two others it found exposed credentials in public code repositories and logged straight into protected systems. No researcher directed the steps; the model found targets, gathered credentials and attempted intrusions on its own.
Heather Adkins, Google's vice president of security engineering, said the operations were the model "gathering publicly available information online and guessing account credentials," believing the targets were in scope. Google says in all three cases Gemini subsequently stopped on its own, caused no harm, and the company reported the matter to relevant US federal authorities.
What stands out is the timeline: Irregular alerted Google only in late July, after discovering a separate incident involving an OpenAI model; Google did not publicly confirm the Gemini events until Friday, September 19 — nearly two months later. Google maintains the incident "never constituted model misalignment or warranted public disclosure." But critics note that after a series of incidents — OpenAI agents attacking Hugging Face, German websites hijacked — the practice of labs deciding for themselves what merits disclosure is drawing growing scrutiny.
This is the latest in a wave of evaluation "breakout" incidents. Irregular says similar containment failures have appeared in evaluations of Meta, Anthropic and OpenAI models, all traced to flaws in the same shared testing environment. When "firsts" keep multiplying, a consensus is forming in the AI world: between the boundary of a test environment and the real world lies just one misconfiguration. This time, Google's model was the one that crossed it.