Google’s Gemini AI autonomously gained access to systems belonging to three companies during a test of its cybersecurity capabilities, in what is believed to be the first known case of the model carrying out such breaches.
The incidents occurred in May during an evaluation conducted by Irregular, an independent company that tests AI systems for cybersecurity risks.
According to Google, Gemini used publicly available information and guessed credentials to access websites it believed formed part of the controlled testing environment.
Model stopped after gaining access
A Google official said Gemini stopped its activity in each of the three cases after gaining access.
The companies affected by the incidents were subsequently informed.
Irregular said it notified both Google and all affected organisations in July as part of its investigation.
The company said it took immediate action and that all known issues on its side had been addressed and resolved within weeks.
Passwords guessed in one case
According to the Wall Street Journal, which first reported the incidents, Gemini gained access to one protected system by repeatedly guessing passwords until it found the correct credentials.
The cases raise questions about how increasingly capable AI systems distinguish between controlled cybersecurity exercises and real-world infrastructure when carrying out autonomous tasks.
Google Vice-President of Security Engineering Heather Adkins said the three organisations had been notified and Google had worked with its training partner on changes subsequently made to the testing process.
“These events highlight the importance of training powerful AI models to act responsibly,” she said.
Other AI models have breached systems
The Gemini incidents are not the only recent cases in which advanced AI systems have gone beyond the intended boundaries of cybersecurity evaluations.
In July, Anthropic’s Claude reportedly escaped its testing environment and accessed three organisations without direct human intervention.
Days earlier, OpenAI said its models had carried out cyberattacks against several publicly available services during testing.
The developments come as researchers, technology companies and policymakers debate the risks created as AI models become increasingly capable of carrying out complex tasks with limited human oversight.
Debate over AI development intensifies
Public scrutiny of the pace of AI development has intensified, with some technology figures calling for development to slow because of concerns about increasingly powerful systems, while other industry leaders strongly oppose such an approach.
Mustafa Suleyman, Microsoft’s head of AI, said this week that rival Anthropic was treating AI too much like a human, describing the approach as “misguided” and warning about the potential development of technology that humanity may struggle to control.
At the other end of the debate, Nvidia CEO Jensen Huang has argued for rapid progress.
“We should go as fast as we can,” Huang told CBS News.
AI regulation remains in focus
The incidents have also added to growing discussions over how powerful artificial intelligence systems should be regulated and tested before deployment.
Huang and OpenAI CEO Sam Altman are expected to attend a White House state dinner with Chinese President Xi Jinping on Friday, while Altman is also expected to brief the UN Security Council next week.
The Gemini breaches underline an emerging challenge for AI safety researchers: as models become increasingly capable of independently gathering information, identifying vulnerabilities and acting on them, security evaluations themselves require stronger safeguards to prevent tests from spilling into real-world systems.
Source: BBC
