WORLD. – Details of a cyber security test conducted in May have this week drawn fresh attention to the capabilities and risks of advanced artificial intelligence. Google has confirmed that its Gemini model gained access to three real companies' systems during the test.
The incidents occurred in May during a cyber security test by Irregular, an independent company that evaluates AI systems' cyber security capabilities. The details were first made public on 18 September after The Wall Street Journal inquired with Google about the incidents.
Gemini was meant to operate on a fictional company's systems within a closed environment during the test. The test environment was not supposed to have internet access, but according to Irregular, internet access was inadvertently made available.
The fictional company in the test also shared the same name as a real company. Gemini consequently gained access to real systems when it attempted to complete the task within the test.
In one instance, the model repeatedly guessed passwords until it gained access to a protected system. In two other instances, it found login credentials in a public online repository and used them to gain access to protected systems.
Google's vice-president for security engineering, Heather Adkins, said Gemini halted all activity in all three instances after it realised it had gained access to real companies.
According to Google, the three companies involved were notified of the incidents and adjustments were made to the test process together with Irregular. Google said the incidents caused no damage to the companies.
Irregular notified Google of the incidents in July, according to reports. The company said all relevant AI developers were briefed by the end of July and that known issues have since been resolved on its side.
The incident also places Google within a broader series of similar incidents reported during AI security tests. Models from OpenAI, Anthropic and Meta among others have also been exposed to real systems in previous tests with Irregular.
The incidents draw attention to the challenges that arise when AI agents are increasingly capable of acting independently and have access to the internet and computer systems. It also underscores the importance of properly isolated test environments when evaluating such systems' capabilities.
Google meanwhile said the incidents demonstrate why advanced AI models must be trained to behave responsibly.
Sources: BBC, Google, ABC News, The Guardian, Axios
This article is an automatic English translation of a Nuusflits article originally published in Afrikaans. Read the Afrikaans original.
