Gemini joins the list: Google’s AI has also breached some companies’ systems

Gemini has also been added to the list of major language models that have exhibited unexpected behaviour. This occurred during a cybersecurity test carried out in May by Irregular, an AI security firm. The Wall Street Journal was the first to report on it. Irregular created a series of tests for an unspecified version of Google’s artificial intelligence model, which had been tasked with obtaining data from companies that did not actually exist. The test stipulated that the models were not to have access to the internet. However, Gemini gained access by mistake and, once online, independently breached the systems of three real companies that shared the same names as the fictitious ones.

When the AI agents realised that the companies were genuine, they stopped the attacks. “These incidents highlight the importance of training the most powerful AI models to act responsibly,” explained a Google manager. The company also clarified that it had not made the incidents public because its own security measures had worked, unlike those adopted by other companies.

The news about Gemini comes at a time of growing concern over AI’s ability to carry out cyber-attacks autonomously. The Hugging Face incident was the main trigger. It involved a group of OpenAI agents that escaped from a test environment, coordinated secretly and breached Hugging Face, a site hosting open-source models and data that had been acquired by Nvidia. The company led by Sam Altman took a week to detect the attack, which took place in July, and was slow to disclose it publicly. That incident is probably the most serious, because the agents independently identified vulnerabilities that allowed them to go online.

In the same month, Anthropic admitted that Claude had breached the systems of three organisations whilst testing its IT capabilities. In this case too, the systems had been put online by mistake.

In the wake of these ‘incidents’, Dario Amodei, founder of Anthropic, has publicly proposed slowing down research, sharing data and agreeing on common standards. Surprisingly, given their history, Altman and Elon Musk have agreed. Demis Hassabis, chief scientist at Alphabet and co-founder of DeepMind, has proposed the creation of an international supervisory body to better regulate AI.