Hackers use more than 100,000 prompts to make a Google Gemini clone

Google has revealed that its artificial intelligence (AI) chatbot, Gemini, has been targeted by "commercially motivated" hackers attempting to clone it through repeated prompts. In a recent report, the tech giant noted that one campaign submitted more than 100,000 queries to clone the Gemini chatbot.

In the report, Google said it has been targeted by "distillation attacks", or repeated questions designed to know a chatbot's inner workings. The company described the activity as "model extraction", in which would-be copycats probe the system for the patterns and logic that make it work, apparently to build or improve their own AI.

“In the final quarter of 2025, Google Threat Intelligence Group (GTIG) observed threat actors increasingly integrating artificial intelligence (AI) to accelerate the attack lifecycle, achieving productivity gains in reconnaissance, social engineering, and malware development. This report serves as an update to our November 2025 findings regarding the advances in threat actor usage of AI tools. By identifying these early indicators and offensive proofs of concept, GTIG aims to arm defenders with the intelligence necessary to anticipate the next phase of AI-enabled threats, proactively thwart malicious activity, and continually strengthen both our classifiers and model,” the company wrote in its blog.

The company believes the culprits are mostly private companies or researchers seeking a competitive advantage. A spokesperson told NBC News that Google believes the attacks originated from around the world but declined to share additional details about the suspects.

Google believes hackers may soon try to attack other companies with these methods

The scope of attacks on Gemini indicates they are likely, or will soon be, common against smaller companies' custom AI tools as well, said John Hultquist, the chief analyst of Google's Threat Intelligence Group.

"We're going to be the canary in the coal mine for far more incidents," Hultquist said. However, he declined to name suspects. The company considers distillation to be intellectual property theft, it said.

Tech companies have spent billions developing their AI chatbots and large language models and consider their inner workings proprietary information.

Even though they have mechanisms to detect distillation attacks and block the perpetrators, large language models are inherently vulnerable to such attacks because they are accessible to anyone on the internet.

OpenAI, the company behind ChatGPT, accused its Chinese rival, DeepSeek, of conducting distillation attacks to improve its models last year.

Many of the attacks were crafted to tease out the algorithms that help Gemini "reason", or decide how to process information, Google said.

Hultquist said that as more companies design their own custom large language models trained on potentially sensitive data, they become vulnerable to similar attacks.

"Let's say your LLM has been trained on 100 years of secret thinking of the way you trade. Theoretically, you could distil some of that," he said.




GN Awards 2025: Vote for your favorite Gadgets