Google has taken legal action against a Chinese cybercrime network, accusing the group of exploiting the company's Gemini artificial intelligence models to build sophisticated phishing software that defrauded consumers out of millions of dollars.
The lawsuit, filed Friday in the U.S. District Court for the Southern District of New York, targets an operation behind a phishing toolkit called “Outsider.” According to the complaint, the software uses pre-built templates to mimic legitimate financial institutions, government agencies, and retailers, tricking victims into surrendering personal and financial data.
Google alleges the network encouraged hackers to use mainstream AI tools like Gemini to generate custom code for fraudulent websites. The company described a method where seemingly innocent programming prompts are fed into the AI, producing code that can replicate virtually any legitimate site in minutes. “On their own, these prompts appear to be innocent requests for programming assistance,” the lawsuit states, but the resulting code is used to create convincing duplicates.
The Outsider software includes more than 290 templates impersonating entities such as brokerages, phone carriers, and government services. Victims are lured via text messages about account problems or rewards, then directed to fake websites where they input sensitive information. Google said the scheme has already caused millions in losses.
The company also warned that the 290 identified templates are just the beginning. The lawsuit describes the potential for “limitless” phishing sites, noting that Google discovered over one million URLs originating from the Outsider software. The group even provided step-by-step instructions on how to weaponize AI-generated code, effectively lowering the barrier for other criminals.
Google emphasized that AI is accelerating the pace of cyberattacks. “We hope this suit will disrupt the criminal enterprise and protect the online ecosystem,” the company said in a statement. The case comes amid broader concerns about AI's dual-use nature in cybersecurity. Earlier this year, AI firm Anthropic released its “Mythos” cybersecurity model, which can detect vulnerabilities but also poses risks if misused by bad actors targeting critical infrastructure.
The Hill was unable to reach an attorney for the 25 “Doe” defendants named in the suit. The case highlights a growing trend of cybercriminals leveraging generative AI to scale operations, prompting tech companies and regulators to scramble for countermeasures. As AI tools become more accessible, the line between legitimate use and malicious exploitation continues to blur.
