Back to Blog

Criminals Used Google's Own AI to Build 1.6 Million Phishing Sites. Google Just Sued Them.

Last Thursday, Google filed a lawsuit against a China-based cybercrime network called Outsider Enterprise, accusing the group of using Google’s own Gemini AI to generate the HTML behind over a million fake websites.[1] Between November 2025 and April 2026, Google identified more than 1.59 million fraudulent URLs linked to the group. In just two weeks of May 2026, the gang blasted 2.5 million phishing text messages to American phones, triggering 55,000 spam complaints from Android users alone—more than two per minute, around the clock.[2]

That’s not a small operation. That’s an assembly line.

What They Actually Built

Outsider Enterprise’s model was to use Gemini to write the HTML code for convincing replicas of trusted brands: Google, YouTube, the US Postal Service, and the toll payment platform E-ZPass. The pages were polished enough to pass a quick look. Then the group packaged them into ready-made “phishing kits” and sold those kits to other criminals through Telegram.[3]

Think of it like a franchise. The people running Outsider Enterprise weren’t necessarily the ones sending your employees fake delivery notifications. They were the suppliers. The buyers were a whole separate set of operators who purchased the kit and ran their own campaigns with it. Google’s lawsuit targets the top of that chain. It doesn’t touch the buyers who already paid and walked away with the tools.

This is also the first time a major tech company has sued a group for weaponizing its own AI product against users.[1] That’s significant. But it doesn’t make the kits disappear.

Why AI Changes the Math on Phishing

For years, spotting phishing emails was something you could actually teach. Look for bad grammar. Check the sender domain. Hover over links before clicking. That still matters. But it matters less than it used to.

LLM-generated phishing emails now achieve a 54% click-through rate, compared to roughly 12% for human-written phishing.[4] That gap is enormous. And 46% of small and mid-sized businesses reported encountering AI-generated phishing in the past twelve months.[5] The fakes have gotten good enough that the old “just use common sense” advice has real limits.

The reason is pretty simple: AI doesn’t get tired, doesn’t make typos, and can produce a perfect-looking USPS delivery failure notice in seconds. The volume and quality ceiling that used to cap phishing campaigns just doesn’t exist anymore.

This Isn’t Abstract for Small Businesses

I’ve talked to small business owners who think phishing is mainly a big-company problem because big companies have more valuable data. That logic is backwards. Big companies also have dedicated security teams, advanced email filtering, and response playbooks. Small businesses often have none of that, which is exactly why attackers target them.

The phishing kits sold on Telegram aren’t targeted at Fortune 500 companies. They’re designed to be fired off at scale. Whoever ends up on the receiving end of a campaign doesn’t matter to the buyer. What matters is volume. If your employees get a convincing fake that looks like an E-ZPass tolling notice or a USPS package failure, the kit worked on them just as well as it would have worked on anyone else.

And if an employee enters credentials on one of these pages, the attacker often has access to your business email or your company’s accounts within minutes.

What Actually Helps

Awareness training is still worth doing, but your team needs updated examples. Showing people decade-old phishing screenshots with obvious typos doesn’t prepare them for what’s landing in inboxes today. The training should reflect how good the fakes actually look now.

Beyond training, a few technical controls matter a lot here. Multi-factor authentication on business email and any externally accessible system is non-negotiable at this point. Even if an employee clicks the link and enters a password on a fake site, MFA means the attacker can’t actually use what they collected without a second factor they don’t have.

Email filtering that uses AI-based detection catches a lot of this too. The older rule-based filters are looking for known bad links and sender reputation. The newer tools look at the structure and intent of the message itself, which is more likely to flag a well-crafted phishing attempt before it reaches your team.

DNS filtering that blocks newly registered domains is also worth enabling. The fake sites in campaigns like this one are usually registered close to when the campaign runs. Blocking domains less than 30 days old by default kills a significant chunk of the infrastructure these operations use.

None of this is complicated to set up. It just has to actually be done.