Grok’s paywall can't hide its deepfake problem

Welcome to The Download, where we break down the day’s biggest tech news into bite-sized, context-filled pieces.

TL;DR: Elon Musk’s AI chatbot Grok now claims its image-generation tools are only available to paying subscribers. The move follows an explosion of nonconsensual sexual imagery on X, which has drawn threats of regulatory action in the UK and EU. But despite the new paywall, there are still easy workarounds that let any X user generate images.

What happened: A December update to Grok’s image-creation feature resulted in a tsunami of sexual and violent images generated by X users. These nonconsensual and explicit images were predominantly of women, but also included children. Seemingly everyone warned that the tool, which is embedded in X, lacked any meaningful guardrails, making it alarmingly easy to create explicit deepfakes at scale.

Governments around the world have raised concerns, and some US lawmakers are pointing to Grok as proof of why stronger protections against deepfake abuse are needed. Yesterday, the European Commission ordered X to preserve documents related to Grok through 2026, as it investigates whether the company is complying with EU law.

Now: Grok has begun telling users who request image generation that it is “currently limited to paying subscribers,” prompting them to sign up. But the restriction appears half-hearted (at best). The Verge found simple workarounds on both desktop and mobile—such as selecting an image and clicking “Edit image.”

What comes next: Grok’s partial retreat shows that even extremely reluctant tech companies will respond when regulators get serious. If scrutiny continues, X could face major fines or even operational limits in key markets. More importantly, this episode could (emphasis on could) accelerate new regulation of AI image-generation tools, especially those involving people’s bodies or likenesses. —SM