British law enforcement has ramped up pressure on X, the Elon Musk-owned social media platform, to address the rampant distribution of AI-generated sexual images of women and children on the site.
Yesterday, the U.K. watchdog Ofcom announced that it has launched an investigation into the platform as their AI chatbot Grok continues to produce non-consensual deepfake images.
The investigation will examine whether the site complies with the U.K.’s Online Safety Act. Passed in 2023, the act was created to regulate online content, imposing duties on platforms to eliminate harmful content. The law was designed to protect children, but also provide protection for adult users.
While a timeline for the investigation’s verdict was not provided, Ofcom emphasized that they are prioritizing this case. If X is found guilty of breaking the act, it could be fined up to 18 million pounds or 10% of its global revenue, whichever is more.
If they do not take proper action, a regulator could seek a court order to prevent partnerships with X.
The British government, including Technology Secretary Liz Kendall, has expressed support for banning the platform in the country if significant harm is found.
“The content which has circulated on X is vile. It’s not just an affront to decent society, it is illegal,” said Kendall, per BBC. “Let me be crystal clear – under the Online Safety Act, sharing intimate images of people without their consent, or threatening to share them, including pictures of people in their underwear, is a criminal offence for individuals and for platforms.”
“This means individuals are committing a criminal offence if they create or seek to create such content, including on X, and anyone who does this should expect to face the full extent of the law,” she added.
Calls for X to address the usage of Grok have been international. In Indonesia, regulators blocked Grok to “protect women, children and the broader public from the risks of fake pornographic content generated using artificial intelligence technology.”
Malaysian regulators quickly followed suit, announcing a temporary ban on Sunday.
The Ministry of Electronics and Information Technology in India also reportedly reached out to X last week, warning them that they may be violating the law.
Although Musk and X have claimed they would prohibit users from making “illegal content,” Grok is estimated to create thousands of sexualized AI-generated images of real people each hour after being prompted by users.
On Friday, X reportedly made the Grok reply box, the AI-system driving the “trend,” a premium feature available only to paying subscribers. Users, however, can still create deepfakes through the Grok app.
The posting of the images has slowed down in numbers, but according to Sen. Ron Wyden, D-Ore, the decision to make Grok premium is to make users “pay for the privilege of producing horrific images on the X app.”
As of now, U.S. regulators have not indicated that they will take action as well. The Trump administration has previously defended the platform, calling the European Commission’s fine an “attack on American free speech.” Announced in December, the $140 million fine was related to X breaking online content rules.
