Dutch court to Elon Musk’s xAI: Stop Grok from creating ‘undressing’ AI photos or…

A Dutch court has issued a landmark ruling against Elon Musk’s artificial intelligence company, xAI, ordering it to immediately stop its chatbot, Grok, from generating non-consensual sexualised photos without their consent in the Netherlands. The preliminary injunction has been handed down by the Amsterdam Court and it specifically prohibits Grok from creating or distributing images that “undress” adults or children.

According to a report by news agency Reuters, the court’s decision comes with a massive financial penalty for non-compliance – if xAI fails to follow the order, it faces fines of 100,000 euros ($115,350) per day. Furthermore, the judge ordered xAI to stop offering Grok on the social media platform X (formerly Twitter) as long as it remains in breach of these rules.

What happened in courtroom

The case was brought by Offlimits, a Dutch nonprofit dedicated to fighting online sexual abuse. During a hearing earlier this month, the legal battle centered on whether the company is responsible for how people use its tools. In its defence, the xAI lawyers argued that the company cannot stop every single instance of abuse and should not be punished for the actions of “malicious users”. They pointed out that safeguards were tightened in January to restrict image generation to paid subscribers.

However, Offlimits director Robbert Hoving told the court that these safeguards were insufficient. In a March 9 courtroom demonstration, Hoving showed that Grok was still capable of “undressing” digital images of real people without their consent.

“The burden is on the company to make sure its tools are not used to make non-consensual sexual images,” Hoving stated following the ruling.

xAI ruling ‘one of the firsts’ in Europe

The report said that this ruling is one of the first times a European judge has held an AI company directly responsible for the output of its creative tools – setting a major precedent as regulators across the continent step up scrutiny under the EU’s Digital Services Act.

The decision arrives alongside several other major actions. First is that the European Commission opened a formal investigation into X in January over risks tied to Grok, including the spread of manipulated explicit images. On Thursday (March 26), the European Parliament backed a total ban on AI “nudifier” apps designed to create or manipulate sexually explicit content.