🔗 Share this article

Elon Musk's online platform X has caused major backlash after people utilized its AI chatbot Grok to manipulate pictures of females by taking off their garments.

Multiple cases have surfaced where the technology has been used to virtually strip women to render them in swimwear without permission, as well as to put them in inappropriate situations.

Developer Response and Legal Stance

XAI, the company responsible for Grok, did not respond to inquiries, other than an automatically-generated response stating "traditional news outlets lies".

A Home Office official stated that new laws is being proposed to ban nudification software. Under a upcoming criminal offence, anyone who distributed such tools would "be subject to a jail term and significant fines".

"X or Grok could prevent these types of misuse if they chose to," said a law professor, adding they "operate with impunity."

Regulator Stance and Tool Capabilities

The communications watchdog Ofcom said tech firms must "assess the potential" of people accessing illegal content on their platforms, but stopped short of confirming whether it was presently investigating X or Grok in regarding AI-generated pictures.

Grok is a no-cost chatbot—with some paid-for functions—which reacts to X users' queries when they mention it in a post.

It is frequently used to provide commentary or more background to other posters' remarks, but people on X are also able to modify an uploaded image through its AI image editing function.

Background of Controversy and Policy

The tool has been faulted for allowing users to produce images and footage with nudity and sexually suggestive content, and it was earlier alleged to have been involved in creating graphic content of celebrities.

"X has been enabling the production and sharing of these pictures for a long time without taking any steps and we have not seen any challenge by oversight bodies," Professor McGlynn added.

XAI's own terms of service prohibits "depicting likenesses of people in a sexually explicit manner".

In a statement, Ofcom said it was against the law to "create or distribute non-consensual intimate images or illegal content" and confirmed this included sexual deepfakes made with AI.

It said services such as X were required to take "appropriate actions" to "minimize the danger" of users coming across illegal material on their services and to take down it swiftly when they are notified of it.