The online platform X has provoked significant backlash after people utilized its artificial intelligence assistant Grok to manipulate pictures of females by stripping their attire.
Several instances have emerged where the system has been employed to remove clothing from female subjects to make them appear in bikinis non-consensually, as well as to place them in sexually explicit situations.
Company Reaction and Legal Position
XAI, the company that created Grok, remained silent to questions, aside from an auto-reply message stating "legacy media lies".
A government representative stated that new laws is being introduced to prohibit image stripping tools. Under a new law, anyone who provided such technology would "face a prison sentence and heavy penalties".
"The platform is capable of halting these types of abuse if they chose to," said a law professor, adding they "seem to have impunity."
Regulator Position and Tool Functionality
The regulator Ofcom said tech firms must "consider the potential" of people viewing illegal content on their platforms, but stopped short of confirming whether it was at this time looking into X or Grok in regarding AI-generated images.
Grok is a no-cost chatbot—with some additional features—which responds to X users' requests when they reference it in a tweet.
It is commonly used to offer reactions or additional context to other users' comments, but people on X are also capable of edit an posted picture through its artificial intelligence picture modification feature.
Background of Controversy and Rules
The chatbot has been criticised for enabling users to produce photos and clips with nudity and sexually suggestive material, and it was previously alleged to have been involved in making inappropriate content of public figures.
"X has been allowing the production and sharing of these pictures for months without taking any steps and we have not seen any intervention by regulators," Professor McGlynn added.
XAI's own acceptable use policy forbids "portraying likenesses of people in a graphic fashion".
In a statement, Ofcom said it was against the law to "make or distribute deepfakes or child sexual abuse material" and confirmed this included explicit AI images generated with AI.
It said platforms such as X were required to take "appropriate steps" to "minimize the risk" of users coming across unlawful content on their services and to remove it quickly when they become aware of it.
