The social media network X has sparked widespread backlash after individuals used its AI chatbot Grok to alter pictures of women by taking off their clothing.
Multiple cases have been seen where the system has been employed to digitally undress women to make them appear in bikinis without permission, as well as to insert them in inappropriate scenarios.
Developer Reaction and Legal Position
XAI, the company behind Grok, did not respond to questions, aside from an automatically-generated message stating "traditional news outlets lies".
A Home Office official stated that new laws is being brought forward to prohibit image stripping software. Under a upcoming law, anyone who provided such technology would "be subject to a jail term and significant fines".
"The platform could prevent these types of abuse if they chose to," said a law professor, adding they "seem to have impunity."
Regulator Position and Platform Capabilities
The communications watchdog Ofcom said platforms must "consider the risk" of people accessing illegal content on their networks, but stopped short of confirming whether it was at this time investigating X or Grok in concerning artificial intelligence pictures.
Grok is a no-cost chatbot—with some paid-for features—which responds to X users' queries when they tag it in a post.
It is frequently used to give commentary or additional background to other users' remarks, but people on X are also able to edit an posted photo through its artificial intelligence picture modification tool.
Background of Criticism and Policy
The system has been condemned for permitting users to generate images and clips with explicit content and sexually suggestive content, and it was in the past accused of generating inappropriate material of public figures.
"X has been allowing the generation and distribution of these images for a long time without taking any action and we have not seen any intervention by regulators," the expert added.
XAI's own terms of service forbids "depicting images of persons in a graphic way".
In a comment, Ofcom said it was illegal to "make or share deepfakes or illegal content" and clarified this included explicit AI images generated with AI.
It said services such as X were obligated to take "necessary measures" to "reduce the risk" of users coming across illegal material on their platforms and to remove it quickly when they become aware of it.
