The online platform X has caused significant outrage after people utilized its AI chatbot Grok to alter images of females by taking off their attire.
Multiple cases have emerged where the tool has been used to remove clothing from female subjects to render them in bikinis non-consensually, as well as to place them in inappropriate scenarios.
Developer Reaction and Regulatory Position
XAI, the organization behind Grok, offered no comment to inquiries, except for an automatically-generated response stating "legacy media lies".
A government official stated that legislation is being proposed to ban nudification software. Under a upcoming law, anyone who provided such technology would "receive a jail term and substantial fines".
"The platform has the ability to stop these forms of misuse if they chose to," said a law professor, adding they "operate with a lack of consequences."
Regulator Position and Platform Functionality
The communications watchdog Ofcom said technology companies must "assess the risk" of people accessing illegal content on their networks, but did not confirm whether it was presently examining X or Grok in concerning artificial intelligence images.
Grok is a free artificial intelligence—with some premium features—which answers X users' queries when they reference it in a message.
It is often used to give commentary or additional background to other users' comments, but people on X are also can edit an uploaded photo through its artificial intelligence picture modification tool.
Background of Controversy and Policy
The system has been faulted for permitting users to generate pictures and footage with explicit content and sexualised material, and it was previously alleged to have been involved in generating graphic content of public figures.
"X has been enabling the production and sharing of these images for months without taking any steps and we have not seen any action by regulators," the expert added.
XAI's own acceptable use policy forbids "depicting images of persons in a sexually explicit manner".
In a comment, Ofcom said it was unlawful to "generate or distribute deepfakes or illegal content" and clarified this applied to explicit AI images created with AI.
It said services such as X were mandated to take "necessary actions" to "lessen the risk" of users encountering illegal material on their platforms and to delete it swiftly when they learn of it.
