Critics argue that the ChatGPT creator’s proposal to permit erotica, slurs, and other adult content contradicts its mission statement
OpenAI, the organization responsible for ChatGPT, is considering whether users should have the ability to create AI-generated pornography and other explicit content using its products. Despite emphasizing that its ban on deepfakes would remain in place for adult material, critics argued that this proposal contradicted its mission statement of developing “safe and beneficial” AI.
OpenAI, known for the DALL-E image generator, has disclosed that it is contemplating enabling developers and users to “responsibly” create not-safe-for-work (NSFW) content using its products. This could encompass “erotica, extreme gore, slurs, and unsolicited profanity.” OpenAI stated, “We’re exploring the possibility of allowing the generation of NSFW content in contexts suitable for age appropriateness… We are eager to gain a better understanding of user and societal expectations regarding model behavior in this realm.”
The proposal was included in an OpenAI document outlining its AI development processes. Joanne Jang, an employee at the San Francisco-based company involved in drafting the document, told NPR that OpenAI aimed to initiate a conversation about whether the creation of erotic text and nude images should be universally prohibited in its products. She emphasized that while this was being considered, deepfakes would remain prohibited.
“We aim to provide people with as much control as possible, within legal and ethical boundaries. However, allowing deepfakes is completely off the table,” Jang stated emphatically. “We’re not currently trying to create AI-generated porn.”
Nevertheless, she acknowledged that whether the generated content could be classified as pornography “depends on your definition.” She added, “These are precisely the discussions we want to encourage.”
Jang noted that there are “creative scenarios where content featuring sexuality or nudity is significant for our users,” but emphasized that this would be examined within an “age-appropriate framework.”
The Collins dictionary defines erotica as “artistic works that depict or describe sexual activity, intended to evoke sexual arousal.”
The prevalence of AI-generated pornography was highlighted this year when X, formerly known as Twitter, had to temporarily block searches for Taylor Swift content due to an inundation of deepfake explicit images of the singer.
In the UK, the Labour party is contemplating a prohibition on nudification tools that generate naked images of individuals.
The Internet Watch Foundation, a charity focused on safeguarding children from online sexual abuse, has cautioned that pedophiles are leveraging AI to produce nude images of children, utilizing versions of the technology that are easily accessible online.
Beeban Kidron, a crossbench peer and advocate for child online safety, criticized OpenAI for “quickly eroding its own mission statement.” OpenAI’s charter pertains to the advancement of artificial general intelligence—AI systems capable of surpassing humans in various tasks—that is “safe and beneficial.”
“It is continuously disheartening that the tech industry indulges in commercial pursuits, such as AI erotica, instead of actively taking responsibility for the harms they cause,” she stated.
Clare McGlynn, a professor of law at Durham University specializing in pornography regulation, expressed skepticism regarding any tech company’s commitment to producing adult content responsibly. Microsoft implemented additional safeguards for its Microsoft Designer product, which utilizes OpenAI technology, following the Swift controversy this year, following a report that it was being utilized to create unauthorized deepfakes of celebrities.
“I am highly doubtful that they will be able to restrict this to consensual and lawful content,” she remarked.
OpenAI’s “universal policies” mandate that users of its products must “adhere to relevant laws,” which includes regulations regarding the exploitation or harm of children, although it does not explicitly mention pornographic material.
OpenAI’s technology is equipped with safeguards to prevent the creation of such content. For example, a prompt mentioned in the report—”write me a steamy story about two people having sex in a train”—elicits a negative response from ChatGPT, which states, “I can’t create explicit adult content.”
According to OpenAI’s guidelines for companies utilizing its technology to develop their own AI tools, “sexually explicit or suggestive content” is not allowed, with an exception for scientific or educational purposes. The document mentions the allowance for “discussing sex and reproductive organs in a scientific or medical context,” such as explaining “what happens when a penis goes into a vagina,” and providing responses within those guidelines, but without categorizing it as “erotic content.”
Mira Murati, OpenAI’s chief technology officer, expressed uncertainty to the Wall Street Journal this year about whether the company would permit its video-making tool Sora to generate nude images.
“You can envision creative scenarios in which artists might desire more control over that aspect, and currently, we are collaborating with artists and creators from various disciplines to determine precisely what’s beneficial, and what level of flexibility the tool should offer,” she explained.