With a major training dataset found to include child sexual abuse material, are bans on creating such imagery feasible?
Child abusers are using AI-generated “deepfakes” to blackmail their victims into recording their own abuse, initiating a cycle of sextortion that can persist for years.
In the UK, producing simulated child abuse imagery is illegal, and both the Labour and Conservative parties agree on the need to ban all explicit AI-generated images of actual individuals.
However, there is minimal international consensus on how to regulate this technology. Furthermore, despite government efforts, generating additional images remains as simple as pressing a button, as explicit imagery is deeply embedded in the core of AI image generation.
In December, Stanford University researchers made a troubling finding: within one of the largest training datasets for AI image generators, there were potentially hundreds, if not thousands, of instances of child sexual abuse material (CSAM).
There could be numerous additional instances. The dataset in question, Laion (Large-scale AI Open Network), comprises roughly 5 billion images. At half a second per picture, it would conceivably take a lifetime to review them all—assuming one is young, fit, healthy, and can forego sleep. Hence, researchers had to automatically scan the database, comparing questionable images with records maintained by law enforcement. They also trained a system to identify similar photos before promptly forwarding them to the authorities for examination.
In reaction, the creators of Laion removed the dataset from download. They clarified that they had never actually distributed the problematic images since the dataset essentially comprised a lengthy list of URLs pointing to images hosted elsewhere on the internet. In fact, by the time the Stanford researchers conducted their study, nearly a third of the links were inactive, making it difficult to determine how many of them had previously contained CSAM.
However, the repercussions are already apparent. AI systems trained on Laion-5B, the dataset in question, are widely employed globally, with the illicit training data firmly embedded in their neural networks. As a result, AI image generators are capable of producing explicit content, including that of adults and children, because they have been exposed to it.
Laion is probably not the only case. This dataset was created as an “open-source” resource, assembled by volunteers and made available to the public for independent AI research. Consequently, it was extensively utilized to train open-source models, such as Stable Diffusion, an image generator that, as one of the pivotal releases of 2022, played a role in catalyzing the artificial intelligence revolution. However, this also meant that the entire dataset was openly accessible for anyone to explore and scrutinize.
Unlike Laion, the same level of transparency is not provided by its competitors. For example, OpenAI offers only a “model card” for its Dall-E 3 system, indicating that its images were “drawn from a combination of publicly available and licensed sources.”
“We have made an effort to filter the most explicit content from the training data for the Dall-E 3 model,” the company states. However, the effectiveness of these efforts must be accepted on trust.
Ensuring a dataset is entirely free of inappropriate content is extremely challenging, which is why organizations like OpenAI advocate for restrictions in the first place. Unlike Stable Diffusion, downloading Dall-E 3 to operate on personal hardware is not feasible. Instead, all requests must go through OpenAI’s systems. For the majority of users, ChatGPT serves as an intermediary layer, modifying requests on the fly to provide the image generator with more detailed information.
As a result, OpenAI and competitors like Google, who employ a similar strategy, have additional mechanisms to ensure their generators remain free of inappropriate content. These mechanisms include restricting the types of requests that can be submitted and filtering generated images before they reach the end user. AI safety experts argue that this approach is more robust than relying solely on a system that has been trained to avoid creating such images.
Even for “foundation models,” which are the most potent and least restricted outcomes of the AI revolution, the value of a completely clean set of training data is uncertain. For example, an AI model that has never been exposed to explicit imagery may struggle to recognize it in real-world scenarios or to follow instructions on how to report it to authorities.
“We must preserve room for open-source AI development,” stated Kirsty Innes, the director of tech policy at Labour Together. “This realm could hold the most effective tools for addressing future issues.”
In the near future, the proposed bans primarily target specialized tools. According to a policy paper co-authored by Innes, the suggested approach involves addressing only the creators and hosts of single-purpose “nudification” tools. However, in the long run, combating explicit AI imagery will encounter similar challenges to other issues in this field: how do you control a system whose workings are not completely understood?