Reports of AI-generated child sexual abuse material have surged more than 8,400% over the past two years. According to a report from Edinburgh-based Childlight - the Global Child Safety Institute, reports of this type of content rose from roughly 4,700 in 2023 to more than 400,000 in 2025. Researchers say the problem must now be treated by governments as a preventable public health crisis.
What happened?
The report states that deepfake images generated by tools such as 'nudify' apps are at the center of this surge. These apps can take an ordinary photo, upload it, and turn it into a fake nude or sexualized image within seconds. Childlight stresses that the true scale may be even larger than official reports suggest, since most platforms and parents still don't know how to report this kind of content.
Why can't detection systems catch AI-generated content?
The infrastructure for combating child abuse material has operated on the same logic for decades: tag known illegal files with a digital fingerprint (hash) to block their re-sharing. Microsoft's PhotoDNA and the NCMEC's CyberTipline database in the US are the cornerstones of this model. The logic is simple — once an image is detected and entered into the database, every new share of that same file is automatically caught. Generative AI breaks this logic: every generation produces a unique file, meaning each image is 'new' and unknown to the system. Hash matching becomes useless against infinitely varied generated content.
This also represents a shift in quality as much as quantity. Previously, producing this kind of material required either an actual record of abuse or labor-intensive manual editing with tools like Photoshop — both of which naturally limited production. Nudify apps have removed that barrier: an ordinary photo of a child shared on social media can be turned into abuse material within seconds, with no technical knowledge required. Once the cost of production dropped to nearly zero, a crime that used to be rare became scalable. The explosion in the numbers is therefore no surprise; the real question is why oversight infrastructure still isn't ready for this scale.
Who is affected, and how?
| Feature | Traditional abuse material | AI-generated material |
|---|---|---|
| Required for production | Actual record of abuse or manual editing | A single photo + nudify app |
| Production speed | Days, weeks | Seconds |
| Detection method | Hash matching (PhotoDNA, etc.) | Largely ineffective — every file is unique |
| Victim profile | Child directly subjected to abuse | Any child whose photo can be found online |
The table reveals the real breaking point: the condition for becoming a victim is no longer being directly abused, but simply having a photo that can be found online. This expands the pool of potential victims to every child who uses social media. App stores and payment providers also face a new area of responsibility: most nudify apps try to dodge liability by banning abusive use in their terms of service, even as their marketing language often signals the opposite.
What's next?
Frameworks like the European Union's AI regulation and the UK's Online Safety Act largely target content hosted on platforms; they aren't focused on blocking generation at the tool level. This gap is likely to become a priority for lawmakers in the coming period — regulations that directly ban nudify apps or require app stores to remove them could be on the table. On the technical side, classifiers need to move beyond hash matching to directly analyze image content, an approach that will raise the risk of false positives and fuel privacy debates. Another point worth watching is how far major AI companies will go in restricting open-source image models, since most nudify apps are built on open models rather than closed ones.
The 8,400% figure alone can be misleading, since it's calculated from a low 2023 baseline. But the absolute number — over 400,000 — shows this isn't a statistical anomaly. What's happening here differs from the exaggerated risk rhetoric common in AI debates: this is a measurable, concrete harm, paired with inadequate tools in the hands of the institutions meant to prevent it. Treating this as child protection policy, rather than squeezing it under the umbrella of AI ethics, is the right frame for now.
Yorumlar (0)
Henüz yorum yok. İlk yorumu sen yaz.
Yorum yapmak için Sinyal'i indir
Yorumlar Sinyal hesabıyla yapılır. Mobil uygulamada giriş yap, yorum bırak.