Deepfakes and Children: Protecting the Most Vulnerable in the AI Era

Years ago, when I first saw a deepfake video, I was fascinated by the technology. Today, with the advent of generative artificial intelligence, I feel more concerned. What was once a niche technical curiosity has become a powerful tool in the hands of cybercriminals who threaten the most vulnerable – our children. The rapid advancement of AI dramatically simplifies the creation of hyper-realistic deepfake videos and photos, including child sexual abuse material (CSAM), and existing defense mechanisms often struggle to keep pace.

The Exponential Rise of AI-Generated Child Abuse

The increase in AI-generated CSAM is alarming. 2025 reports from organizations like the National Center for Missing and Exploited Children (NCMEC) and the Internet Watch Foundation (IWF) indicate an exponential growth in this material. For example, the IWF recorded a more than 260-fold increase in AI-generated child sexual abuse videos in 2025 compared to the previous year, with 65% of these videos falling into the most extreme Category A. A study by UNICEF, ECPAT, and Interpol across 11 countries found that at least 1.2 million children reported in the past year that their images had been manipulated into sexually explicit deepfakes. In some countries, this amounts to one in twenty-five children, roughly equivalent to one child in every typical classroom.

What once required specialized skills and computational power is now accessible through free mobile applications. These “nudification” tools, which remove clothing from photos, had been downloaded over 705 million times globally by 2025.

Global Legislative and Technological Response

The global community is striving to respond to this threat. The European Union will ban AI systems that generate CSAM or “nudification” content by December 2026. The EU AI Act, which came into force on August 1, 2024, emphasizes the protection of minors and requires transparency for AI-generated content. In January 2026, the European Parliament called for stronger and faster enforcement of EU laws, specifically the DSA and AI Act, to prevent the creation of non-consensual sexual deepfakes.

In the United States, the Take It Down Act, passed in May 2025, criminalizes the publication of non-consensual sexual deepfakes and requires platforms to remove such content within 48 hours. As of July 2026, New York Governor Kathy Hochul announced the final rules for the SAFE for Kids Act, which, among other things, prohibits AI-generated child sexual abuse material and requires social media platforms to restrict addictive features for children under 18 without parental consent.

Technology companies like Meta claim to be investing in AI for proactive detection of CSAM and abusive content. In the first half of 2026, they removed millions of accounts and pieces of content. However, the effectiveness of these systems is under debate. For instance, in February 2026, US Internet Crimes Against Children (ICAC) investigators reported that Meta's AI software generates a large volume of “junk” reports, draining resources and hindering investigations. In March 2026, Meta was ordered to pay $375 million in New Mexico for deceptive claims about platform safety and failure to protect children.

What This Means for Parents and Schools

For parents and schools, it is crucial to recognize that the threat of AI-generated deepfakes is real and constantly evolving. The fact that content is not “real” does not mean it doesn't cause real harm. Children whose images are exploited experience fear, shame, and a loss of trust, leading to long-term emotional impacts.

Checklist for Parents and Educators:

1. Educate and Communicate: Talk to children about the risks of deepfakes and how to identify manipulated content. Explain that even generated content is abuse.
2. Set Clear Rules: Use parental controls and set time limits for device usage. Consider restricting access to generative AI tools that lack robust safety features.
3. Report and Respond: If you encounter suspicious content, report it immediately to relevant platforms and law enforcement agencies (e.g., NCMEC CyberTipline).
4. Promote Digital Literacy: Teach children to critically evaluate content they see online and verify sources. Schools should integrate digital literacy and deepfake threat prevention into their curricula.
5. Trust Your Judgment: If something feels “off” or too good to be true, it probably is. In the digital world, caution is essential.

I see many parents and educators feeling helpless in the face of rapidly evolving technology. But this is precisely where our strength lies – in judgment and the ability to distinguish the essential from the trendy. We cannot rely solely on legislation, which always “catches up” with reality. We must be proactive in prevention and education.

Consequence of Inaction and Next Steps

If we, as a society, parents, and schools, do not actively resist the spread of AI-generated abuse, we risk normalizing this content and deepening the trauma of victims. We lose control over the digital environment in which our children grow up. The consequences can be devastating: from psychological problems and social isolation of victims to the erosion of trust in digital media as a whole.

It's not enough just to follow the news. It's time to act. For schools and organizations looking to systematically enhance digital safety and educate on AI risks, one-o-one.cz offers comprehensive programs and workshops through our onesafety project. We help create strategies to combat these new threats and effectively communicate with both children and parents. The first step can be an audit of existing safety measures and educational programs in your institution. This should be followed by the implementation of clear rules and regular training for all stakeholders.

FAQ

Q: How can I recognize an AI-generated deepfake? A: Deepfakes are becoming increasingly sophisticated, but often have subtle anomalies: unnatural eye movements, distorted edges, inconsistent lighting, strange skin textures, or lip synchronization. Reputable sources recommend verifying information and being skeptical of content that seems too perfect or too shocking. It's always good to seek multiple sources and context.

Q: Are all AI image generation tools dangerous? A: No, many AI tools are designed for creative and productive purposes. The dangerous ones are those that lack sufficient safety features or are explicitly misused to create harmful content. The key is to use tools from reputable developers and be aware of their potential for misuse.

Q: What should I do if my child becomes a victim of deepfake abuse? A: Seek support immediately. Contact local police or organizations specializing in child online safety, such as NCMEC (National Center for Missing and Exploited Children) or IWF (Internet Watch Foundation). Use tools like Take It Down, which helps remove abusive images from the internet. Providing psychological support for the child is also crucial.