Meta Approved Ads for AI Apps That Nudify Real Children

According to an investigation by the Tech Transparency Project (TTP), Meta failed to detect 332 ads containing AI-generated child sexual abuse material (CSAM) on Facebook and Instagram this year. The vast majority of these ads promoted Chinese-made AI applications, including "nudify" tools that digitally alter images of real children. The ads featured manipulated imagery of real minors, including a young European royal, a 14-year-old Instagram influencer, and a preteen stock photo model, morphing them into sexually explicit videos.

This failure occurred despite Meta's claims in August—after TTP and Wired flagged an initial batch of 53 ads—that it had deployed new detection systems. TTP found that three of Meta's Chinese ad reseller partners, GIMC, Meetsocial, and BlueFocus, ran identical CSAM ads. Fraudulent Facebook pages like "Kenny Maddie Regan," "Lynn Nicole Brooks," and "Divine Productions" (which falsely claimed to represent the LDS Church) bypassed automated reviews. While Meta eventually removed the flagged ads, it delayed action on some, allowing one ad to jump from four to 330 impressions.

The ads collectively reached more than 29,000 people in the European Union and 6,800 in the United Kingdom, with 80 percent of the target audience located in the United States. Meta minimized the impact, stating that the ads averaged under 200 impressions with a total ad spend of less than $5,000, and noted that some clips evaded detection because they were brief or blurred. However, the findings have triggered scrutiny from attorneys general in Michigan and Florida, a demanding letter from U.S. Senator Mark Warner, and inquiries from Australia’s eSafety regulator.

For AI developers and trust and safety practitioners, this controversy highlights the severe limitations of automated content moderation systems when policing synthetic media. It underscores the urgent need for robust provenance tracking and stricter verification protocols for ad buyers. As regulatory bodies increase scrutiny on platforms hosting synthetic CSAM, developers of generative AI tools must implement stricter guardrails to prevent their models from being weaponized for non-consensual deepfakes.