Glonce

San Francisco orders Meta to stop "allowing" AI child abuse ads

policyThe Glonce editorsWired🔥73Glonce rating73Impact72Novelty65Relevance75Surprise73Players90Credibility85
San Francisco orders Meta to stop "allowing" AI child abuse ads

San Francisco's city attorney, David Chiu, has sent Meta a four-page cease-and-desist letter demanding the company stop "allowing" paid ads that include AI-generated child sexual abuse content, and explain how those ads kept running on its platforms. The letter follows WIRED reporting, based on research from the Tech Transparency Project (TTP), that found Meta had run more than 350 such ads in recent months. The ads used AI to turn real still images of minors, including a member of a European royal family whom WIRED does not name, into short video clips depicting them performing sexual acts. Clicking the ads directed users to AI image- and video-generation apps, some of which could digitally undress people or create other nonconsensual intimate images.

In total, the ads reached more than 29,000 accounts in European Union countries and ran across Facebook, Instagram and Threads, also targeting accounts in the US, Australia and India. WIRED's first report on the ads, at the start of August, covered a batch of 53 of them; by September 8, when WIRED reported that the total had passed 350, more than 250 additional ads had run on Meta's platforms since that initial report.

Chiu's statement to WIRED says Meta "has failed to address a known issue of AI-generated child sexually explicit ads, while profiting from them," and that the scale and persistence of the problem show its efforts are "woefully inadequate." In the letter itself, sent to Meta's lawyers, Chiu writes that no company that claims to review and approve every ad before accepting payment for it should let its systems be used this way, and that the findings "do not appear to reflect an isolated failure that Meta corrected when first alerted to the problem." The letter demands Meta stop the ads, explain how they evaded its moderation systems, detail how it escalates cases to the National Center for Missing and Exploited Children (NCMEC), and describe how it handles the advertisers behind the content and repeat offenders. It sets a 28-day deadline for a response.

Meta disputes San Francisco's authority to act, saying there is "no indication" the ads were shown in the city and that this puts the matter "outside the city attorney's jurisdiction"; the company's ad library does not break down where in the US an ad ran. City Attorney's Office spokesperson Alex Barrett-Shorter rejects that argument, saying San Francisco consumers can reach the ads through Meta's own platforms regardless, calling Meta's response "astonishing" and saying the company "appears less concerned about whether they're guilty of exploiting children than whether San Francisco should say anything about it," and that Meta "has bigger problems to solve here than determining our jurisdiction."

Meta told WIRED that many of the flagged ads had already been taken down before they were reported and that all of them have since been removed for violating its policies; the company says the 300-plus ads researchers found had a combined spend of under $5,000, and most drew fewer than 200 impressions. A Meta spokesperson said the company does not tolerate nudify apps or child exploitation, whether real or AI-generated, and that its policies bar child and adult sexual exploitation and nudity outright, with every ad reviewed against those policies through a mix of automated and manual checks.

The letter says the City Attorney's Office has reviewed many of the ads TTP found. TTP's research describes Meta sometimes taking more than a week to act on abuse reports, during which the ads were seen by hundreds more people, and says some removed ads were not initially marked as containing child sexual abuse content. Katie Paul, TTP's director, says Meta is "not effectively" addressing the problem: since TTP's September 8 report, she says, new ads have run featuring some of the same children TTP had already flagged to Meta, and ads containing child sexual abuse content were still appearing as of September 9. Meta has previously sued a Hong Kong-based company linked to a nudifier platform and says it has removed hundreds of thousands of nudify-app ads in total, but researchers say they keep finding thousands of ads that push people to such apps, and WIRED itself found hundreds of "nudification" ads still running on Meta's platforms while reporting this story. Meta did not immediately respond to WIRED's question about the new ads Paul described.

Key facts

  • San Francisco city attorney David Chiu sent Meta a four-page cease-and-desist letter over more than 350 AI-generated ads that turned real photos of minors, including a member of a European royal family, into sexual video clips.
  • The ads reached more than 29,000 accounts in EU countries and ran on Facebook, Instagram and Threads, also targeting users in the US, Australia and India; WIRED's first report in early August covered just 53 of them.
  • Meta says there is "no indication" the ads ran in San Francisco and calls the matter outside the city attorney's jurisdiction; City Attorney's Office spokesperson Alex Barrett-Shorter calls that response "astonishing."
  • Meta says the flagged ads together cost under $5,000 and most drew fewer than 200 impressions, and that all have now been removed; Tech Transparency Project director Katie Paul says new ads featuring the same children kept appearing after her group's September 8 report.
  • The letter gives Meta 28 days to explain how the ads evaded moderation and how it escalates cases to the National Center for Missing and Exploited Children.

Why it matters

San Francisco's city attorney is demanding Meta explain how AI-generated child sexual abuse content kept running as paid ads on its platforms, and the letter treats the failure as systemic rather than a one-off: Chiu writes that the findings "do not appear to reflect an isolated failure that Meta corrected when first alerted to the problem." The case also shows how far ad-based distribution of "nudify" and AI image-generation apps has outrun Meta's own enforcement: the company has sued a Hong Kong-based company linked to one such app and says it has removed hundreds of thousands of nudify-app ads in total, yet researchers say they still keep finding thousands of similar ads, and WIRED found hundreds of "nudification" ads running during its own reporting.

Who it affects

The minors whose real photos were turned into sexual videos, including a member of a European royal family whom WIRED does not name, are the direct victims; the ads reached more than 29,000 accounts in EU countries and also targeted users in the US, Australia and India, running on Facebook, Instagram and Threads. Meta is the direct target of the city attorney's demand, along with its legal, child-safety and advertising staff, whom Chiu's office wants in "prompt discussions." The National Center for Missing and Exploited Children is named as the body Meta must detail how it reports such ads to.

How to use it

The letter is a demand, not a ruling: Meta has 28 days to explain how the ads evaded its automated and manual review, how it escalates cases to NCMEC, and how it handles the advertisers and repeat offenders behind the content. Anyone who spots similar ads can report them the way TTP did, to Meta and to NCMEC directly; Meta's ad library lets the public search ads currently and recently running on its platforms, though WIRED notes it does not show where in the US a given ad was shown, the gap Meta cites in disputing San Francisco's jurisdiction.

How solid is it

The story rests on a primary document, the four-page letter, which WIRED says it reviewed, plus named on-record sources on both sides: Chiu, City Attorney's Office spokesperson Alex Barrett-Shorter, an unnamed Meta spokesperson, and Katie Paul, director of the Tech Transparency Project, whose researchers found the ads. Meta's own figures on spend, impressions and removal timing are attributed to the company rather than independently verified; the jurisdiction dispute itself turns partly on Meta's ad library, which WIRED and TTP say does not break out where in the US an ad ran.

Risks and caveats

The article does not say exactly when the letter was sent, only that Chiu's office has already delivered it to Meta's lawyers, and it does not describe any penalty if Meta misses its 28-day deadline, only that a response is demanded. Meta disputes that San Francisco has jurisdiction at all. The article does not name the advertisers or accounts behind the ads, does not name or give the nationality of the royal family member depicted, and does not say whether Meta agreed to the "prompt discussions" the letter requests. Despite Meta's past enforcement against nudify apps, Katie Paul says new ads featuring the same children she had flagged kept appearing after TTP's September 8 report, and Meta did not immediately respond to WIRED's question about it.

“These ads are deeply disturbing and cause harm to real children. Meta claims to have zero tolerance for child exploitation, yet the scale and persistence of these issues suggest current efforts are woefully inadequate. This is unacceptable, and Meta must do better.”

— David Chiu, San Francisco city attorney