xAI sues Minnesota to block its nudification law
xAI has sued Minnesota Attorney General Keith Ellison to block a state law that targets "nudification" technology, filing the suit just days before the statute was due to take effect on August 1. The law, HF 1606, was signed by Governor Tim Walz in May and prohibits site owners from letting people "access, download, or use" software that nudifies images of real people, though it carves out tools that still require real technical skill to misuse, such as Photoshop. It creates a civil cause of action for private individuals and lets the state attorney general seek penalties of up to $500,000 per violation.
xAI's complaint argues the law violates the First Amendment because it is overbroad: it can impose liability even for images generated with consent, images of adults that are merely suggestive, or images with humorous or artistic intent, such as a widely shared deepfake of Chris Christie and JB Pritzker sumo wrestling. The company also contends that existing Minnesota and federal law, including the TAKE IT DOWN Act, already bars nonconsensual distribution of nude deepfakes and does so in a narrower, more constitutional way. It says it has deployed "near-perfect, state-of-the-art technical controls" to prevent users from generating nude images, and argues that strict liability is unfair when users manage to evade those controls in violation of Grok's terms of service. As evidence it takes violations seriously, xAI points to a separate lawsuit it filed against one of its own users for allegedly generating CSAM with Grok, a suit filed more than two months after Minnesota's law passed.
The filing follows a crisis in January, when Grok's image tools flooded the internet with sexually explicit deepfakes, including images of minors, for days before xAI intervened; at one point Elon Musk posted that "Grok now hitting #1 on the App Store in one country after another!" while the images were still circulating. A report from the Center for Countering Digital Hate found that over an 11-day period from December 29 to January 8, Grok generated about 3 million sexualized images, roughly 23,000 of them depicting children, a rate the group called "one sexualized image of a child every 41 seconds." The episode triggered investigations by the EU and the UK and criticism from officials in France, India, and Malaysia, while the US federal government stayed largely quiet; states including California and Minnesota moved to legislate.
Not all of the images Grok produced are necessarily illegal under existing law; a suggestive deepfake of a minor in a bikini sits in a legal gray area, though one expert previously told The Verge that other images Grok generated, including some showing minors' faces covered in semen, likely did violate existing law. Whether Minnesota's law survives xAI's challenge is now up to the courts.
Key facts
- xAI sued Minnesota AG Keith Ellison just days before HF 1606, the state's "nudification" law signed by Gov. Tim Walz in May, was set to take effect on August 1
- The law allows penalties up to $500,000 per violation plus a private civil right of action; xAI calls it unconstitutionally overbroad under the First Amendment
- The Center for Countering Digital Hate found Grok generated about 3 million sexualized images over an 11-day span from December 29 to January 8, including roughly 23,000 depicting children
- xAI argues existing laws, including the federal TAKE IT DOWN Act, already ban nonconsensual deepfakes in a narrower, constitutional way, unlike Minnesota's statute
- xAI has separately sued one of its own users for allegedly using Grok to generate CSAM, a suit filed more than two months after the Minnesota law passed
Why it matters
Minnesota's HF 1606 is billed as the first law of its kind targeting "nudification" software, and xAI's challenge is the first major test of whether such statutes survive First Amendment scrutiny. The case follows directly from Grok's January deepfake incident, so the outcome will shape how far US states can go in regulating AI image-generation tools without a federal framework in place.
Who it affects
Minnesota residents who could sue under the law's private right of action, Attorney General Keith Ellison and Governor Tim Walz's administration, xAI and Grok Imagine users in the state, and other companies offering similar image-editing tools who will watch the ruling as a precedent. Other states considering comparable statutes, including California, have a direct stake in the outcome too.
How to use it
The law was due to take effect August 1, and xAI's suit seeks to block it before then; until a court rules, xAI says it has "no practical choice but to restrict Grok Imagine's image-editing features in various ways" for Minnesota users to avoid exposure to the statute's penalties. Companies running similar nudification-adjacent tools in Minnesota face the same choice between restricting features or risking liability while the case is pending.
How solid is it
The account draws on xAI's actual court filing, quoted directly, and on a published Center for Countering Digital Hate report with a specific methodology and time window. The regulatory backdrop, including EU and UK investigations, is corroborated by multiple governments' public statements, giving the piece a solid factual foundation even though the framing is openly critical of xAI.
Risks and caveats
This is an active lawsuit; the claims in xAI's complaint are the company's own legal arguments, not settled findings, and a court has not yet ruled on the law's constitutionality. Not every image Grok generated is clearly illegal under current law, the bikini-deepfake example cited being a case in point, so the scale of Grok's actual legal exposure remains uncertain. The source article is written with an openly sarcastic, opinionated tone rather than as neutral reporting, though the underlying facts, the lawsuit and the CCDH report, are independently documented.
“Liability also attaches ... even if the company has deployed near-perfect, state-of-the-art technical controls to prevent the generation of nude images”
— xAI's lawsuit against Minnesota