Meta removes terms language critics read as allowing news-site scraping for AI
Meta has updated its commercial terms again after backlash, removing language that suggested it could scrape content from websites, including news sites, to train its AI. The Verge reports that the passage no longer appears on the commercial terms page.
The removed sentence read: "Meta may use any content from such domains, web pages, or URLs to provide, develop, improve, research, train, and maintain the integrity of, the Meta Company Products, services, and features provided under these Commercial Terms and any other applicable supplemental terms."
Many people, including Neiman Lab and James Grimmelmann, a professor of digital and information law at Cornell, interpreted the wording as an attempt to give Meta the right to use content scraped from sites to train its AI products for free, and to supersede the blocking instructions in a site's robots.txt file.
Meta first pushed back. Andy Stone, Meta's vice president of communications, and spokesperson Daniel Roberts said: "There is no change to how Meta uses web crawlers; any suggestion to the contrary is false." The latest update goes further than that, according to The Verge: the controversial language is gone entirely, and Meta issued a new statement.
In a follow-up statement to The Verge, Roberts said Meta had heard the criticism and wanted to clarify. The crawling terms it consolidated, he said, only applied to its ads and business products, and Meta has now made that clear in the terms themselves. He added that the change does not alter how Meta uses web crawlers, that content owners can still use robots.txt to signal their preferences, and that "This is not about scraping news sites, and these updates don’t give us new rights that supersede those preferences."
Key facts
- Meta updated its commercial terms again after backlash and removed language suggesting it could scrape content from sites, including news sites, to train its AI.
- The deleted sentence said Meta may use any content from such domains, web pages, or URLs to provide, develop, improve, research, train, and maintain its products, services, and features.
- Neiman Lab and Cornell law professor James Grimmelmann, among others, read it as a bid to use scraped content for AI training for free and override robots.txt blocks.
- Spokesperson Daniel Roberts says the consolidated crawling terms only applied to Meta's ads and business products and that robots.txt preferences still stand.
Why it matters
Whether AI developers must respect a site's robots.txt is a live question for publishers. Wording in a company's terms that appeared to let it train on scraped content, and to override those blocking instructions, drew a quick reaction from commentators. Meta removed the passage entirely rather than only defending it, which is a step beyond its first denial.
Who it affects
Publishers and other site owners who rely on robots.txt to signal what crawlers may do are the group the dispute concerns, and news sites were named specifically. Businesses using Meta's ads and business products are the audience the commercial terms address, and Meta says the crawling terms applied to those products.
How to use it
Per Roberts' statement, content owners can use robots.txt to signal their preferences to Meta, as they could before. Anyone who wants to check the current position can read the commercial terms page, where the quoted passage no longer appears.
How solid is it
The removal is reported by The Verge, which says the language no longer appears on the commercial terms page. Meta's explanation comes from Roberts, a company spokesperson, and Andy Stone, Meta's vice president of communications, so it is Meta's own account. The interpretation that the wording allowed scraping news sites is attributed to Neiman Lab, Grimmelmann and others, and the article's subtitle says experts hold this view.
Risks and caveats
The article does not say that Meta actually scraped any news sites or trained AI on scraped news content. It does not say when the original language was added or who wrote it, and it does not say which Meta AI products or models the language concerned beyond "its AI". It reports no legal action, regulator response or publisher reaction beyond the interpretation attributed to Neiman Lab, Grimmelmann and others. No calendar date is given for the update, only "today".
“This is not about scraping news sites, and these updates don’t give us new rights that supersede those preferences.”
— Daniel Roberts, Meta spokesperson, in a follow-up statement to The Verge