NEWS · COMPANIES · #2152
Meta removes language that suggested it could scrape news sites to train AI
Meta updated its commercial terms to remove a clause that said the company "may use any content from such domains, web pages, or URLs" to develop, improve, or train its products — language critics said implied it could scrape news sites and override robots.txt. Meta spokespeople said the change clarifies the consolidated crawling terms applied only to ads and business products and does not alter how the company uses web crawlers; content owners can still signal preferences via robots.txt.
KEY POINTS
- Meta updated its commercial terms to remove a clause that said the company "may use any content from such domains, web pages, or URLs" to develop, improve, or train its products — language critics said implied it could scrape news sites and override robots.txt.
- Meta spokespeople said the change clarifies the consolidated crawling terms applied only to ads and business products and does not alter how the company uses web crawlers; content owners can still signal preferences via robots.txt.
- The change affects how Meta publicly frames its rights to web content for AI training and reassures publishers about the continued role of robots.txt and crawler practices.
WHY IT MATTERS
The change affects how Meta publicly frames its rights to web content for AI training and reassures publishers about the continued role of robots.txt and crawler practices.
SOURCES & TIMELINE
2Meta recently updated its commercial terms of use, declaring that the company can “use any content from [a business account’s] domains, web pages, or URLs to provide, develop, improve, research, train, and maintain the integrity of” Meta products and services. According to legal experts interviewed by Nieman Lab , this looked like a “remarkable provision” that could give Meta permission to scrape the website of any …
Language that experts say allowed the scraping of news sites to train its AI has been removed from Meta’s terms. After backlash, Meta is updating its terms again to remove language that suggested it could scrape content from sites, including news sites, to train its AI. Specifically, the following language no longer appears on the commercial terms page : … Meta may use any content from such domains, web pages, or U…