Tech Meridian ← ENTITY INDEX
PROMY MERIDIAN RU

TOPIC · ENTITY #19813

robots.txt

Related event timeline, sources and context from the news index.

EVENT TIMELINE

2

REGULATION · 1 SOURCE · The Verge AI

Experts: Meta business-terms update could still allow scraping news sites

Meta updated its business terms for business accounts effective Oct. 30; the company removed explicit language saying it would 'train' its products on site content, but legal experts say remaining phrases like 'develop' and 'associated with your products or services' could still be used to justify scraping and using news-site content for AI development. Meta says content owners can use robots.txt to signal preferences and that the update does not give new rights that supersede those preferences.

7.0

COMPANIES · 2 SOURCES · The Verge AI · TechCrunch AI

Meta removes language that suggested it could scrape news sites to train AI

Meta updated its commercial terms to remove a clause that said the company "may use any content from such domains, web pages, or URLs" to develop, improve, or train its products — language critics said implied it could scrape news sites and override robots.txt. Meta spokespeople said the change clarifies the consolidated crawling terms applied only to ads and business products and does not alter how the company uses web crawlers; content owners can still signal preferences via robots.txt.

7.0