AI Giants Pledge Standardized Method for Publishers to Block Content Scraping

Major AI companies, including OpenAI, Google, Microsoft, and Anthropic, have committed to developing a single, standardized protocol that will allow media outlets and other websites to prevent their content from being scraped for training artificial intelligence models. This initiative aims to simplify the process for publishers, who currently must implement multiple blocking commands for various web-crawling bots. While the solution will not be fully implemented for several months, with Microsoft targeting February 2027 and Google later in 2027, Anthropic has already begun rolling it out. This development addresses growing criticisms and legal challenges from news publishers regarding unauthorized use of their content for AI training.