This news probably won’t have much effect on stock prices, but I think it’s pretty significant.
Google, OpenAI, Microsoft, and Anthropic have agreed to support a single, unified rule: websites can add just one line to their robots.txt file to tell all four, “Don’t use my content for training.”
How did it work before? Each company had its own crawler name, so site owners had to block them one by one. Miss one and it was all for nothing. Now it’s all done in one line.
The timelines differ: Google says it will start supporting it next year; OpenAI says in the next few months; Microsoft has set February 1 next year as its date; and Anthropic says it has already started. For now, this rule is still just a draft, not an official standard.
Why push this now? I can think of two reasons: first, copyright lawsuits are piling up, and everyone wants to keep some evidence that “we respect people’s choices”; second, regulatory pressure—setting the rules yourself is always better than having someone else set them for you.
It won’t have much direct impact on ordinary users. But for content platforms, media companies, and data companies, this could gradually change the price of training data—the data that people are willing to let you use will become increasingly valuable.
This past year, the AI industry has been competing on computing power and models. Now people are starting to seriously address the question of “where the data comes from.” That’s also a sign that the industry is maturing.
The above content is for reference only and does not constitute investment advice.
Follow me, haha!