What the new AI training opt-out token means for your site

Robots.txt just picked up a new job. On October 30, OpenAI, Google, Microsoft, and Anthropic said they will all honor one shared opt-out signal for AI training, a line called __usage_ai-training that a site owner adds once instead of writing four separate rules for four separate crawlers.

What the AI training opt-out token actually does

The token comes from an IETF draft written by Anthropic's Kevin Kelley, and it covers one thing: whether a company can use your pages to train a future model. A site adds two lines to robots.txt, naming the shared crawler token and disallowing it across the site.

• User-Agent: __usage_ai-training
• Disallow: /

Why the timelines matter more than the token

Add the rule today and almost nothing changes until next year. Three of the four companies have not built the listener yet, so your site keeps getting crawled for training under the old rules until each one flips its own switch. Put a reminder on the calendar for January, not a victory lap now.

Training opt-out and search visibility pull in different directions

If you've put any work into showing up inside ChatGPT, Gemini, or Copilot answers, this is the fork to understand before touching robots.txt. Training is about the weights in the next model. Grounding and search are about today's answer citing your page right now. Only Microsoft has drawn a clean line between the two so far, with a separate setting for grounding. If OpenAI, Google, or Anthropic later blur that line the way Microsoft hasn't, a blanket opt-out written today could quietly cost you citations you wanted to keep.

1. 1

   Check who already crawls your site for training

   Pull your server logs or an SEO crawl report for GPTBot, Google-Extended, ClaudeBot, and Bingbot hits before deciding whether blocking training would even change your current traffic.

2. 2

   Add the token if you want your preference on record

   Drop the two lines into robots.txt now. Anthropic already honors it, and the other three will start checking once they ship their own listener.

3. 3

   Recheck in January 2027

   Microsoft's February 1 date and Google's 2027 target both land after the new year. Set the reminder instead of assuming the rule is doing anything yet.

None of this requires a rebuild, just a decision about where your content should and shouldn't end up. If you want a second set of eyes on your robots.txt, your crawl logs, or how your site shows up in AI answers versus AI training sets, send over what you've got and I'll take a look.