For years, opting out of AI training meant writing a separate rule for every AI company — and hoping you had not missed one. On October 30, 2026, that changed: OpenAI, Google, Microsoft, and Anthropic all backed a single shared AI training opt-out signal that any website can express in one robots.txt group.
The signal is a policy token, __usage_ai-training. Add it to your robots.txt file, and every participating lab reads the same preference the same way.
What the four labs actually announced
Each company backed the same mechanism on October 30, 2026:
- OpenAI will honor a common robots.txt signal for multiple providers at once, aiming to implement it in the coming months.
- Google plans to support the shared control in 2027, based on IETF AI Preferences (AIPREF) work.
- Microsoft backs the shared preference and adds page-level controls for Bing, Copilot, and Microsoft 365.
- Anthropic authored the draft and has already begun implementing support, according to same-day reporting.
The coordination is the news: until now, opting out meant separate rules per crawler, repeated for every new crawler.
How the new token works
The mechanism is defined in an Internet-Draft titled “Machine-Readable Training Preference Signal,” authored by Kevin Kelley of Anthropic and dated October 30, 2026. It introduces no new directive names and no new grammar.
It reuses the robots.txt file every site owner already knows. To opt out everywhere except a press kit, a site writes this:
User-Agent: __usage_ai-training Disallow: / Allow: /press-kit/
That expresses a preference against AI training everywhere except /press-kit/. The draft defines AI training as using an asset to modify the learned parameters of a generative AI model.
Three draft rules matter most:
- No wildcard shortcut. User-Agent: * must not match the token; no token group means no preference expressed.
- Specific beats general. A company-specific control can override the shared signal for that company.
- Training is not crawling. The token “is not a directive for crawling behavior.”
The before-and-after in one diagram
BEFORE: one rule per vendor, easy to miss one
GPTBot block → Google-Extended block → ClaudeBot block → Bing block
NOW: one shared token, read by every participating lab
User-Agent: __usage_ai-training → Disallow: / → one preference, four labs
THE CATCH: each lab starts on its own date
Anthropic: already implementing · OpenAI: coming months · Microsoft: February 1, 2027 · Google: 2027
Evidence: the dates and the exact quotes
| Company | Commitment | Date |
|---|---|---|
| Anthropic | Draft author; implementing (reported Oct 30) | Underway |
| OpenAI | Will honor the signal; existing controls stay | Coming months |
| Microsoft | Honors the preferences as it crawls pages | Feb 1, 2027 |
| Shared control support, with other providers | 2027 |
Google’s John Mueller put the pitch in one sentence:
“Instead of managing separate instructions for every AI developer, website owners will be able to use a shared ‘policy token’ within robots.txt to set broad preferences across participating AI providers.”
— John Mueller, Google’s official announcement
OpenAI stressed continuity:
“This new signal will not replace the controls we already provide through robots.txt. Our existing crawler controls will continue to apply and be respected.”
— OpenAI, OpenAI’s official announcement
One hard limit: the draft is an individual Internet-Draft, not an endorsed standard, expiring May 3, 2027 unless revised.
What Microsoft adds on top
Microsoft went further, with three separate controls:
- AI Training: allow or prevent content from being used to train AI models.
- AI Grounding: allow or prevent content from helping generate AI answers in Bing, Copilot, and Microsoft 365.
- Limited Grounding (DisallowExtended): allow grounding in Microsoft’s own products, but not in grounding services Microsoft sells to third parties.
Its timeline is the most specific: temporary grounding controls on December 15, 2026; enforcement from February 1, 2027; a March 1, 2027 deadline to replace NoArchive and NoCache; temporary preferences expire September 30, 2027. Training preference also extends to single pages via a meta tag or HTTP header, per Search Engine Land’s report on Microsoft’s Bing announcement.
What this does not do
It has four clear limits:
- It does not block crawling. A crawler can still fetch a page for search or other purposes.
- It does not remove old data. Already-collected datasets and trained models are untouched.
- It does not cover AI answers. Training opt-out is not grounding opt-out.
- It is voluntary. It binds only companies that honor it, and private agreements can override it.
One Google nuance: using the shared token without naming Google-Extended counts as disallowing it, while explicitly allowing it keeps pages eligible for Gemini Apps grounding clicks.
How to profit: sell the opt-out audit now
Every site owner with original content now has a decision to make — and most will not make it alone. That is a service, and the window is open while the announcement is fresh.
- Audit the current file. List every AI rule in place: GPTBot, Google-Extended, ClaudeBot, Bing controls, and useless wildcard blocks.
- Decide training vs. grounding separately. Premium guides and courses usually get a “no” on training; free marketing pages stay open for visibility.
- Write the new group. Add __usage_ai-training with Allow exceptions, keeping company-specific controls alongside it.
- Set the Microsoft calendar. Diary December 15, 2026, February 1, 2027, and March 1, 2027.
- Verify, don’t assume. Fetch the live robots.txt, confirm the token group is present and correctly spelled, and re-check after every migration.
- Report in plain English. One page: what is opted out, what stays visible, and which dates to revisit.
Pitch bloggers, course creators, and agencies first: the most original content, the least in-house help, and the most to lose.
Frequently asked questions
Does one token really cover all four labs?
Yes, for labs that honor it, once each lab’s support is live.
Will this keep my pages out of Google Search or AI answers?
No. The token governs training only. It does not block crawling, search indexing, or grounding.
Do I still need GPTBot and Google-Extended rules?
Yes, for now. Both OpenAI and Google say their existing controls continue alongside the shared signal.
Does opting out delete my content from existing models?
No. It expresses a preference for future training only.
Is this an official internet standard yet?
Not yet. It is an individual Internet-Draft, the IETF draft “Machine-Readable Training Preference Signal”, expiring May 3, 2027 unless revised, and the commitments are voluntary.
Should a small site bother with this?
Yes. One small edit protects original content; re-check it when Google and Microsoft support goes live in 2027.
Keep Reading
Sources: OpenAI’s official announcement on honoring the shared robots.txt signal; Google’s official October 30, 2026 announcement of the shared AI training control; the IETF draft titled “Machine-Readable Training Preference Signal”; Search Engine Land’s report on Microsoft’s Bing announcement.