Microsoft split AI into two reads, and only one of them can be measured from a page.

Bing, OpenAI and Google each draw the line around the answer-time read in a different place. One robots.txt block under an "AI" comment can shut a business out of ChatGPT search and Gemini grounding while Bing ranks it exactly as before.

· 9 minute read

On October 30, 2026, Microsoft published a post on the Bing blog that gives publishers two separate switches where there used to be one tangled pair. AI Training covers using content to change a model's learned parameters. AI Grounding covers using content as input to an answer being generated right now. Bing begins honoring each preference as it crawls from February 1, 2027, and after March 1, 2027 the old NoArchive and NoCache tags stop counting as either. Most readers will file this under publishers gaining control. Here is a more useful reading. Run one ordinary robots.txt, a block of AI crawlers under a comment reading "AI," through the three engines' documentation: it refuses training at OpenAI and Google, removes the business from ChatGPT search and Gemini grounding, and from March 1, 2027 leaves Bing free to train on and ground the same pages. Microsoft has documented, in its own words, that being learned from and being retrieved are different machine reads, and the retrieved read is the one a business can measure from its own pages.

Microsoft now defines AI training and AI grounding as separate uses of a page

Microsoft's Bing team defines AI Training and AI Grounding as two independent uses of the same web page, each with its own setting. In the October 30, 2026 post, AI Training "covers using an asset to modify the learned parameters of a generative AI model," while AI Grounding "covers using content as input to help inform AI-generated responses." The post states that "your training setting does not change your grounding setting, and vice versa," and both default to Allow when nothing is published.

The two settings live in different places. Training can be set for a whole site or a path in robots.txt under the user agent __usage_ai-training, or on a single page through a Bingbot meta tag or an X-Robots-Tag header. Grounding exists only at the page level, through the same meta tag or header, with three values: Allow, DisallowExtended and Disallow. DisallowExtended limits grounding to Microsoft-operated experiences, which the post lists as AI Mode, AI Overview, Copilot and Microsoft 365. It excludes grounding services Microsoft provides to third parties, such as Web IQ and grounding through API access in Foundry.

Neither setting touches ordinary search. Microsoft says the usage preferences "do not change eligibility for traditional Bing search results."

Bing's old NoArchive tag removed a business from AI answers and training at once

Bing's NoArchive tag has treated training and answer retrieval as a single decision since September 2023. In that month's announcement on the Bing Webmaster blog, Microsoft said content tagged NOARCHIVE "will not be included in Bing Chat answers, not be linked to in the answers," and would not be used to train its foundation models. NOCACHE allowed inclusion in chat answers but limited use to URLs, titles and snippets.

A business that set NoArchive to stop its pages from feeding a model also removed itself from Bing's chat answers. NoArchive began as a search directive about cached copies, so a page could carry it for reasons that had nothing to do with AI. One directive governed two reads, and the cost of refusing the first was paid in the second.

The 2026 post retires that coupling. After March 1, 2027, NoArchive and NoCache "will no longer be read as a preference for training or grounding." A page whose only AI signal is NoArchive will then fall to the default and be treated as allowing both. A site that wants to hold its grounding restrictions through the transition must publish a page-level AI Grounding preference and record a temporary one in Bing Webmaster Tools, which opens for this on December 15, 2026. A business that wants to go on refusing training now has to say so with the training signal itself.

OpenAI and Google draw the line between training and retrieval in different places

OpenAI and Google both separate training from search, but neither draws the line where Microsoft does. OpenAI runs two crawlers. GPTBot collects content that "may be used in training our generative AI foundation models," and OAI-SearchBot "is used to surface websites in search results in ChatGPT's search features." OpenAI's documentation says "Each setting is independent of the others," and that sites opted out of OAI-SearchBot "will not be shown in ChatGPT search answers."

Google's Google-Extended token bundles the two reads Microsoft has split. Google's crawler documentation describes it as governing both the training of future Gemini models and grounding, which Google defines as providing content from the Google Search index to the model at prompt time. The same entry says Google-Extended does not affect a site's inclusion in Google Search and is not a ranking signal. It also says Google intends to support the shared AIPREF training signal, the same __usage_ai-training user agent Bing reads, in early 2027.

Microsoft and Google are moving toward one shared training signal. Retrieval controls are not converging: each engine names and scopes them differently.

A robots.txt that blocks "AI" as one thing shuts two answer surfaces and leaves Bing open

AI visibility measurement earns its place when one ordinary configuration is run through each engine and given a verdict for each read. Take a service business whose robots.txt blocks GPTBot, OAI-SearchBot and Google-Extended under a comment reading "AI," with no rule for Bingbot and no __usage_ai-training group. Its pages carry a leftover NoArchive tag and no Bingbot ai-grounding value.

ChatGPT search: excluded, because OAI-SearchBot is blocked and OpenAI removes opted-out sites from ChatGPT search answers. Gemini grounding: excluded, because Google scopes Google-Extended to grounding as well as training. Google Search: unchanged, since Google states the token does not affect inclusion in Google Search, so the Google surface lost is Gemini grounding. Bing ranking: unchanged, since Microsoft's usage preferences leave traditional Bing results alone. Bing grounding: excluded today through NoArchive, and allowed after March 1, 2027, when Bing stops reading NoArchive as an AI preference and the unset value defaults to Allow. Bing training opens on the same date, since the file never published the training signal.

The owner wrote one block to refuse training. The file does refuse training at OpenAI and Google. It also removes the business from two answer surfaces, and from March 1 it leaves Bing free to train on and ground the same pages. The block controlled the read that leaves no measurable trace and gave up the reads that name a business in an answer.

Every verdict above came from things anyone can fetch: a robots.txt file, a meta tag, a response header and three engines' published documentation. No fetch shows whether the same pages moved a model's weights. Grounding can be checked from outside; training cannot.

A deterministic AI visibility check answers three questions, and each answer names its fix

A deterministic AI visibility check reaches each engine's verdict by answering three questions in order, and every answer points to one repair. Can the retriever crawl the page? Microsoft is explicit that "An Allow setting does not override a robots.txt rule blocking Bingbot from crawling." If the answer is no, unblock that engine's crawler in robots.txt.

Is the page permitted for this use? The check reads robots.txt, the Bingbot meta tag and the X-Robots-Tag header, which carries the same instruction where no visitor sees it. If the answer is no, publish Allow for AI grounding in the meta tag or the header.

Has the engine applied the setting? The page cannot show that: Microsoft writes that "Publishing a setting is not the same as activating it," because Bing must recrawl first. If the setting is published but not yet live, request a recrawl through Bing Webmaster Tools.

The check reports what was published, never what a model remembers. Same page, same robots.txt, same verdicts every time.

The strongest case for blocking AI grounding comes from businesses that sell the page

The strongest objection to leaving AI grounding open is that grounding is the read that replaces the visit. A model is trained on a page once. A grounded answer uses the page every time someone asks, and if the answer is complete, the person never clicks through. Some publishers' product is the page itself (a news site, a recipe archive, a reference library). For them, grounding turns the work into someone else's answer and removes the ad impression or subscription prompt that paid for it. DisallowExtended gives that publisher a narrower refusal, cutting off third-party grounding services while Microsoft's own surfaces stay open.

The objection is correct for that publisher, and it is the reason the split matters. A business whose product is a service (a repair, an appointment, a consultation, a delivery) is not selling the page. For that business, the answer that names it is the outcome the page existed to produce. Blocking grounding to protect a visit that would have ended in the same recommendation gives up the recommendation itself.

The real limit on this argument is evidence about size. Microsoft states the direction of the cost in one sentence: "If you select DisallowExtended or Disallow for grounding, you will see reduced brand visibility in AI experiences." It publishes no number, and no public dataset yet measures how often a grounded answer sends a customer rather than simply answering the question. A service business should publish Allow for AI grounding today: the company that runs the system has stated which way the cost runs, and the outcome the publisher fears, an answer that replaces the visit, is the outcome a service business wants. DisallowExtended does not change that judgment. It carries the same stated cost as Disallow, so it serves only a business with a specific objection to third-party grounding services such as Web IQ or Foundry, and that business pays for the objection in visibility.

Refusing AI training costs no visibility that any engine has documented

The objection that most threatens this argument is that AI training shapes visibility too. A model that learned a brand during training can name it in an answer with no retrieval at all, so a skeptic will say refusing training is not free: a business that opts out of every training crawler may fade from answers written from memory. The mechanism is real. What the skeptic cannot point to is a stated cost.

The engines' own documentation draws the cost line around retrieval alone. Microsoft attaches a visibility cost to exactly one refusal, grounding, where its post warns of reduced brand visibility in AI experiences; its definition of training carries no such warning. Training changes a model's parameters with no link back to the page and no attribution in any single answer, so a brand named from memory arrives without a source the business could click, verify or measure. OpenAI runs GPTBot and OAI-SearchBot as independent settings and documents a visibility consequence for only one of them: opting out of OAI-SearchBot removes a site from ChatGPT search answers. Two companies that run answer engines have each written down which refusal costs a business its place in the answer, and both named the retrieval read.

Parametric recall also fails the test a measurement has to pass. No robots.txt line, meta tag or header shows whether a model learned a brand, so nothing on the page can confirm or rule it out. Whether the retrievers can reach the page is fixed by the page itself, and that is the read worth measuring.

The block controlled the read that leaves no measurable trace and gave up the reads that name a business in an answer.

Grounding is the read with a stated price. Microsoft has written it into its controls as a separate use, with a separate switch and a named cost, and OpenAI gives answer-time search its own crawler. A business can refuse training on principle and lose nothing any engine has said it costs. A business that refuses grounding, or blocks "AI" as one thing without knowing which read it blocked, has removed itself from the answer while its ranking stays put. The ranking will never show the difference.

Further reading

  1. Microsoft Bing (Krishna Madhavan), the primary announcement separating AI Training and AI Grounding preferences, with definitions, syntax, dates and the visibility warning blogs.bing.com/search/2026/10/your-content-your-choice-clearer-preferences-for-training-and-grounding
  2. Microsoft Bing (Fabrice Canel), the 2023 announcement that tied NOARCHIVE and NOCACHE to both Bing Chat answers and model training blogs.bing.com/webmaster/september-2023/Announcing-new-options-for-webmasters-to-control-usage-of-their-content-in-Bing-Chat
  3. OpenAI, crawler documentation separating GPTBot (training) from OAI-SearchBot (ChatGPT search) (living documentation, read 2026-11-12) developers.openai.com/api/docs/bots