Skip to main content
cd ../notes (back to all notes)
category
"AI Search Visibility"
read
7 min
date

Block Google AI Training Without Leaving AI Overviews

Google-Extended and the Search preview controls do different jobs. Here is which one limits AI training and which one changes AI Overviews.

Yes. Google uses separate controls for AI training outside Search and for Search itself. Google-Extended is a robots.txt control for AI training and grounding in some of Google’s systems outside Search. AI Overviews and AI Mode are part of Google Search, and Google lists different controls for those.

Google’s guide to AI features in Search says robots.txt rules for Googlebot are the control for how your site is crawled for Search. To limit what Search shows from your pages, it points to nosnippet, data-nosnippet, max-snippet and noindex. Google-Extended is listed on that page only as the way to limit AI training and grounding in some of Google’s other systems. Google’s crawler documentation describes Google-Extended the same way, as a standalone token for training future models and for grounding in specific Google AI products.

So you are making two decisions:

  1. Do you want your content used for that AI training and grounding? That is the Google-Extended decision, made once for the site or for chosen folders.
  2. How much of each page do you want shown in Search, including AI Overviews and AI Mode? That is a preview control decision, made page by page.

For most service pages, I would leave the second one alone. The first is a values call, and you can make it in a few minutes once you know what each setting does.

There is one thing to check before you change anything. Google’s guidance for AI features says your pages need to meet the same technical requirements as the rest of Search, and that includes not blocking Googlebot. If an old robots.txt rule blocks Googlebot from your service pages, fix that first.

If your goal is to get recommended more often in AI search, these settings are not the lever. That work lives in your pages. My post on why AI search skips local service businesses covers what answer engines need to see before they recommend you.

Which Google control changes what?

Each control does one job, and mixing them up can hide pages you wanted found.

ControlWhere it goesWhat Google says it changes
Google-ExtendedA group in robots.txtWhether crawled content is used for AI training and grounding in some Google systems outside Search
nosnippetRobots meta tagNo text snippet for the page in search results, including AI Overviews and AI Mode
max-snippetRobots meta tagA maximum number of characters for the page’s text snippet
data-nosnippetHTML attribute on part of a pageKeeps that part of the page out of snippets
noindexRobots meta tag or HTTP headerDrops the page from Google Search results

Read the table from top to bottom as a scale. Google-Extended sits apart from your Search settings. The snippet controls change how a page appears in Search, including in AI Overviews. And noindex takes the page out of Google Search results.

That scale matters because the preview controls cut both ways. Google’s robots meta tag documentation says nosnippet applies to all forms of search results, and its snippet guide describes nosnippet as the way to stop a snippet showing in search results. So if you add nosnippet to a service page to keep it out of AI Overviews, the regular search result loses its text snippet too. For a page whose whole job is to earn a click or a call, that is a real cost.

The most targeted option is data-nosnippet. Google lets you mark part of a page with this attribute so that text is not used in a snippet. If one block on a page is the problem, such as a detailed process you would rather explain on a call, you can fence off that block and leave the rest of the page working for you.

How do you add Google-Extended to your robots.txt?

You add one group to the robots.txt file at the root of your domain. Google’s documentation and the Robots Exclusion Protocol standard both say the file lives at the top level of the site, such as example.com/robots.txt, and that a file in a subfolder will not be used.

A group that keeps all of your content out of Google-Extended looks like this:

User-agent: Google-Extended
Disallow: /

A few details make this work:

  • Keep your Googlebot rules as they are. Google-Extended is its own token, and Google’s guide to AI features in Search names Googlebot rules as the control for Search crawling.
  • Write the Google-Extended group in full. Google follows the most specific group that matches a crawler’s user agent. Your general rules for all crawlers, the group that starts with an asterisk, are not combined with it. So a Google-Extended group needs to say everything you want it to follow.
  • You can be selective. Google’s own example allows one folder and disallows the folder around it. If only part of your site should stay out of AI training, such as client resources or a members area, disallow just that path.
  • Match the paths exactly. Google treats robots.txt paths as case-sensitive, so /Resources and /resources are different rules. A disallow of a single slash covers the root and every page below it, which is why the example above blocks the whole site for Google-Extended.

If your site runs on a website builder, you may not be able to edit robots.txt directly. Google notes that some hosting services and content management systems offer a search settings page instead. Look there before you ask a developer to change files, and write down what you changed and when, so the next person who touches the site knows the setting is deliberate.

Keep one limit in mind. Google states that robots.txt is not a way to keep a web page out of Google and cannot force a crawler to follow its rules. The robots.txt standard says its rules are not a form of access authorization or a substitute for real content security. So protect anything private with a password, and use noindex for a page that only needs to stay out of Google Search results.

When should a small business limit AI Overviews?

Rarely, and only for a specific page or section with a clear reason.

For most service businesses, AI Overviews are a place you want to be. Google says there are no additional requirements to appear in AI Overviews or AI Mode, and that meeting the regular technical requirements for Search covers those AI formats too. If a buyer asks a question your service page answers well, a supporting link in that answer puts you in front of them.

Limiting AI Overviews makes sense in narrower cases:

  • A page you want read in full. Detailed pricing logic or a scope explanation that only makes sense in context. Here, data-nosnippet on the sensitive block, or max-snippet for the whole page, is usually enough.
  • Content you license or sell. If the page itself is the product, such as paid guides or templates, keeping it out of snippets protects the thing you charge for.
  • A page that should not be in Search at all. A thank-you page, a client portal or an internal resource. That calls for noindex or a password, not a snippet control.

If none of those apply, leave the preview controls alone and put the time into the pages themselves. Google’s guidance for AI features points to the same basics as the rest of Search, such as helpful content written for people. The FAQ sections that bring better leads post shows one way to structure clear answers, and the service page blocks that turn traffic into calls post covers the rest of the page.

If you do change a preview control or add noindex, give it time before judging the result. Google says it has to recrawl a page before it sees a change like this, and that a recrawl can take months. Make one change, note the date, and check your Search Console Performance report before you change anything else.

If you would rather have someone look at your site’s settings and pages together, you can apply to work with me.

Frequently asked questions

Does noindex work on a page I also blocked in robots.txt?

No. Google says the page must not be blocked by robots.txt for noindex to work, because the crawler has to reach the page to see the rule. A blocked page can still appear in results if other pages link to it.

Can a site have more than one robots.txt file?

One per host. Google says a site can have only one robots.txt file, at the root of the host it applies to. A subdomain such as shop.example.com needs its own file, because a robots.txt file only covers its own host.

How soon does Google see a robots.txt change?

Google says it generally caches robots.txt for up to 24 hours. The robots.txt standard also says crawlers should not use a cached copy for more than 24 hours unless the file is unreachable.

Will nosnippet remove my page from Google?

No. Google says nosnippet stops a text snippet from showing for the page. The rule that drops a page from Google Search results is noindex.

Start a conversation

What could work better?

I look at what is breaking, what you already use, and what your team will realistically adopt.

Let’s talk