Table of Contents
Do not index
Open figma.com/robots.txt and search for "
Google-Extended". You'll find this:User-Agent: Google-Extended
Disallow: /That line tells Google it cannot use Figma's content to ground answers in Gemini.
Check yours now. Open yourdomain.com/robots.txt, Ctrl-F "
Google-Extended". 10 seconds, no tools.Most people who added that line did it to stop AI training on their content. Fair. But it is worth knowing what you traded.
Google says Google-Extended does not affect Search or AI Overviews - it only controls Gemini training and grounding. But a study found sites blocking it were retrieved less often by both Gemini and AI Overviews than comparable sites.
Be careful with that: it is a correlation, not proof. Sites that block one crawler usually block many, and locked-down sites may be less quotable anyway. Nobody has shown the block causes the drop.
What is clear is that companies who live on being found are choosing the opposite. Zapier explicitly allows it, and adds a signal saying ai-train=yes, search=yes, ai-input=yes.
So the question is simply whether you meant it.
If you want AI to recommend you, allow it, and give it something worth quoting - pages that answer your buyers' real questions with clear comparisons and real numbers. If you would rather protect your content, keep the block and know the cost.
Either way, do not let it be an accident from a
robots.txt someone copied in 2023.Once you have let AI in, it needs something worth citing. That's what we built Outrank for: answer-shaped articles, straight to your CMS. Start free.
Tibo