gptbot · claudebot · perplexitybot · free
Which AI crawlers can read your site?
One robots.txt line can remove you from ChatGPT search or Perplexity without touching your Google rankings. See exactly which assistant crawlers you allow, block, or partly block — and what each one feeds.
questions people ask
Should I block GPTBot?
Blocking GPTBot stops OpenAI using your pages for model training. It does not stop ChatGPT search from citing you — that is OAI-SearchBot. Many sites block the first believing they are protecting rankings, and accidentally block the second. This tool shows each one separately.
Does Google-Extended affect my rankings?
No. Google-Extended only controls whether your content trains Gemini. Google Search and AI Overviews use Googlebot, and blocking Googlebot removes you from search itself.
What is llms.txt?
A proposed plain-text file at your site root that tells language models what your site is and which pages matter most, in the spirit of robots.txt. It is harmless to publish; there is not yet evidence it changes citations. We report whether you have one.
Why does Bing matter for AI visibility?
ChatGPT's web search and Microsoft Copilot draw on Bing's index. A site that is indexed by Google but not by Bing can rank well and still never be cited by those assistants. Blocking Bingbot has the same effect as not existing to them.
Want the pages that earn the citations written for you?
IT Master researches, writes, checks with rival AI models and publishes to your site — then runs this probe every night on your queries.