GPTBot: What It Does and How to Control It
OpenAI's crawler that collects publicly available web pages to train its generative AI models. It is separate from ChatGPT-User (real-time fetches) and OAI-SearchBot (search indexing).
What happens if you allow or block it
Your pages may be included in the vendor's training datasets.
your content stops being used for OpenAI model training — ChatGPT search answers and user-triggered browsing keep working.
The robots.txt rule
This is the exact rule auditme.dev publishes for GPTBot:
User-agent: GPTBot Allow: / Disallow: /auth/ /_next/ /private/ /dashboard /settings /profile Disallow: ... (25 more paths)
The full path list lives in our robots.txt.
Frequently Asked Questions
Is GPTBot allowed on auditme.dev?
Yes. Our robots.txt explicitly allows GPTBot to crawl public pages while keeping private areas (auth, dashboard, internal APIs) disallowed.
Does blocking GPTBot hurt Google rankings?
No. GPTBot is OpenAI's crawler and works independently from Googlebot. Blocking it never changes your Google rankings — it only changes your content stops being used for OpenAI model training — ChatGPT search answers and user-triggered browsing keep working.
How do I allow or block GPTBot on my own site?
Add a "User-agent: GPTBot" block to your robots.txt with Allow or Disallow rules, then verify the file parses correctly.