OAI-SearchBot: What It Does and How to Control It
OpenAI's search crawler. It indexes the pages that power ChatGPT's search feature and the source links ChatGPT shows next to answers.
What happens if you allow or block it
Your pages can be indexed and cited in the vendor's search and AI answers.
your pages stop appearing as sources and citations inside ChatGPT search answers.
The robots.txt rule
This is the exact rule auditme.dev publishes for OAI-SearchBot:
User-agent: OAI-SearchBot Allow: / Disallow: /auth/ /_next/ /private/ /dashboard /settings /profile Disallow: ... (25 more paths)
The full path list lives in our robots.txt.
Frequently Asked Questions
Is OAI-SearchBot allowed on auditme.dev?
Yes. Our robots.txt explicitly allows OAI-SearchBot to crawl public pages while keeping private areas (auth, dashboard, internal APIs) disallowed.
Does blocking OAI-SearchBot hurt Google rankings?
No. OAI-SearchBot is OpenAI's crawler and works independently from Googlebot. Blocking it never changes your Google rankings — it only changes your pages stop appearing as sources and citations inside ChatGPT search answers.
How do I allow or block OAI-SearchBot on my own site?
Add a "User-agent: OAI-SearchBot" block to your robots.txt with Allow or Disallow rules, then verify the file parses correctly.