Claude-User: What It Does and How to Control It
Fetches pages in real time when an individual Claude user asks it to open or read a specific URL during a conversation.
What happens if you allow or block it
The vendor can open your page when one of its users asks it to.
Claude cannot retrieve your page for a user who requested it directly.
The robots.txt rule
This is the exact rule auditme.dev publishes for Claude-User:
User-agent: Claude-User Allow: / Disallow: /auth/ /_next/ /private/ /dashboard /settings /profile Disallow: ... (25 more paths)
The full path list lives in our robots.txt.
Frequently Asked Questions
Is Claude-User allowed on auditme.dev?
Yes. Our robots.txt explicitly allows Claude-User to crawl public pages while keeping private areas (auth, dashboard, internal APIs) disallowed.
Does blocking Claude-User hurt Google rankings?
No. Claude-User is Anthropic's crawler and works independently from Googlebot. Blocking it never changes your Google rankings — it only changes Claude cannot retrieve your page for a user who requested it directly.
How do I allow or block Claude-User on my own site?
Add a "User-agent: Claude-User" block to your robots.txt with Allow or Disallow rules, then verify the file parses correctly.