Robots.txt Checker
Fetch any site's robots.txt and validate every rule: blocked pages, conflicting directives, broken sitemap references, and crawl budget mistakes.
How It Works
We read the file, parse the rules, and translate them into plain language.
Enter a domain or URL
Paste any website to fetch its robots.txt automatically.
Rules are validated
Each directive is checked against the Robots Exclusion Protocol.
Review the findings
See blocked pages, conflicts, and sitemap references in plain language.
What We Check
Every directive that decides what Google can and cannot crawl.
Rule Validation
Checks syntax of User-agent, Disallow, Allow, and Sitemap directives.
Blocked Page Detection
Shows which of your site's pages are being hidden from search engines.
Directive Conflicts
Flags contradictory Allow and Disallow rules that produce unexpected behavior.
Sitemap Reference Check
Verifies the Sitemap directive points at an existing, valid XML sitemap.
Crawl Budget Insight
Explains how your rules direct (or waste) Google's crawl budget.
Plain-Language Output
Translates raw robots.txt syntax into what it actually means for your site.
Who Uses the Robots.txt Checker
Site owners who need to know nothing important is blocked.
Frequently Asked Questions
Is Robots.txt Blocking Your Pages?
One bad rule can hide your best content from Google. Verify your file in seconds.
Check Robots.txt Now