SEO
Robots.txt Validator
Validate pasted robots.txt content locally in your browser and check user-agent groups, allow and disallow rules, sitemap lines and common mistakes.
Tool
Validate pasted robots.txt content locally in your browser and check user-agent groups, allow and disallow rules, sitemap lines and common mistakes.
This tool runs in your browser. Your input is processed locally and is not uploaded.
About this Tool
Paste robots.txt content to inspect user-agent groups, allow and disallow rules, sitemap declarations and unusual directives. The validator highlights common formatting problems and risky rules such as accidentally blocking the whole site.
Examples
- User-agent: *\nDisallow: /admin/
- User-agent: *\nDisallow: /
- Validate robots.txt before publishing
Frequently Asked Questions
Does this tool fetch my live robots.txt?
No. Paste the robots.txt content manually. The validation runs locally in your browser.
Can it detect a full-site block?
Yes. It can warn when a rule such as Disallow: / appears to block all crawling for a user-agent group.
Does it guarantee how every crawler behaves?
No. Robots.txt interpretation can vary slightly. The tool gives practical syntax and rule hints, not crawler-specific guarantees.