Does this tool fetch my live robots.txt?
No. Paste the robots.txt content manually. The validation runs locally in your browser.
SEO
Robots.txt Validator
Validate pasted robots.txt content locally in your browser and check user-agent groups, allow and disallow rules, sitemap lines and common mistakes.
Tool
Robots.txt Validator
Validate pasted robots.txt content locally in your browser and check user-agent groups, allow and disallow rules, sitemap lines and common mistakes.
This tool runs in your browser. Your input is processed locally and is not uploaded.
About this Tool
Paste robots.txt content to inspect user-agent groups, allow and disallow rules, sitemap declarations and unusual directives. The validator highlights common formatting problems and risky rules such as accidentally blocking the whole site.
Examples
- User-agent: *\nDisallow: /admin/
- User-agent: *\nDisallow: /
- Validate robots.txt before publishing
Frequently Asked Questions
Can it detect a full-site block?
Yes. It can warn when a rule such as Disallow: / appears to block all crawling for a user-agent group.
Does it guarantee how every crawler behaves?
No. Robots.txt interpretation can vary slightly. The tool gives practical syntax and rule hints, not crawler-specific guarantees.
Related Tools
URL List to Sitemap
Create an XML sitemap from a list of URLs.
TOOLSitemap Validator
Validate XML sitemap structure and common sitemap fields.
TOOLXML Sitemap Viewer
Turn an XML sitemap into a readable URL table.
TOOLXML Sitemap to CSV
Convert XML sitemap entries into CSV rows.
TOOLRobots.txt Generator
Create robots.txt rules for crawlers and sitemap discovery.
TOOLRobots.txt Tester
Test URLs and paths against robots.txt allow and disallow rules.
TOOL