Updated · Built and checked by the PlinzoTools team, formulas tested against published sources
Test whether a specific path is allowed or blocked by a robots.txt file, using Google's own matching rules.
/admin/public/page
Matched rule: Allow: /admin/public/
This tool tests whether a specific path is allowed or blocked for a given user-agent, using Google's longest-match rule: among all matching rules, the longest matching path wins; if the longest Allow and Disallow rules are equally long, Allow wins. Rules are matched as literal path prefixes only -- * wildcards and $ end anchors are not expanded.
Paste your robots.txt content, enter a user-agent, and enter a path to test. The result (allowed/blocked) and the exact matching rule appear instantly.
1. Find the rule group whose user-agent exactly matches the one you entered (case-insensitive), falling back to the wildcard "*" group if none does.
2. Among that group's rules, find every rule whose path is a literal prefix of the tested path (no wildcard or end-anchor expansion).
3. The LONGEST matching rule wins; ties go to Allow.
The tool starts with a small sample file: a wildcard group that disallows /admin/ and then allows /admin/public/. Test the path /admin/public/page and two rules match by prefix: Disallow: /admin/ (7 characters) and Allow: /admin/public/ (14 characters). The longer rule wins, so the result is Allowed and the matched rule shown is Allow: /admin/public/. Now change the path to /admin/settings. Only the Disallow rule matches, so the result is Blocked. Change it to /blog/post and nothing matches, which gives the "No matching rule (default allow)" message.
This is a good way to learn why rule length matters. If you rewrote the Allow line as Allow: /admin/p (8 characters), it would still beat the 7-character Disallow for paths that start with /admin/p, but it would also accidentally open /admin/private-reports/. Testing a handful of sensitive paths against a draft file before it goes live catches exactly this kind of overlap.
Knowing what the tester does and does not do helps you interpret its verdicts correctly.
The large label says Allowed or Blocked, and the line beneath shows the exact rule responsible. If it says the rule is Disallow with a path, you know precisely which line to edit. If the rule is an Allow, check whether a longer Disallow was intended to override it. "Default allow" means your file has nothing to say about that path, which is the normal state for public pages.
Remember that a Blocked verdict describes crawling only. It does not remove a page from an index, and it is not a security control: robots.txt is publicly readable and only respected by well-behaved crawlers. Pages that must stay private need authentication, and pages that should stay out of search results need a noindex directive that crawlers are allowed to fetch.
The default is allowed -- robots.txt is an opt-out mechanism, not opt-in.
Not directly -- it prevents crawling. A blocked page can sometimes still appear in search results (without a crawled description) if it's linked from elsewhere. Use a noindex meta tag to actually prevent indexing -- see the Meta Robots Tag Checker.
In this tester, if any group lists your user-agent by name, only those groups are used. The wildcard group applies only when no group names the crawler. Repeat any shared rules inside the bot-specific group if you want them to apply.
No. It only evaluates the text you paste into the box, entirely in your browser. Copy the current contents of your live file into it, or paste a draft to test before publishing.
Rule paths are matched the way Google documents: from the start of the path, with an asterisk matching any characters and a trailing dollar sign anchoring the end. If the pattern still does not behave as you expect, check the case of the path (matching is case sensitive) and whether a longer Allow or Disallow rule is winning.
No rules apply, so the result is allowed with the message that no matching rule was found.
Reference: Google Search Central