Six ways for a page to be unindexable, and most tools check two
Free, and it shows you its reasoning instead of just a verdict.
Can Google index this page? It sounds like a yes-or-no question and it has about six independent ways to come out as no. robots.txt, the meta robots tag, the X-Robots-Tag header, a canonical pointing somewhere else, the HTTP status, and whether the page is in your sitemap at all.
The part other tools get wrong is robots.txt precedence, which is genuinely fiddly. Longest matching rule wins, Allow breaks a tie, wildcards and end-of-string anchors both count, and an exact user-agent match means the star group stops applying to you entirely. Get that wrong and you tell someone their page is blocked when it is not.
So this shows you the exact line from your robots.txt that decided the outcome. You can check my reasoning rather than trust my verdict, which matters because I might be the one who is wrong.
How to use it
- Open the page you are asking about.
- Click the icon. It reads the page signals and fetches robots.txt and your sitemap from that site.
- Read the verdict, then read the reason under it.
- If the answer is no, the rule that caused it is quoted directly so you know what to edit.
The canonical trap
A page can be perfectly crawlable, return a clean 200, sit in your sitemap, and still never appear in search, because its canonical tag points at a different URL. You have told Google to index the other one instead.
This happens constantly with parameters and pagination, and it is easy to miss because nothing about it looks broken. So the canonical target is shown plainly, and if it points somewhere other than the page you are on, that is called out rather than listed as one line of nine.
What it can access, and why
| Permission | Why it needs it |
|---|---|
| Reads the current tab | Only when you click the icon, to read the meta robots tag and the canonical. |
| Fetches robots.txt and your sitemap (optional) | Those two files decide the answer, and they live at the site root rather than on the page. Requested when you run a check. |
Full detail in the privacy policy.
FAQ
Does blocked in robots.txt mean the page will not appear in search?
Not quite, and the difference matters. robots.txt stops crawling, not indexing. A URL that is linked from elsewhere can still show up as a bare result with no description, because Google knows it exists but was told not to look. If you actually want a page out of the index, a noindex tag is the tool, and that requires the page to be crawlable so the tag can be read.
Why does it say my rule does not apply to me?
You probably have a group for a specific user-agent as well as a star group. Once a crawler finds a group naming it exactly, the star group is ignored completely, including rules you assumed were global. That surprises people every time and it is exactly the case this was built to surface.
Does it check the X-Robots-Tag header?
Yes. It is a common blind spot because you cannot see it in the page source, and a noindex delivered by header behaves identically to one in a meta tag. Plenty of CDN and framework configurations set it without anybody noticing.
Will this tell me why a page is not ranking?
No, and it is worth being clear about the boundary. This answers whether a page is eligible to be indexed. Ranking is a completely different question about competition and relevance, and no extension can answer it.