Six ways for a page to be unindexable, and most tools check two
Free, and it shows you its reasoning instead of just a verdict.
Can Google index this page? It sounds like a yes-or-no question and it has about six independent ways to come out as no. robots.txt, the meta robots tag, the X-Robots-Tag header, a canonical pointing somewhere else, the HTTP status, and whether the page is in your sitemap at all.
The part other tools get wrong is robots.txt precedence, which is genuinely fiddly. Longest matching rule wins, Allow breaks a tie, wildcards and end-of-string anchors both count, and an exact user-agent match means the star group stops applying to you entirely. Get that wrong and you tell someone their page is blocked when it is not.
So this shows you the exact line from your robots.txt that decided the outcome. You can check my reasoning rather than trust my verdict, which matters because I might be the one who is wrong.
The short version
Robots.txt and Index Checker runs all six checks at once and shows you which one made the decision.
- Ask one question, can Google index this page, and get all six reasons it might not checked at once.
- Read the actual robots.txt rule that matched, with the reasoning shown, instead of a red cross.
- Catch the canonical pointing somewhere you did not intend, which is the one that silently costs you the page.
Best for: anyone whose page has been missing from search for two weeks with no explanation.
Replaces: checking robots.txt in one tab, the meta tag in view-source, and the headers in devtools.
How to use it
- Open the page you are asking about.
- Click the icon. It reads the page signals and fetches robots.txt and your sitemap from that site.
- Read the verdict, then read the reason under it.
- If the answer is no, the rule that caused it is quoted directly so you know what to edit.
The canonical trap
A page can be perfectly crawlable, return a clean 200, sit in your sitemap, and still never appear in search, because its canonical tag points at a different URL. You have told Google to index the other one instead.
This happens constantly with parameters and pagination, and it is easy to miss because nothing about it looks broken. So the canonical target is shown plainly, and if it points somewhere other than the page you are on, that is called out rather than listed as one line of nine.
What it can access, and why
| Permission | Why it needs it |
|---|---|
| Reads the current tab | Only when you click the icon, to read the meta robots tag and the canonical. |
| Fetches robots.txt and your sitemap (optional) | Those two files decide the answer, and they live at the site root rather than on the page. Requested when you run a check. |
Full detail in the privacy policy.
The things people ask before installing
Why not just use Search Console? Use both. Search Console tells you what Google decided last time it looked, which can be weeks ago. This tells you what the page says right now, on any site, including one you do not own.
Why does it need to fetch anything? robots.txt and the X-Robots-Tag header live outside the page, so there is no way to read them without a request. It asks for that permission at the point it needs it, not at install.
Is it free? Yes, permanently. No tier above this one exists.
Free, and it stays free. No account, no sign-up, and nothing leaves your browser.
FAQ
Does blocked in robots.txt mean the page will not appear in search?
Not quite, and the difference matters. robots.txt stops crawling, not indexing. A URL that is linked from elsewhere can still show up as a bare result with no description, because Google knows it exists but was told not to look. If you actually want a page out of the index, a noindex tag is the tool, and that requires the page to be crawlable so the tag can be read.
Why does it say my rule does not apply to me?
You probably have a group for a specific user-agent as well as a star group. Once a crawler finds a group naming it exactly, the star group is ignored completely, including rules you assumed were global. That surprises people every time and it is exactly the case this was built to surface.
Does it check the X-Robots-Tag header?
Yes. It is a common blind spot because you cannot see it in the page source, and a noindex delivered by header behaves identically to one in a meta tag. Plenty of CDN and framework configurations set it without anybody noticing.
Will this tell me why a page is not ranking?
No, and it is worth being clear about the boundary. This answers whether a page is eligible to be indexed. Ranking is a completely different question about competition and relevance, and no extension can answer it.