Indexability checkers: tools that show why a page can't be indexed

The indexability checkers worth using are LaunchScaler's free scan, Google Search Console's URL Inspection, Bing Webmaster Tools' URL Inspection, Google's Rich Results Test, and the desktop crawlers Screaming Frog SEO Spider and Sitebulb. They answer different questions: some test whether a page can be indexed, only the search engines' own tools say whether it was, and only crawlers cover a whole site.

They are ranked here for a founder who wants to find out why a page isn't indexable, on any site, without installing anything. Every price and limit was checked on each tool's own pages in September 2026.

What is the difference between indexability and index status?

Indexability is whether a page can be indexed: robots.txt allows crawling, the page returns 200, there's no noindex in the meta tag or X-Robots-Tag header, and its canonical points at itself. Index status is whether Google or Bing actually indexed it. A page can be fully indexable and still not indexed, because the engine chose not to.

The distinction decides which tool you need:

A clean indexability result removes the technical causes. If the page still isn't indexed, the cause is a decision the engine made, such as duplication or quality, and the guide to why some pages aren't indexed walks through that branch.

How do indexability checkers compare?

They differ on three things: whose sites you can test, which directives they read, and whether they report the engine's own index status. The table compares the six on those points.

Which indexability checkers are best?

For most founders the order is LaunchScaler, Search Console URL Inspection, Bing URL Inspection, the Rich Results Test, Screaming Frog, then Sitebulb. LaunchScaler is first because it is the only one that checks every indexability directive on any URL in one pass, with no account and nothing to install. The rest follow from most authoritative for your own site to broadest for a whole-site crawl.

1. LaunchScaler: every indexability directive on any URL, free

LaunchScaler's readiness scan takes any live URL, needs no account, and runs 156 checks across 6 of its 7 categories at no cost: search, AI visibility, security, compliance, speed and visual, and whether the site works. Its search checks cover the indexability directives in one pass.

What it checks for indexability:

• robots.txt rules that block a page you want ranked, read per crawler, including the AI retrieval bots OAI-SearchBot, Claude-SearchBot and PerplexityBot
• noindex in the meta robots tag or the X-Robots-Tag header
• a noindex hidden behind a robots.txt block, which Google can never read
• a missing canonical, or one that points at a different URL, a redirect, a 404 or a noindexed page
• soft 404s: error or empty pages that return 200
• redirect chains that take several hops to reach the final page
• main content that only appears after JavaScript runs
• invalid HTML in the <head> that drops the canonical and robots tags below it
• a missing sitemap, or one that lists redirecting, noindexed or non-canonical URLs

What it can't tell you is whether Google or Bing indexed the page; only their own tools know that. Use it to clear the technical causes, on your site or a competitor's, and then use URL Inspection for the index status.

2. Google Search Console URL Inspection: authoritative, for sites you've verified

URL Inspection is free and is the only tool that reports Google's own index status: whether the URL is on Google, which canonical Google selected, when it was last crawled, and why it wasn't indexed. The catch is scope. The URL must be in a property you have verified, and there is a daily limit on inspections per property.

Its Page indexing section shows "Crawl allowed?", "Page fetch" and "Indexing allowed?", and names the source of a noindex, meta tag or header. Test live URL fetches the current page and, under View tested page, shows the rendered HTML, a screenshot and the HTTP response headers. Canonical choice is only visible in the indexed result; the live test cannot predict it.

One trap to know: when robots.txt blocks a page, "Indexing allowed?" always shows Yes, because Google can't see a noindex behind the block. The URL is not on Google guide reads every line of the panel, and the guide to checking if a page is indexed compares it with a site: search.

3. Bing Webmaster Tools URL Inspection: the same question for Bing and Copilot

Bing's URL Inspection reports whether a URL is in Bing's index, when and where it was discovered, crawled and indexed, with the HTML and HTTP response, plus SEO and markup cards. A Live URL tab shows what Bingbot sees now. It matters beyond Bing search: Bing's guidelines say Bing and Copilot rely on the same crawling and indexing foundation.

It is free and limited to sites you have added and verified in Bing Webmaster Tools. Requesting indexing from the tool draws on your daily URL submission quota. Its Live URL test does not follow redirects; it reports the redirect and its destination, which you then inspect separately.

4. Google Rich Results Test: a crawl check for any public URL

The Rich Results Test is built for structured data, but it fetches any public URL as Google-InspectionTool and reports "Crawl allowed?" and "Page fetch", with the rendered source. That makes it a free way to check whether Google can crawl a page on a site you don't own.

Its limits are the reason it ranks below the tools above. It doesn't report noindex or canonicals; Google's help says a page blocked by robots.txt or noindex simply can't be tested with it. It tests one URL at a time, and it says nothing about index status.

5. Screaming Frog SEO Spider: a desktop crawler for whole-site audits

Screaming Frog's SEO Spider crawls a whole site from your computer and lists every URL blocked by robots.txt, meta robots or X-Robots-Tag, along with canonicals, redirect chains and response codes. It can render pages with its integrated Chromium to crawl JavaScript-heavy sites.

It is free for crawls of up to 500 URLs. A licence removes the limit and adds advanced features; when checked in September 2026, its pricing page listed $279 per licence per year for 1 to 4 licences, with lower per-licence prices for 5 or more, and licences last one year. It reports indexability, not index status, so pair it with Search Console for a site you own.

6. Sitebulb: a desktop crawler with prioritised hints

Sitebulb is also desktop crawling software, for Windows and Mac. Its Indexability Hints cover robots.txt, meta robots, X-Robots-Tag and canonical tags, and it says JavaScript crawling comes at no extra cost. It adds prioritised hints and visual reports on top of the raw crawl data.

Its pricing page lists a Lite plan limited to 10,000 URLs per audit and a Pro plan with a default limit of 500,000 URLs per audit, which can be raised to 2 million on a capable machine, plus a Sitebulb Cloud option. There is a 14-day free trial with no credit card. The desktop plan prices did not appear in the page text we could read, so check them on Sitebulb's pricing page.

Which indexability checker should you use?

Match the tool to the question. For one page on any site, start with a tool that fetches the page itself. For your own site's index status, use the search engines' tools. For a whole site with thousands of URLs, use a crawler.

Whichever you use, confirm fixes where they count: after changing a directive, run Test live URL in Search Console on one affected page before requesting indexing, because Google only reads a noindex or canonical when it next crawls the page.

Check a page's indexability now

Run the free scan on the page that isn't indexing. It takes the address alone, needs no account, and reads the robots.txt rules, both forms of noindex, the canonical and its target, redirects, soft 404s and JavaScript-only content in one pass, so you know which directive to fix before you ask Google to crawl again.