How to Use Google Search Console to Diagnose Indexing Issues

Quick overview

Summary

• Review the Page Indexing report by page type and business value to separate intentional exclusions from important URLs that require investigation.
• Inspect individual pages, compare Google’s indexed information with the live test, and check crawling, HTTP responses, noindex , canonicals, discovery, and rendering.
• Fix the underlying website issue before requesting indexing or validating a group-level correction, then monitor recrawling, indexing, and performance separately.

When an important page disappears from Google, Search Console is usually the best place to begin —although it will not politely finish the investigation for you.

Google Search Console can show whether Google knows about a URL, whether it crawled the page, whether indexing is allowed, which canonical it selected, and why the page may have been excluded.

However, Search Console reports what Google observed . It does not automatically determine which issue matters, whether the exclusion is intentional, or exactly what change you should make.

To diagnose an indexing issue, use the Page Indexing report to identify patterns across the website, then inspect important URLs individually. Compare Google’s stored information with the live version and review crawl access, HTTP responses, indexing directives, canonical URLs, discovery sources, internal links, and rendered content. Fix the underlying issue before requesting indexing or starting validation.

What Is an Indexing Issue?

Google Search broadly processes pages through three stages:

• Crawling: Google discovers and downloads a page.
• Indexing: Google processes the page and may store information about it.
• Serving: Google may show the indexed page when it is relevant to a search.

An indexing issue exists when a page that should appear in Google is not added to the index, or when Google indexes a different URL from the one you intended.

Possible reasons include:

• Google has not discovered the page
• Google cannot crawl it
• The page contains noindex
• The URL redirects
• The page returns an error
• Google considers it a duplicate
• Google selected another canonical URL
• The page was crawled but not selected for indexing
• Internal discovery signals are weak
• Essential content is unavailable after rendering

For a detailed explanation of these causes, review why Google may not index a page .

Where Is the Page Indexing Report?

Open the relevant Search Console property and navigate to:

Indexing → Pages

The Page Indexing report provides an overall view of the URLs Google knows about.

It divides them into two broad groups:

• Indexed
• Not indexed

It also groups non-indexed URLs by reason.

Common categories include:

• Page with redirect
• Not found
• Excluded by noindex
• Blocked by robots.txt
• Duplicate without user-selected canonical
• Duplicate, Google chose a different canonical
• Crawled – currently not indexed
• Discovered – currently not indexed
• Server error
• Soft 404

These categories are diagnostic starting points. They are not all errors.

An old URL that correctly redirects should usually appear as Page with redirect. A deleted page may correctly return 404. A parameter duplicate may correctly remain outside the index.

The first question is not, “How do I remove every exclusion?”

The first question is, “Are the right URLs excluded?”

Should You Worry About the Total Number of Excluded Pages?

Not by itself.

Many websites contain URLs that should not be indexed independently, such as:

• Redirected URLs
• Filter and sorting parameters
• Tracking URLs
• Account pages
• Shopping cart pages
• Internal search results
• Duplicate category paths
• Thank-you pages
• Deleted content
• Non-canonical variations

A large number of excluded URLs does not automatically indicate a Technical SEO problem.

Search Console can make the number look dramatic. The number still needs context.

Instead, ask:

• Are important service, product, or category pages excluded?
• Did exclusions increase unexpectedly?
• Did the change begin after a redesign or migration?
• Is an entire template affected?
• Are URLs appearing under the wrong reason?
• Does the pattern match the intended website behaviour?

Interpret the report by page type, template, purpose, and business value.

How Do You Review an Indexing Reason?

Open one of the reasons listed under Why pages aren’t indexed.

Search Console will display information about the category and a sample of affected URLs.

Review the examples for shared patterns:

• The same template
• The same directory
• A particular product or page type
• A change beginning on a specific date
• Important URLs mixed with low-value URLs
• Recently migrated pages
• A shared canonical
• A shared robots directive
• A common redirect rule

A repeated pattern often points to a shared cause.

For example, if many service pages suddenly appear under Excluded by noindex , the likely problem may be a template or plugin setting rather than dozens of unrelated page edits.

If many URLs appear under Page with redirect, determine whether the redirects are intentional and whether they lead to the correct destinations.

The Page Indexing report identifies the category. It does not replace page-level investigation.

How Do You Inspect an Individual URL?

Enter the exact URL into the URL Inspection bar at the top of Search Console.

Depending on the page, Search Console may report:

• URL is on Google
• URL is not on Google
• URL is unknown to Google
• Page is not indexed
• Page is indexed but has issues

The inspection details may include:

• Last crawl date
• Crawler type
• Whether crawling was allowed
• Whether indexing was allowed
• Page fetch result
• Referring page
• Sitemap discovery
• User-declared canonical
• Google-selected canonical

These fields help establish where the process may have failed.

Always inspect the exact preferred URL. Small differences in protocol, hostname, trailing slash, capitalization, or parameters may represent different URLs.

What Is the Difference Between the Indexed Test and Live Test?

The initial URL Inspection result describes what Google knows about the indexed or previously processed version.

Test Live URL checks the page as it exists now.

The two results may differ.

For example:

• Google’s stored version may contain an old noindex
• The live page may now allow indexing
• Google may have encountered a previous server error
• The current server response may be successful
• The stored canonical may point elsewhere
• The live canonical may now be correct

A successful live test does not mean the page has already been indexed .

It means the current page may be technically accessible and eligible for processing. Google still needs to recrawl and evaluate the updated version.

This distinction is essential when confirming whether a fix has reached the live website.

How Do You Check Whether Crawling Is Allowed?

Review the crawl information in URL Inspection .

When Search Console says crawling is not allowed, inspect the website’s robots.txt file:

https://example.com/robots.txt

A rule such as:

User-agent: *
Disallow: /services/

would block compliant crawlers from requesting URLs under /services/.

Remember that robots.txt controls crawler access. It is not the same as noindex .

Blocking a page can prevent Google from viewing its current content and updated indexing directives.

Crawl access may also be affected by:

• Firewall rules
• Security plugins
• Hosting failures
• DNS errors
• Authentication requirements
• CDN settings
• Rate limiting
• Server errors

If the live test cannot fetch the page, the problem may exist in the website infrastructure rather than within Search Console.

What Should You Check in the Page Fetch Result?

A normal live page should generally return a successful 200 HTTP response.

Search Console may instead report:

• Redirect
• Not found
• Soft 404
• Server error
• Access denied
• Another fetch problem

A redirecting URL is generally not indexed independently. Google evaluates the destination.

A 404 or 410 response indicates that the page is unavailable.

A server error may prevent Google from accessing the page.

A soft 404 can occur when the server returns 200, but the page appears empty, missing, or too similar to an error page.

Review both the technical response and the visible page content.

How Do You Check Whether Indexing Is Allowed?

URL Inspection can report whether indexing is permitted.

When indexing is blocked, inspect the page for a robots meta directive:

<meta name="robots" content="noindex">

Also check whether an X-Robots-Tag header delivers the instruction.

Accidental noindex directives often appear after:

• A staging site goes live
• A redesign launches
• An SEO plugin is reconfigured
• A template is copied
• A CMS visibility option changes
• A temporary development restriction remains active

If the page should appear in search, remove the unintended directive.

Run the live test again and confirm that indexing is allowed before requesting another crawl.

NOT GETTING ENOUGH TRAFFIC OR LEADS?

Find what is holding your website back.

Get a focused SEO review that identifies the issues, missed opportunities, and priority fixes.

Request an SEO Review View Services

How Do You Compare Canonical URLs?

URL Inspection may display:

• User-declared canonical
• Google-selected canonical

The user-declared canonical is the preferred URL specified by the page.

The Google-selected canonical is the URL Google decided to treat as representative.

When you want the inspected page indexed independently, these URLs should normally agree.

If Google selects another URL, investigate whether:

• The page duplicates another URL
• The canonical tag points elsewhere
• Internal links use a different version
• The sitemap lists another URL
• HTTP and HTTPS versions conflict
• WWW and non-WWW versions conflict
• Parameters duplicate the content
• Multiple pages target the same intent
• The inspected page contains little unique value

A canonical mismatch is not always an error. Google may have correctly selected a stronger duplicate.

Determine whether it selected the URL you actually want users to find.

Canonical signals should be assessed together rather than in isolation:

• Canonical tags
• Redirects
• Internal links
• Sitemap inclusion
• Content similarity
• Protocol and hostname consistency

How Did Google Discover the Page?

URL Inspection may display discovery information such as:

• Referring page
• Sitemap
• No referring sitemap detected

This information can help you understand how Google found the URL, but it may not provide a complete list of every discovery source.

Use it as one piece of evidence.

If an important page has weak discovery signals:

• Add relevant internal links
• Include the canonical URL in the sitemap
• Link from a service, product, or category hub
• Confirm that navigation links are crawlable
• Avoid relying only on forms or internal search
• Avoid hiding essential links behind unsupported interactions

An important page should be part of the website’s navigable structure, not merely listed in a sitemap.

How Do You Review Rendered HTML and Resources?

JavaScript-heavy websites may deliver minimal content in the initial HTML and load the main page after rendering.

URL Inspection may provide access to:

• Rendered HTML
• A screenshot
• Loaded resources
• JavaScript messages

Check whether Google can see:

• Main page content
• Important internal links
• Product or service information
• Headings
• Canonical tags
• Robots directives
• Structured data
• Content loaded through JavaScript

A page can look perfectly normal in a browser while essential content quietly fails to appear in Google’s rendered version.

Compare the rendered result with what a normal visitor sees.

If important content, links, or directives are absent from rendered HTML, the problem may require development work rather than a Search Console configuration change.

How Should You Interpret Common Indexing Statuses?

Discovered – currently not indexed

Google knows about the URL but has not crawled it yet.

Review:

• Internal linking
• Sitemap inclusion
• Server reliability
• Excessive generated URLs
• Website size
• Whether the page is new

See the dedicated guide to Discovered – currently not indexed for internal-linking, sitemap, URL-proliferation, server, and crawl-prioritization checks.

Crawled – currently not indexed

Google crawled the page but did not add it to the index at that time.

Review:

• Content originality
• Page completeness
• Duplicate search intent
• Canonical signals
• Internal links
• Whether the page deserves an independent result

For a deeper investigation of content value, duplication, canonicalization, and internal signals, see how to fix Crawled – currently not indexed .

This status is not always solved through a technical change.

Duplicate, Google chose a different canonical

Google considers another URL the representative version.

Review:

• Canonical tags
• Internal links
• Sitemap entries
• Redirects
• Content similarity

Blocked by robots.txt

Google cannot crawl the page because of a robots rule.

Determine whether the restriction is intentional.

Excluded by noindex

Google detected a noindex instruction.

Remove it only when the page should appear in search.

Page with redirect

The inspected URL redirects elsewhere.

Confirm that the redirect is intentional and that the destination is correct.

Not found

The URL returns a missing-page response.

Restore it, redirect it, or leave it removed according to the intended outcome.

When Should You Request Indexing?

Request Indexing is useful after you have:

• Published a new page
• Removed an unintended noindex
• Corrected a canonical
• Fixed a server problem
• Restored missing content
• Improved the page
• Added internal links
• Corrected a redirect

It is not a universal repair button.

Requesting indexing does not override:

• robots.txt blocks
• noindex
• Redirects
• Error responses
• Duplicate content
• Canonical conflicts
• Limited page value

Run the live test first. Confirm that Google can access the current page and that the intended directives are present.

Then request indexing when appropriate.

When Should You Use Validate Fix?

For some grouped issues, the Page Indexing report provides a Validate Fix option.

Use this after correcting a shared issue affecting multiple URLs.

Examples include:

• A template-level noindex was removed
• A server problem was resolved
• A redirect rule was corrected
• A blocked directory was made crawlable
• A shared template error was fixed

Validation tells Google that the issue has been addressed and asks it to check affected URLs again.

Do not begin validation before the correction is fully implemented.

When several example URLs still fail, validation may not complete successfully.

For one updated page, URL Inspection and Request Indexing may be more appropriate.

How Do You Monitor the Result?

Search Console does not always update immediately.

After implementing a change:

• Test the live URL.
• Confirm the technical correction.
• Request indexing or start validation when appropriate.
• Allow Google time to recrawl.
• Inspect the URL again later.
• Monitor the relevant Page Indexing category.
• Check whether impressions begin appearing in the Performance report.

Indexing and ranking are different outcomes .

A page can be indexed without earning impressions or clicks. Visibility also depends on relevance, content quality, competition, search demand, links, and other signals.

First confirm that the page is technically accessible and indexable. Evaluate search performance separately.

Page-level review

Google Search Console Indexing Checklist

Page Indexing report

• Which category contains the URL?
• Are similar pages affected?
• Is the exclusion intentional?
• Did it begin after a site change?
• Is a shared template involved?

URL Inspection

• Is the URL indexed?
• When was it last crawled?
• Is crawling allowed?
• Is indexing allowed?
• Did Google fetch the page successfully?
• What is the user-declared canonical?
• What is the Google-selected canonical?
• Was the URL found through a sitemap or referring page?

Live test

• Does the current page pass?
• Does it return 200?
• Has an unintended noindex been removed?
• Is the intended canonical present?
• Can Google render the main content?
• Are important resources available?

Website checks

• Does the page receive internal links?
• Is the canonical URL in the sitemap?
• Is the content original and complete?
• Does the page deserve an independent result?
• Are redirects and duplicates controlled?

When Search Console reveals related problems across crawling, canonicalization, sitemaps, redirects, and site architecture, use a broader framework to find and fix Technical SEO issues across the website.

When Should You Get Professional Search Console Support?

Search Console may identify the category without revealing the full cause.

Template-level directives, canonical conflicts, rendering failures, migration problems, and generated URL patterns often require evidence from the live site, a crawler, server responses, and page-level comparisons.

When important URLs remain excluded or the diagnosis affects many pages, Google Search Console and indexing recovery can connect Search Console data with live technical evidence and produce a prioritized correction and validation plan.

Common questions

Frequently Asked Questions About Google Search Console Indexing Issues

How do I use Google Search Console to find out why a page is not indexed?

Start with the Page Indexing report to identify the exclusion category and determine whether similar URLs are affected. Then inspect the exact page with URL Inspection and review crawl access, page fetch, indexing permission, canonical information, discovery sources, and the live URL result before deciding what needs to be fixed.

What is the difference between the Page Indexing report and URL Inspection?

The Page Indexing report provides a site-level view of indexed and non-indexed URLs grouped by reason. URL Inspection provides page-level evidence for one exact URL, including Google’s indexed data, the last crawl, page fetch, indexing permission, canonical selection, discovery information, and the current live test.

Why does the live URL test pass when the page is still not indexed?

A successful live test means Google can currently access the page and that the URL may be technically eligible for indexing. It does not mean Google has already recrawled, processed, or selected the page for the index. The live test also does not evaluate every condition, including all duplicate-content and canonical-selection issues.

What does “URL is not on Google” mean?

“URL is not on Google” means the inspected URL is not currently included in Google’s index. The page may be undiscovered, blocked, redirected, marked noindex , treated as a duplicate, affected by an error, or not selected for indexing. Review the detailed inspection fields and live result to identify the relevant condition.

How can I check which canonical URL Google selected?

Open URL Inspection for the exact page and review the User-declared canonical and Google-selected canonical fields. If they differ, compare the page’s canonical tag, internal links, sitemap entry, redirects, URL variations, and content similarity to determine why Google selected another URL as the representative version.

When should I use Validate Fix instead of Request Indexing?

Use Validate Fix after correcting a shared issue affecting a group of URLs in the Page Indexing report, such as a template-level noindex , server error, or blocked directory. Use Request Indexing for an individual new or corrected page after confirming with the live test that the underlying issue has been resolved.

Final Thoughts

Google Search Console is most useful when treated as a diagnostic system rather than an automated fix.

Start with the Page Indexing report to identify patterns. Then use URL Inspection to investigate priority pages individually.

Compare Google’s stored information with the live page. Review crawl access, HTTP responses, indexing directives, canonical signals, discovery paths, internal links, and rendered content.

Most importantly, determine whether the excluded URL should actually be indexed.

The objective is not to force every URL into Google. It is to ensure that important, useful, canonical pages can be discovered, crawled, processed, and considered for indexing.

Continue reading

Related Technical SEO articles

Technical SEO

How to Find and Fix Technical SEO Issues on Your Website

A practical framework for diagnosing, prioritizing, fixing, and validating technical SEO problems.

Read the article →

Google Indexing

Why Is Google Not Indexing My Pages?

Identify discovery, crawling, noindex, canonical, sitemap, content, and internal-linking causes.

Read the article →

XML Sitemaps

XML Sitemap Submitted but Pages Are Not Indexed

Understand what sitemap Success means and diagnose why submitted URLs remain unindexed.

Read the article →