Search Console Error and Coverage Report Descriptions

In this page, we will go over:

• What is Crawling?
• What is Indexing?
• Reading the Coverage Report
  • Report Elements
    • Status
    • Reason
  • Error Types
    • Errors
    • Warning
    • Valid
    • Excluded
• Mobile Usability Report Errors
• How to Fix these Issues

Search Console often sends emails like the ones below (actually issue can differ). This is based on the Coverage report and Mobile Usability Report in Google Search Console (example below). These errors are fairly common, some can be no big deal, some are false errors, some need additional analysis, and some need to be addressed immediately.

These issues have to do with the crawling and indexation of your site.

What is Crawling?

Google finds the pages of your site by crawling them. In the SEO world, crawling means following your links and “crawling” around your website. When bots come to your website (any page), they follow other linked pages also on your website.

This is one reason why we create sitemaps, as they contain all of the links in our blog and Google’s bots can use them to look deeply into a website.

The way we stop crawling certain parts of our site is by using the Robots.txt file.

What is Indexing?

Indexing is the process of adding web pages into Google search.

Depending upon which meta tag is used (index or NO-index), Google will crawl and index the sites pages. A no-index tag means that that page will not be added to the web search’s index and will not show on the Search Engine Result Page.

Reading the Coverage Report

Report Elements

The top-level report shows the index status of all pages that Google has attempted to crawl on your site, grouped by status and reason.

Example:

Status

Each page has one of the following status’ that is dependent on if the page was indexed in Google or not.

Indexation means Google has crawled this page, added it to its index and theoretically the page shows up in the Search Engine Results page.

• Error: The page has not been indexed. See the specific error type description below to learn more, and how to fix the error. You should concentrate on these issues first.
• Warning: Page is indexed, or was until recently, and has an issue that you should be aware of.
• Excluded: The page is not included in the index for reasons that you typically cannot affect. The page might be in an intermediate stage of the indexing process, or is deliberately excluded by you (for example adding a noindex tag) and is therefore behaving as expected.
• Valid: The page was indexed. (does not mean this page doesn’t have issues, just means that it was indexed by Google)

Reason

Each of the status’ (valid, warning, error, excluded) has a specific reason for showing up. Data in the table are grouped by reason; each row can describe one or more URLs. (See Status type descriptions below for a description of each status type, and how to handle it)

Validation

The status of a user-initiated validation flow for this issue. You should prioritize issues that are failed or not started.

Error Types

Errors

Pages with errors have not been indexed.

• Server error (5xx): Your server returned a 500-level error when the page was requested. Sometimes these occur when the site server was down at the time Google crawled the page, make sure all of the pages and working or need to be fixed. See Fixing server errors.
• Redirect error: The URL was a redirect error. Could be one of the following types: it was a redirect chain that was too long; it was a redirect loop; the redirect URL eventually exceeded the max URL length; there was a bad or empty URL in the redirect chain.
• Submitted URL blocked by robots.txt: You submitted this page for indexing, but the page is blocked by robots.txt. Try testing your page using the robots.txt tester to see if the page is blocked by your robots.txt file.
• Submitted URL marked ‘noindex’: You submitted this page for indexing, but the page has a ‘noindex’ tag either in a meta tag or HTTP header. If you want this page to be indexed, you must remove the noindex tag.
• Submitted URL seems to be a Soft 404: You submitted this page for indexing, but the server returned what seems to be a soft 404. These can be false positives if the page is working properly, Google properly just made a mistake when crawling it. If this page is not supposed to exist on the site, try redirecting it.
• Submitted URL returns unauthorized request (401): You submitted this page for indexing, but Google got a 401 (not authorized) response. Either remove authorization requirements for this page, or else allow Googlebot to access your pages by verifying its identity.
• Submitted URL not found (404): You submitted a non-existent URL to Google for indexing. See Fixing 404 errors.
• Submitted URL has crawl issue: You submitted this page for indexing, and Google encountered an unspecified crawling error that doesn’t fall into any of the other reasons. Try debugging your page using the URL Inspection tool.

Warning

Pages with a warning status might require your attention, and may or may not have been indexed, according to the specific result.

• Indexed, though blocked by robots.txt: The page was indexed, despite being blocked by robots.txt (Google always respects robots.txt, but this doesn’t help if someone else links to it). This is marked as a warning because we’re not sure if you intended to block the page from search results. If you do want to block this page, robots.txt is not the correct mechanism to avoid being indexed. To avoid being indexed you should either use ‘noindex’ or prohibit anonymous access to the page using auth. You can use the robots.txt tester to determine which rule is blocking this page. Because of the robots.txt, any snippet shown for the page will probably be sub-optimal. If you do not want to block this page, update your robots.txt file to unblock your page.

Valid

Pages with a valid status have been indexed.

• Submitted and indexed: You submitted the URL for indexing, and it was indexed.
• Indexed, not submitted in sitemap: The URL was discovered by Google and indexed. We recommend submitting all important URLs using a sitemap.
• Indexed; consider marking as canonical: The URL was indexed. It has duplicate URLs, but we consider this one to be canonical. It is not explicitly marked as canonical, and so we recommend explicitly marking it as canonical.

Excluded

These pages are typically not indexed, and we think that was your intention.

• Excluded by ‘noindex’ tag: When Google tried to index the page it encountered a ‘noindex’ directive and therefore did not index it. If you do not want this page indexed, congratulations! If you do want this page to be indexed, you should remove that ‘noindex’ directive.
• Blocked by page removal tool: The page is currently blocked by a URL removal request. If you are a verified site owner, you can use the URL removals tool to see who submitted a URL removal request. Removal requests are only good for a specified period of time (see the linked documentation). After that period, Googlebot may go back and index the page even if you do not submit another index request. If you don’t want the page indexed, use ‘noindex’, require authorization for the page, or remove the page.
• Blocked by robots.txt: This page was blocked to Googlebot with a robots.txt file. You can verify this using the robots.txt tester. Note that this does not mean that the page won’t be indexed through some other means. If Google can find other information about this page without loading it, the page could still be indexed (though this is less common). To ensure that a page is not indexed by Google, remove the robots.txt block and use a ‘noindex’ directive.
• Blocked due to unauthorized request (401): The page was blocked to Googlebot by a request for authorization (401 response). If you do want Googlebot to be able to crawl this page, either remove authorization requirements, or allow Googlebot to access your page.
• Crawl anomaly: An unspecified anomaly occurred when fetching this URL. This could mean a 4xx- or 5xx-level response code; try fetching the page using Fetch as Google to see if it encounters any fetch issues. The page was not indexed. If the pages are working, validate the fix.
• Crawled – currently not indexed: The page was crawled by Google, but not indexed. It may or may not be indexed in the future; no need to resubmit this URL for crawling.
• Discovered – currently not indexed: The page was found by Google, but not crawled yet. Typically, Google tried to crawl the URL but the site was overloaded; therefore Google had to reschedule the crawl. This is why the last crawl date is empty on the report.
• Alternate page with proper canonical tag: This page is a duplicate of a page that Google recognizes as canonical. This page correctly points to the canonical page, so there is nothing for you to do.
• Duplicate without user-selected canonical: This page has duplicates, none of which is marked canonical. We think this page is not the canonical one. You should explicitly mark the canonical for this page. Inspecting this URL should show the Google-selected canonical URL.
• Duplicate non-HTML page: This non-HTML page (for example, a PDF file) is a duplicate of another page that Google has marked as canonical. Typically only the canonical URL will be shown in Google Search. If you like, you can specify a canonical page using the Link HTTP header in a response.
• Duplicate, Google chose different canonical than user: This page is marked as canonical for a set of pages, but Google thinks another URL makes a better canonical. Google has indexed the page that we consider canonical rather than this one. We recommend that you explicitly mark this page as a duplicate of the canonical URL. This page was discovered without an explicit crawl request. Inspecting this URL should show the Google-selected canonical URL.
• Not found (404): This page returned a 404 error when requested. Google discovered this URL without any explicit request or sitemap. Google might have discovered the URL as a link from another site, or possibly the page existed before and was deleted. Googlebot will probably continue to try this URL for some period of time; there is no way to tell Googlebot to permanently forget a URL, although it will crawl it less and less often. 404 responses are not a problem, if intentional. If your page has moved, use a 301 redirect to the new location. Read Fixing 404 errors.
• Page removed because of legal complaint: The page was removed from the index because of a legal complaint.
• Page with redirect: The URL is a redirect, and therefore was not added to the index. Queued for crawling: The page is in the crawling queue; check back in a few days to see if it has been crawled.
• Soft 404: The page request returns what we think is a soft 404 response. This means that it returns a user-friendly “not found” message without a corresponding 404 response code. Google recommends returning a 404 response code for truly “not found” pages, or adding more information to the page to let us know that it is not a soft 404. Learn more.
• Submitted URL dropped: You submitted this page for indexing, but it was dropped from the index for an unspecified reason. Evaluate the page and make sure it should be indexed.
• Duplicate, submitted URL not selected as canonical: The URL is one of a set of duplicate URLs without an explicitly marked canonical page. You explicitly asked this URL to be indexed, but because it is a duplicate, and Google thinks that another URL is a better candidate for canonical, Google did not index this URL. Instead, we indexed the canonical that we selected. (Google only indexes the canonical in a set of duplicates.) The difference between this status and “Google chose different canonical than user” is that here you have explicitly requested indexing. Inspecting this URL should show the Google-selected canonical URL.

Mobile Usability Report Errors

The Mobile Usability report shows which pages in your property have usability problems when viewed on mobile devices. Many of these issues include “content wider than screen”, “clickable elements too close together”, etc.

The following information is shown in the report:

Status: A page has two possible states. Error: The page is not mobile friendly Valid: The page is mobile friendly. Pages: The count of pages in Error state with this issue.

Errors

The following errors can appear in the Mobile Usability report, note that a lot of these are false positives or there may be nothing we can do.

• Uses incompatible plugins: The page includes plugins, such as Flash, that are not supported by most mobile browsers We recommend redesigning your page using modern, broadly-supported web technologies, such as HTML5. Read more about web animation guidelines.
• Viewport not set: Your page does not define a viewport property, which tells browsers how to adjust the page’s dimension and scaling to suit the screen size. Because visitors to your site use a variety of devices with varying screen sizes—from large desktop monitors, to tablets and small smartphones—your pages should specify a viewport using the meta viewport tag. Learn more in Responsive Web Design Basics.
• Viewport not set to “device-width”: Your page defines a fixed-width viewport property, which means that it can’t adjust for different screen sizes. To fix this error, adopt a responsive design for your site’s pages, and set the viewport to match the device’s width and scale accordingly. Read how to correctly Set the Viewport.
• Content wider than screen: Sometimes this shows up when there are too many links in one sentence, a lot of the times it is a false positive. This report indicates pages where horizontal scrolling is necessary to see words and images on the page. This happens when pages use absolute values in CSS declarations, or use images designed to look best at a specific browser width (such as 980px). To fix this error, make sure the pages use relative width and position values for CSS elements, and make sure images can scale as well. Read more in Size Content to Viewport.
• Text too small to read: This report identifies pages where the font size for the page is too small to be legible and would require mobile visitors to “pinch to zoom” in order to read. After specifying a viewport for your web pages, set your font sizes to scale properly within the viewport. Read more about font size best practices in Use Legible Font Sizes.
• Clickable elements too close together: This report shows the URLs for sites where touch elements, such as buttons and navigational links, are so close to each other that a mobile user cannot easily tap the desired element with their finger without also tapping a neighboring element. To fix these errors, make sure to correctly size and space buttons and navigational links to be suitable for your mobile visitors. Read more in Size Tap Targets Appropriately. Test the site out on your mobile device to get a better idea of where these issues might be happening!

How to Fix these Issues

If your site has a lot of the issues listed above, or a client has specifically asked you to look into one of these issues after receiving an email, try to read this document through and if you need more clarification, submit it as a “one-off” project in Airtable, assign it to the SEO team and they will look into it and then submit the issue for validation which indicates to Google the issue has been fixed.

Many of these issues are simple fixes or nothing to worry about unless most pages on the client’s site are experiencing them!