Crawl Errors
Crawl errors are pages the search engine reached but could not successfully retrieve or process. HTTP error status codes, inaccessible pages, broken pagination chains, and blocked resources all prevent engines from fully crawling a site. These eight checks identify the specific crawl errors that waste crawl budget and reduce the number of pages engines can discover and index.
Pages returning 4xx status — browsers and crawlers cannot retrieve the content.
Pages returning 5xx status — server-side failures preventing crawling and indexing.
The crawler could not fetch the page at all — network failure or server connection refused.
A gap in the rel=next/prev sequence — crawlers cannot follow the full paginated series.
rel=next creates a circular chain — page X eventually points back to itself.
The first page in a paginated series has a rel=prev — an impossible reference.
A paginated URL exists but is not linked from the main page body — discoverable only via rel=next.
An internal resource (CSS, JS, image) is blocked by robots.txt or access controls.
Audit every crawl erroracross your site
Run a full audit to find every crawl error, with affected URLs grouped by error type so you can fix the root causes systematically.