Where is the Crawl stats report in Google Search Console?

Crawl stats is not in Search Console's left-hand menu. Open Settings (the gear icon), then Crawl stats, and open the report from there. It only exists for root-level properties: a Domain property such as example.com, or a URL-prefix property at the root such as https://example.com/. A property for a folder like https://example.com/blog/ has none. Google says sites with fewer than 1,000 pages rarely need it.

How to open it

1. Switch to a Domain property or a root URL-prefix property. If you only have a folder-level property, add a Domain property for the site.
2. Choose Settings at the bottom of the menu.
3. Find Crawl stats on the settings page and open the report.

What the report shows

The top chart plots three totals per day:

• Total crawl requests: every request Googlebot made, successful or not. Repeat requests for the same URL count each time, and so does every hop of a redirect chain.
• Total download size: bytes fetched. A resource cached and reused by several pages is fetched once.
• Average response time: across all fetched resources, each page resource counted separately.

Below it, host status checks three things over the past 90 days: whether robots.txt could be fetched, whether DNS resolved, and whether the server answered. Each shows a green mark, an issue more than a week ago, or a recent issue in the last week.

Then four breakdowns, each as a share of requests:

Google groups 404s as "possibly good": a removed page should return 404, and there is no need to fix every one.

How a robots.txt failure stops crawling

robots.txt matters more than anything else in the report. A 200 (even an empty file), 403, 404 or 410 all count as a successful fetch. A 429 or 5xx counts as a failure, and then:

1. For the first 12 hours, Google stops crawling the site and keeps asking for the file.
2. From 12 hours to 30 days, it uses the last good copy.
3. After 30 days, it crawls without restrictions if your homepage is reachable, and stops if not.

A red host status entry for robots.txt is therefore an emergency: it can halt all crawling.

Patterns that need action

• Requests fall while response time rises. Google lowers its crawl rate when a server slows down or returns errors. Fix the server before anything else.
• Googlebot is overloading you. Return 503 or 429 for a short time only, no more than two or three days, or crawling may stay lower afterwards.
• You want more crawling. You cannot request it. Keep sitemaps current, remove duplicate and unwanted URLs, and make sure important pages are not blocked.
• The report counts more requests than your logs. That can happen when robots.txt was unavailable for a long time, because Google counts the crawls it would have made.

Two limits to keep in mind: requests are listed by the URL actually fetched, not consolidated to canonical URLs, and resources hosted on other domains do not appear. There is also no API for this report, so your server logs remain the only per-URL record of Googlebot's visits. For whether any of this limits your indexing, see crawl budget.

How AutoPSEO helps

Google's API offers no crawl stats, so AutoPSEO cannot copy this report. What it does instead: the Index Tracker records the last crawl date Google reports for each URL you track, and flags indexed pages that have not been crawled for 120 days or more. With a Bing Webmaster Tools key connected, AutoPSEO also shows Bing's daily crawl stats and its list of crawl issues.

Sources: Google's Crawl stats report help and Search Console API reference.