How do I check my robots.txt in Google Search Console?

Q: How do I check my robots.txt in Google Search Console?

Open the robots.txt report under Settings to see each file Google found, when it was fetched and any parse errors. Check single URLs with URL Inspection.

Open Settings and then the robots.txt report. It lists the robots.txt files Google found for your site, when each was last checked, whether the fetch worked, the file size and any lines Google couldn't parse. To find out whether one particular URL is blocked, inspect that URL and read the Crawl allowed? field. The standalone robots.txt Tester that older guides describe is no longer one of Search Console's tools.

What the report shows

According to Google's robots.txt report help:

It is only available for domain-level properties: a Domain property, or a URL-prefix property without a path (https://example.com/, not https://example.com/shop/).
On a Domain property it covers the top 20 hosts by crawl rate, since each host has its own file.
For each file you see its path, fetch status, when it was last checked, its size in bytes and the number of issues. Errors make Google skip a rule; warnings don't.
Files appear if Google fetched them, or tried to, in the last 30 days.

Status	What it means
Fetched	Google got the file. It ignores lines it couldn't parse and uses the rest
Not fetched: Not found (404)	There is no file, which is fine: Google may crawl every URL
Not fetched: any other reason	Something else went wrong; see the table below

After you change the file

Google generally caches robots.txt for up to 24 hours, so a change isn't instant. If you have unblocked important URLs or fixed a fetch error, open the file's menu in the report and choose Request a recrawl. Otherwise there's no need: Google refetches robots.txt often.

How Google handles a broken robots.txt

What Google receives	What happens
A 2xx response	The rules are applied
A redirect	Google follows at least five hops, then treats an unresolved chain as a 404
A 4xx error other than 429	Treated as if there were no file: nothing is blocked
A 5xx error or network failure	Crawling stops for 12 hours, then the last good copy is used for up to 30 days; after that, if the site is otherwise available, Google acts as if there were no file
A file over 500 KiB	Everything past the limit is ignored

Source: How Google interprets robots.txt. The 5xx row is the one to fear: a server that errors on this one file can pause crawling of the whole site.

Testing rules before they go live

For a URL that is already live, URL Inspection's Crawl allowed? field is the quickest answer.
For a draft file, Google points developers to its open-source robots.txt library for testing locally.
When rules conflict, Google applies the most specific one, the rule with the longest matching path, and if two are equally specific, the less restrictive one wins.

What robots.txt can't do

robots.txt controls crawling, not indexing. Google says it isn't a way to keep a page out of Google: a blocked URL can still be indexed, without a description, if other pages link to it. To keep a page out of results, use noindex or a password, as described in removing a URL.

How AutoPSEO helps

AutoPSEO's bulk URL inspection shows the robots.txt state Google reports for each URL, for up to 500 URLs per run from a pasted list or a sitemap. That makes it quick to confirm a rule change didn't block a whole section by accident.