The number of URLs a search engine will fetch from a site in a given period. Wasting it on duplicates, redirect chains or broken pages delays the discovery of content that matters. It is a real constraint mainly for large sites, where internal links and a clean URL structure help crawlers reach the pages that count.
Several things spend it badly: duplicate URLs without a canonical, redirect chains that take several hops to reach the final page, broken links, and orphan pages that no internal link points to, which a messy site structure tends to create.
robots.txt is the first file a crawler reads, and blocking folders that do not need to be indexed leaves more of each visit for the ones that do. It controls crawling, not indexing, so a blocked URL can still be indexed without its content if other pages link to it.
Example
An online shop's filters create a new URL for every combination of size, colour and price, so crawlers spend their visits on near-identical listings. Blocking the filter parameters in robots.txt leaves more of each crawl for new product pages.
Related terms
Where seodada covers it
Site Audit crawls up to 200 pages, renders JavaScript and sorts what it finds into errors, warnings and notices, with a page-by-page table of status codes and internal links. The technical SEO guide covers site structure and robots.txt.
