Crawl-to-index lag

Definition

Crawl-to-index lag is the interval that starts when Googlebot first fetches a URL and ends when the URL is first observed in Google's index. It isolates the part of indexing that happens inside Google after the indexer has done its job. A tool that reports fast crawls but long lag is delivering discovery; whether Google keeps the page is a separate question.

The measure pairs naturally with time to first crawl. Together they split total time to index into a part the indexer influences and a part it does not:

time to index = time to first crawl + crawl-to-index lag

Typical values

IndexChex documentation for standard submissions describes the lag in two steps: indexing begins 5 to 12 hours after the job completes, and most URLs that will be indexed appear 24 to 48 hours after the crawl. Because the average first crawl arrives in under 2 minutes in IndexChex's weekly measurements on recurring URL sets (October 2026), almost all of the elapsed time for a standard job is lag rather than crawl delay.

Google's URL Inspection help says indexing "typically takes only a day or so, but can take much longer in some cases." That sentence refers to pages submitted by their owners, but it lands in the same range and confirms that a lag of hours to a day is normal, not a sign of failure.

What happens during the lag

Google does not document a fixed pipeline length, and this page does not claim one. What Google does describe in its public documentation is that after a fetch, pages may be rendered, duplicates are grouped and a canonical is chosen, and the page is evaluated before it is stored. Each of these can add delay. Practical factors that tend to lengthen the lag:

• The host site is slow, so the fetch is partial or retried.
• The page is a near duplicate of another URL and waits on canonical selection.
• The content is thin or templated, so it is deprioritized or dropped.
• The page relies on JavaScript for its main content and the link appears only after rendering.

These overlap with the broader causes on why indexing rates vary.

Measuring it properly

Start event

The cleanest start event is a Googlebot request in the target server's access logs, verified as genuine Googlebot. For backlinks on third-party sites, logs are usually unavailable, so the practical start is the time the indexer reports the crawl. Note which one you used.

End event

Index presence is only known when someone checks. If a URL was absent at hour 12 and present at hour 24, its lag is somewhere in that interval. Reporting the interval, or its upper bound, is more honest than reporting the check time as if it were the moment of indexing. Checking more often narrows the interval and costs more checks.

URLs that never index

A URL that is crawled but never indexed has no lag value. Dropping it silently makes average lag look better and hides the failure. Report two numbers: the lag distribution among URLs that were indexed, and the share that were not indexed by the end of the window. Indexing metrics defined gives the formal definitions, and benchmark your own indexer shows how to build these into a test.

Use the median

Lag is skewed. A few slow URLs pull the mean upward, so the median and a high percentile (such as the 90th) describe it better than a single average.

Lag and the checking schedule

Because most uptake for standard jobs falls 24 to 48 hours after the crawl, a check at hour 6 will report many URLs as not indexed that will be indexed a day later. Treating those early negatives as failures leads to wasted resubmissions. IndexChex's scheduled re-checks run 1 to 5 days after submission for this reason. The indexing timeline maps the full schedule.

In the benchmark

The IndexChex weekly benchmark reports time to first crawl and indexation rates as averages. How those figures are collected, including what is and is not published about the method, is on the methodology page. The IndexChex entry explains the product being measured. Readers comparing tools should check whether a vendor's speed figure is a crawl time, a lag, or a total; the guide to reading indexing claims covers that check.