When Googlebot crawls a URL, it may follow 5 or, alternatively, 20 redirects at the same time.
Google has stated that they will follow five redirects at the same time. This experiment aims to find out a bit more about how far Google will go.
It’s a simple experiment where the below link redirects forever. By tracking Googlebot activity, I can find out how many of the redirects Google has followed, how frequently they are crawled, and how long it takes to get to a certain point.
The Chrome browser stopped at 21 redirects (so does my Android browser), Firefox hit 20, Edge did 110, while IE won at 111 redirects!
In the initial run of this test years ago, it made a very quick crawl of 20 redirects. The next day it had a second go; this time, the crawling was slower, with a few seconds between redirects, and it still made 20 redirects.
In 2022 I re-ran the test, and it showed that Googlebot only crawled 5 redirects at first but later came back and made 20.
This all shows that how Googlebot deals with redirects can vary, so don’t make any assumptions, as the system may change tomorrow.
The link below is the test link. It will 301 redirect forever, each time incrementing the number in the destination URL. If you click on it in your browser, you will get a “too many redirects” error, and the number in the address bar is how many redirects were followed before the error. Infinite Redirect Test Link
Since the initial test, Googlebot and the Smartphone bot have recrawled several times, each time reaching 20 redirects. The Smartphone bot has also continued on to crawl up to the 36th redirect. It looks like it is possible to get pages found that are behind a lot of redirects!
I’ve shut down the infinite redirects as it was wasting the time of crawlers such as Bing, Yandex and Ahrefs. Some crawlers made it into the thousands of redirects before stopping!
A new test confirms now only 5 redirects on a request indexing. But three days later, Googlebot came back and crawled to the 20th redirect in 10 seconds.