The Waraas.Com Logo - Click To Go To The Waraas.Com Homepage

Experiment: Googlebot Tracker 4200

Thumbnail image for Experiment: Googlebot Tracker 4200

Below you will see all of the Googlebot (Google web crawler) visits to my entire website (Waraas.Com). I built this tool out of boredom, and because I want to see the crawler data.

G o o g l e b o t

A Googlebot is the web crawling bot (sometimes also called a "spider") used by Google. It's a crucial part of Google's indexing algorithm, helping to discover new and updated URL's to be added to the Google search engine.

This tool tracks ALL of the website visits, including the bad URLs (404s), sitemaps, feeds, images, etc. The data will be open and free to the public.

Overtime, I will compile the data and share the results so we can all better understand how the Googlebot's work.

Refresh (F5) this page to see any new data.

Updates:

9/28/25 - I updated the Googlebot Tracker 4200 code to get the "redirect URL", "post type", and "content type" data from the Googlebots now. I also updated the code so it handled NULL better.

9/25/25 - I updated the /robots.txt file to not crawl the Googlebot tracker pages (googlebot-tracker?page= AND googlebot-tracker/?page=) on this website. I also fixed a error that showed broken URL's.

6/12/24 - I added two things; a pagination, and a simple function that shows the Googlebot sessions count for each URL on this website.

5/6/24 - I added a function to track if the Googlebot was using SSL.

4/10/24 - Because of some fake Googlebot traffic, I have updated the Googlebot Tracker to now log the hostname, and if the bot is on a public or private IP address. Because of those two changes, I have updated the Googlebot Tracker version from 4000 to 4200. The tracker is now called The Googlebot Tracker 4200 :)

4/16/24 - The tracker now logs the Googlebots "Session ID" and the page load time of each Googlebot visit.

Googlebot Sessions Per URL:

This report displays the number of times Googlebot has visited each URL on Waraas.com. It provides insights into how frequently the site's content is being accessed by Google's web crawler.

Title Slug Visit Count
Homepage / 620
Robots.txt /robots.txt 7084
Sitemap.xml /sitemap.xml 86
Experiment: Googlebot Tracker 4200 /googlebot-tracker/ 46808
Who Is Jon Waraas? /who-is-jon-waraas/ 121
Privacy Policy /privacy-policy/ 113
Googlebot Accessing Broken URLs on Waraas.com /googlebot-broken-url-access/ 113
Jon's Experiments /experiments/ 100
Jon's Projects /projects/ 89
Contact Jon Waraas /contact-jon/ 80
I'm Launching My First Free WordPress Plugin! /blog/launching-first-free-wordpress-plugin/ 62
Terms Of Service /terms/ 56
Is Google Search Console Data Accurate? /blog/is-google-search-console-data-accurate/ 53
Did A Googlebot Really Crawl My Website's Dedicated IP Address? /blog/did-a-googlebot-really-crawl-my-websites-dedicated-ip-address/ 47
An Update on Our Cabin Outhouse Project /blog/an-update-on-our-cabin-outhouse-project/ 45
How Much Does It Cost to Build an Outhouse in Idaho? /blog/how-much-does-it-cost-to-build-an-outhouse-in-idaho/ 43
From Curiosity to Ownership: Our Adventure In Gilmore, Idaho /blog/from-curiosity-to-ownership-my-adventure-in-gilmore-idaho/ 42
How Much Did a 12x20 Amish-Built Shed Cost? /blog/how-much-did-a-12x20-amish-built-shed-cost/ 41
Our Plans For Camp Bluebird This Month /blog/our-plans-for-camp-bluebird-this-month/ 40
Launching AntlerSheds.Com! /blog/launching-antler-sheds-com/ 37
The Art of Bot War: What Are Fake Googlebots? /blog/the-art-of-bot-war-what-are-fake-googlebots/ 32
Googlebot Testing: 404 vs 410? /blog/googlebot-testing-404-vs-410/ 29
Googlebot Accessing Waraas.com via IP Address /direct-ip-access-by-googlebot/ 26
The DOJ Wants Google To Sell Chrome - What A SEO'er Thinks /blog/doj-wants-google-to-sell-chrome-what-seo-thinks/ 23
The Googlebot Tracker 4000 Is Live! /blog/googlebot-tracker-4000-is-live/ 22
How Developers Can Manually Add Yotpo To WordPress Theme Files /blog/how-to-add-yotpo-to-wordpress-theme-files/ 14
Googlebot Activity on Other Websites /googlebot-activity-other-websites/ 5
Why Googlebot May Request /robots.txt via IP /blog/why-googlebot-may-request-robots-txt-via-ip/ 3
Hypothesis: How Googlebot Tracker 4200 Got External Crawl Data /blog/googlebot-tracker-external-crawl-data/ 3
Googlebot IPs & Hostnames Logged by Googlebot Tracker 4200 /googlebot-ips-hostnames-logged-by-googlebot-tracker-4200/ 0


56995
Average crawls per day: 110.67
Total broken URLs: 546
Googlebot direct IP accesses: 93
Googlebot External URL crawls: 55

View All Googlebot Sessions:

The list below displays each individual Googlebot visit to this website.

Visit Time: October 8th, 2025
Page URL: https://waraas.com/robots.txt
Referrer URL: No URL
Hostname: crawl-66-249-65-168.googlebot.com
User Agent: Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)
Session ID: rjlnmo6r99b6o8k6gbnjelkchk
IP Address: 66.249.65.168
HTTP Status: 200
Load Time: 0.004 seconds
Visit Time: October 8th, 2025
Page URL: https://waraas.com/robots.txt
Referrer URL: No URL
Hostname: crawl-66-249-65-169.googlebot.com
User Agent: Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)
Session ID: 82runtfp5lv3vpbcjpsmrjligd
IP Address: 66.249.65.169
HTTP Status: 200
Load Time: 0.002 seconds
Visit Time: October 8th, 2025
Page URL: https://waraas.com/
Referrer URL: No URL
Hostname: crawl-66-249-65-169.googlebot.com
User Agent: Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/99.0.4844.84 Mobile Safari/537.36 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)
Session ID: j5vpsno83tf1akbeackkc94odf
IP Address: 66.249.65.169
HTTP Status: 200
Load Time: 0.005 seconds
Visit Time: October 8th, 2025
Page URL: https://waraas.com/
Referrer URL: No URL
Hostname: crawl-66-249-65-168.googlebot.com
User Agent: Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)
Session ID: 6k20feejutg36elceel8gmjlnt
IP Address: 66.249.65.168
HTTP Status: 200
Load Time: 0.006 seconds
Visit Time: October 8th, 2025
Page URL: https://waraas.com/robots.txt
Referrer URL: No URL
Hostname: crawl-66-249-65-169.googlebot.com
User Agent: Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)
Session ID: ofi8pd0o4ms43ui1spa84omum6
IP Address: 66.249.65.169
HTTP Status: 200
Load Time: 0.004 seconds
Visit Time: October 8th, 2025
Page URL: https://waraas.com/
Referrer URL: No URL
Hostname: crawl-66-249-65-168.googlebot.com
User Agent: Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/140.0.7339.207 Mobile Safari/537.36 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)
Session ID: tm5sveee0oftuf5ei5c2md3mug
IP Address: 66.249.65.168
HTTP Status: 200
Load Time: 0.005 seconds
Visit Time: October 7th, 2025
Page URL: https://waraas.com/robots.txt
Referrer URL: No URL
Hostname: crawl-66-249-65-168.googlebot.com
User Agent: Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)
Session ID: mr91mvgd728bub0gro4rk0sbvh
IP Address: 66.249.65.168
HTTP Status: 200
Load Time: 0.005 seconds
Visit Time: October 7th, 2025
Page URL: https://waraas.com/robots.txt
Referrer URL: No URL
Hostname: crawl-66-249-65-168.googlebot.com
User Agent: Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)
Session ID: 6v1l3doi2k2thu9j34mssip3re
IP Address: 66.249.65.168
HTTP Status: 200
Load Time: 0.004 seconds
Visit Time: October 7th, 2025
Page URL: https://waraas.com/robots.txt
Referrer URL: No URL
Hostname: crawl-66-249-65-168.googlebot.com
User Agent: Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)
Session ID: upp5qbvuemsaddo4oiop45c1r5
IP Address: 66.249.65.168
HTTP Status: 200
Load Time: 0.002 seconds
Visit Time: October 7th, 2025
Page URL: https://waraas.com/robots.txt
Referrer URL: No URL
Hostname: crawl-66-249-65-169.googlebot.com
User Agent: Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)
Session ID: pbolhoafoop28a2r4e418tuvrg
IP Address: 66.249.65.169
HTTP Status: 200
Load Time: 0.004 seconds
123...5700Next »

Googlebot Related Blog Posts:

This category encompasses all of my laboratory work related to my experiments on the Google web crawler, known as "Googlebot".

A thumbnail image for Hypothesis: How Googlebot Tracker 4200 Got External Crawl Data
Hypothesis: How Googlebot Tracker 4200 Got External Crawl Data
2 Comments
My Googlebot Tracker 4200 has logged 55,000+ crawl entries, including some from other sites. Here are my hypotheses on why external Googlebot data appears....
A thumbnail image for Why Googlebot May Request /robots.txt via IP
Why Googlebot May Request /robots.txt via IP
0 Comments
Learn why Googlebot may request /robots.txt via a website's IP address instead of its domain. Possible reasons include DNS verification, duplicate detection, network diagnostics, security checks, and ...
A thumbnail image for Did A Googlebot Really Crawl My Website's Dedicated IP Address?
Did A Googlebot Really Crawl My Website's Dedicated IP Address?
0 Comments
How the Googlebot tracker 4200 revealed unexpected insights in my latest tests....
A thumbnail image for Googlebot Testing: 404 vs 410?
Googlebot Testing: 404 vs 410?
0 Comments
In this article, I explore the impact of HTTP status codes 410 and 404 on Googlebot's crawl budget....
A thumbnail image for The Art of Bot War: What Are Fake Googlebots?
The Art of Bot War: What Are Fake Googlebots?
0 Comments
Here I discuss what Googlebots are, and how to detect the little creatures. ...
A thumbnail image for The Googlebot Tracker 4000 Is Live!
The Googlebot Tracker 4000 Is Live!
0 Comments
I built a Googlebot tracker using sessions, PHP, and SQL....
This totally free tool checks your WordPress® website for hacks and malware in seconds. No email, phone, or billing information required!
(Results will be displayed on a new page)

Ever since building my first website in 2002, I've been hooked on web development. I now manage my own network of eCommerce/content websites full-time. I'm also building a cabin inside a old ghost town. This is my personal blog, where I discuss web development, SEO, cabin building, and other personal musings.

Feedburner Image