The Waraas.Com Logo - Click To Go To The Waraas.Com Homepage

Experiment: Googlebot Tracker 4200

Thumbnail image for Experiment: Googlebot Tracker 4200

Below you will see all of the Googlebot (Google web crawler) visits to my entire website (Waraas.Com). I built this tool out of boredom, and because I want to see the crawler data.

G o o g l e b o t

A Googlebot is the web crawling bot (sometimes also called a "spider") used by Google. It's a crucial part of Google's indexing algorithm, helping to discover new and updated URL's to be added to the Google search engine.

This tool tracks ALL of the website visits, including the bad URLs (404s), sitemaps, feeds, images, etc. The data will be open and free to the public.

Overtime, I will compile the data and share the results so we can all better understand how the Googlebot's work.

Refresh (F5) this page to see any new data.

Updates:

6/12/24 - I added two things; a pagination, and a simple function that shows the Googlebot sessions count for each URL on this website.

5/6/24 - I added a function to track if the Googlebot was using SSL.

4/10/24 - Because of some fake Googlebot traffic, I have updated the Googlebot Tracker to now log the hostname, and if the bot is on a public or private IP address. Because of those two changes, I have updated the Googlebot Tracker version from 4000 to 4200. The tracker is now called The Googlebot Tracker 4200 :)

4/16/24 - The tracker now logs the Googlebots "Session ID" and the page load time of each Googlebot visit.

Googlebot Sessions Per URL:

This report displays the number of times Googlebot has visited each URL on Waraas.com. It provides insights into how frequently the site's content is being accessed by Google's web crawler.

TitleSlugVisit Count
Homepage/47
Robots.txtrobots.txt238
Sitemap.xmlsitemap.xml24
Terms Of Serviceterms7
An Update on Our Cabin Outhouse Projectan-update-on-our-cabin-outhouse-project5
I'm Launching My First Free WordPress Plugin!launching-first-free-wordpress-plugin4
Privacy Policyprivacy-policy4
Is Google Search Console Data Accurate?is-google-search-console-data-accurate3
Googlebot Testing: 404 vs 410?googlebot-testing-404-vs-4103
Our Plans For Camp Bluebird This Monthour-plans-for-camp-bluebird-this-month3
Contact Jon Waraascontact-jon3
Who Is Jon Waraas?who-is-jon-waraas3
Jon's Experimentsexperiments3
The Googlebot Tracker 4000 Is Live!googlebot-tracker-4000-is-live2
How Developers Can Manually Add Yotpo To WordPress Theme Fileshow-to-add-yotpo-to-wordpress-theme-files2
Did A Googlebot Really Crawl My Website's Dedicated IP Address?did-a-googlebot-really-crawl-my-websites-dedicated-ip-address1
How Much Does It Cost to Build an Outhouse in Idaho?how-much-does-it-cost-to-build-an-outhouse-in-idaho1
Experiment: Googlebot Tracker 4200googlebot-tracker1
The Art of Bot War: What Are Fake Googlebots?the-art-of-bot-war-what-are-fake-googlebots0
From Curiosity to Ownership: Our Adventure In Gilmore, Idaho from-curiosity-to-ownership-my-adventure-in-gilmore-idaho0

View All Googlebot Sessions:

The list below displays each individual Googlebot visit to this website.

Visit Time: July 13th, 2024
Page URL: https://waraas.com/googlebot-tracker/?page=27
Referrer URL: No URL
Hostname: crawl-66-249-65-168.googlebot.com
User Agent: Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/125.0.6422.175 Mobile Safari/537.36 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)
Session ID: u6bl090rqg93b5tn2pjfikufkh
IP Address: 66.249.65.168
HTTP Status: 200
Load Time: 0.011 seconds
Visit Time: July 13th, 2024
Page URL: https://waraas.com/robots.txt
Referrer URL: No URL
Hostname: crawl-66-249-65-170.googlebot.com
User Agent: Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)
Session ID: 0qhsgq8s8hafgu09kljjmmrugd
IP Address: 66.249.65.170
HTTP Status: 200
Load Time: 0.003 seconds
Visit Time: July 13th, 2024
Page URL: https://waraas.com/blog/googlebot-testing-404-vs-410
Referrer URL: No URL
Hostname: crawl-66-249-65-170.googlebot.com
User Agent: Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/125.0.6422.175 Mobile Safari/537.36 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)
Session ID: c6ou8oqmsvofd8v3auovc37k6j
IP Address: 66.249.65.170
HTTP Status: 200
Load Time: 0.005 seconds
Visit Time: July 13th, 2024
Page URL: https://waraas.com/privacy-policy/
Referrer URL: No URL
Hostname: crawl-66-249-65-170.googlebot.com
User Agent: Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/125.0.6422.175 Mobile Safari/537.36 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)
Session ID: jea2q873u99kmtstccjlr3hcno
IP Address: 66.249.65.170
HTTP Status: 200
Load Time: 0.003 seconds
Visit Time: July 13th, 2024
Page URL: https://waraas.com/googlebot-tracker/?page=24
Referrer URL: No URL
Hostname: crawl-66-249-65-170.googlebot.com
User Agent: Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/125.0.6422.175 Mobile Safari/537.36 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)
Session ID: 0tungqpf6edhij4lmbr3s1mvuf
IP Address: 66.249.65.170
HTTP Status: 200
Load Time: 0.249 seconds
Visit Time: July 13th, 2024
Page URL: https://waraas.com/robots.txt
Referrer URL: No URL
Hostname: crawl-66-249-65-168.googlebot.com
User Agent: Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)
Session ID: eg09j0l2b5kg238d6782vrmsvn
IP Address: 66.249.65.168
HTTP Status: 200
Load Time: 0.002 seconds
Visit Time: July 13th, 2024
Page URL: https://waraas.com/googlebot-tracker/?page=25
Referrer URL: No URL
Hostname: crawl-66-249-65-170.googlebot.com
User Agent: Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/125.0.6422.175 Mobile Safari/537.36 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)
Session ID: 8ciifm6e7qi62fj5b29i4bupkc
IP Address: 66.249.65.170
HTTP Status: 200
Load Time: 0.130 seconds
Visit Time: July 13th, 2024
Page URL: https://waraas.com/robots.txt
Referrer URL: No URL
Hostname: crawl-66-249-65-168.googlebot.com
User Agent: Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)
Session ID: 63bql0sn6drkh88ikcn7ksusro
IP Address: 66.249.65.168
HTTP Status: 200
Load Time: 0.003 seconds
Visit Time: July 13th, 2024
Page URL: https://waraas.com/googlebot-tracker/?page=1
Referrer URL: No URL
Hostname: crawl-66-249-65-168.googlebot.com
User Agent: Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/125.0.6422.175 Mobile Safari/537.36 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)
Session ID: h8749fae1o7ha25rds75qeiivt
IP Address: 66.249.65.168
HTTP Status: 200
Load Time: 0.254 seconds
Visit Time: July 13th, 2024
Page URL: https://waraas.com/robots.txt
Referrer URL: No URL
Hostname: crawl-66-249-65-169.googlebot.com
User Agent: Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)
Session ID: iuqeh2fbftfpn8er2dpu2voki6
IP Address: 66.249.65.169
HTTP Status: 200
Load Time: 0.003 seconds
Visit Time: July 13th, 2024
Page URL: https://waraas.com/robots.txt
Referrer URL: No URL
Hostname: crawl-66-249-65-168.googlebot.com
User Agent: Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)
Session ID: mdeg6kf3rl6n424mp7v67g99gc
IP Address: 66.249.65.168
HTTP Status: 200
Load Time: 0.005 seconds
Visit Time: July 13th, 2024
Page URL: https://waraas.com/googlebot-tracker/?page=19
Referrer URL: No URL
Hostname: crawl-66-249-65-170.googlebot.com
User Agent: Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/125.0.6422.175 Mobile Safari/537.36 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)
Session ID: 8p3btnq0tlnhprmu87utg4g1q0
IP Address: 66.249.65.170
HTTP Status: 200
Load Time: 0.269 seconds
Visit Time: July 13th, 2024
Page URL: https://waraas.com/googlebot-tracker/?page=2
Referrer URL: No URL
Hostname: crawl-66-249-65-169.googlebot.com
User Agent: Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/125.0.6422.175 Mobile Safari/537.36 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)
Session ID: 1gcqobknb5qn7vu8155tjmidl5
IP Address: 66.249.65.169
HTTP Status: 200
Load Time: 0.273 seconds
Visit Time: July 13th, 2024
Page URL: https://waraas.com/robots.txt
Referrer URL: No URL
Hostname: crawl-66-249-65-170.googlebot.com
User Agent: Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)
Session ID: abh32tc3drkecc4c0bbegtqd41
IP Address: 66.249.65.170
HTTP Status: 200
Load Time: 0.005 seconds
Visit Time: July 12th, 2024
Page URL: https://waraas.com/
Referrer URL: No URL
Hostname: crawl-66-249-65-170.googlebot.com
User Agent: Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/99.0.4844.84 Mobile Safari/537.36 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)
Session ID: ude7sg81hjr9m79k1rnuei79pn
IP Address: 66.249.65.170
HTTP Status: 200
Load Time: 0.004 seconds
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27
Next

Googlebot Related Blog Posts:

This category encompasses all of my laboratory work related to my experiments on the Google web crawler, known as "Googlebot".

A thumbnail image for Did A Googlebot Really Crawl My Website's Dedicated IP Address?
Did A Googlebot Really Crawl My Website's Dedicated IP Address?
0 Comments
How the Googlebot tracker 4200 revealed unexpected insights in my latest tests....
A thumbnail image for Googlebot Testing: 404 vs 410?
Googlebot Testing: 404 vs 410?
0 Comments
In this article, I explore the impact of HTTP status codes 410 and 404 on Googlebot's crawl budget....
A thumbnail image for The Art of Bot War: What Are Fake Googlebots?
The Art of Bot War: What Are Fake Googlebots?
0 Comments
Here I discuss what Googlebots are, and how to detect the little creatures. ...
A thumbnail image for The Googlebot Tracker 4000 Is Live!
The Googlebot Tracker 4000 Is Live!
0 Comments
I built a Googlebot tracker using sessions, PHP, and SQL....
This totally free tool will ping your website to Google, Bing & others to give it a little extra boost.
(Results will be emailed after completion.)

Ever since building my first website in 2002, I've been hooked on web development. I now manage my own network of eCommerce/content websites full-time. I'm also building a cabin inside a old ghost town. This is my personal blog, where I discuss web development, SEO, eCommerce, cabin building, and other personal musings.

Recent Comments:

Brett : Very cool to get the back story and will be neat to watch the progress. Hoping eventually to so do something the similar on the west coast of Canada somewhere. Amazing that in 2006 I first found your site for it's myspace page information and how to build PHP site header/footers for resale. How times change hah. Anyways, keep up the great posts, looking forward to the updates.

Posted on: April 11, 2024

Feedburner Image