Interesting. Dan reports that Google is watching more closely than he expected.

I made an offhand comment on IRC with the URL for the real image (I hardlink it on the server, rather than playing Apache redirect games) and three minutes later... Pow! There was googlebot, looking at it. Turns out that one of the folks on the channel'd looked at it, and they run Opera with the google stuff on it so presumably that's how google got the URL. I will admit to being very impressed with the speed that the crawler struck out with (and it was the crawler, at least according to the log data and the PTR record for the IP address) but still...

Of course, it could have been the experimental IRC sniffer bot they've been playing with.

Posted by jzawodn at November 10, 2003 09:16 PM

Reader Comments
# Scott Johnson said:

True, it could have been the bot, but I've noticed that pages with AdSense get crawled quite often. If google doesn't already have the page in its cache, it will send out googlebot within 30 minutes (usually) to grab a page for AdSense. A bonus for site owners is that the page gets stuffed into the search database at the same time. I'm not familiar with the backend, so it might just all be the same database, but still, it's pretty cool.

on November 10, 2003 11:03 PM
# Hal said:

I think Googlebot will achieve consciousness pretty soon.

on November 10, 2003 11:18 PM
# Dan said:

If it was Google monitoring the channel then it means they were tied into the IRC network. #parrot is pretty small (~30 folks normally) and everyone's got op privs, so there wasn't a bot hiding and watching. (Well, not a google bot at least, as we've got a half-dozen special-purpose bots hanging around)

While I'm not thrilled with the idea of a google toolbar watching the URLs you type, I like it a lot better than the thought that they've just got another server tied into the IRC net I'm on and are watching everything...

on November 11, 2003 05:15 AM
# Dan said:

It's definitely the plugin. Did some testing, and put up the results on the blog.

on November 11, 2003 11:42 AM
# Tim Gilberg said:

what about redirect with robot.txt while google toolbar and the URLs in conjunction

on November 16, 2003 06:35 PM
Disclaimer: The opinions expressed here are mine and mine alone. My current, past, or previous employers are not responsible for what I write here, the comments left by others, or the photos I may share. If you have questions, please contact me. Also, I am not a journalist or reporter. Don't "pitch" me.

 

Privacy: I do not share or publish the email addresses or IP addresses of anyone posting a comment here without consent. However, I do reserve the right to remove comments that are spammy, off-topic, or otherwise unsuitable based on my comment policy. In a few cases, I may leave spammy comments but remove any URLs they contain.