I was looking at some updated stats for my web site and decided to plug the numbers into Excel so that I could more easily visualize the relationships. The chart below is one of several I produced. It shows the percentage of search referrals I can attribute to each of the major search engines for the month of October 2005.
What's interesting to note is that neither AOL or MSN are "major search players" if you look at these numbers and nothing else. All other search engines combined provided more referrals than either AOL or MSN.
The gap between Google and Yahoo! is hard to interpret, since it doesn't come close to matching the publicly available market share numbers. The same is true of the numbers for MSN and AOL. They should be higher.
There are two ways I can think to explain this:
- People who use Google are more likely to be searching for content that's on my site.
- The market share numbers are wrong. Google actually generates more traffic than has been reported and MSN and AOL have been over-estimated.
I suspect that #1 is closer to reality. After all, I most often write about topics that are of interest to an audience that's more technical than average. And I suspect that crowd skews toward Google in a more dramatic fashion than the general population of Internet users. If that's true, it would seem to confirm many of the stereotypes about AOL and MSN users.
What do your numbers look like?
[Yes, I know the chart title is messed up.]
Update: Others have responded with numbers on their own sites:
Posted by jzawodn at October 22, 2005 01:46 PM
It could just mean that your content ranks higher in Google than the other engines. Or maybe that Google more heavily favors blogs in general. Would be interesting to compare the queries and your ranking for each...
If you have a script or something to make mining the apache server logs easy, I'll run my site through it, and post the results for comparision.
My blog is not even close in popularity to yours, so my numbers are more prone to statistical error. However here are my 2 canadian cents:
90% - Google, 6% - Yahoo, 2% - Ask Jeeves, 2% - others
Which kinda confirms your hypothesis about "technical audience", I have even more tech-inclined blog than you (Heck, it is even named after C++ class).
omg, no preview button :(
I just logged into Urchin and pulled the stats from the past week (about 73,000 visits from search engines). The traffic from the top four search engines:
* Google - 72%
* Yahoo - 20%
* MSN - 6%
* AOL - 2%
This is pretty close to your results and most people that come through search engines to my site aren't real techie. They are simply looking for a new cell phone. The biggest search engine refered page right now for me is about a pink Motorola phone, not exactly the topic to reel in the geeks.
My traffic is pathetic (and so is my stats program, analog 6.0) but from what I can ascertain, I have 9 referrers from MSN Search and 6 from Google.
Looking more broadly at referrer, but restricted to just the regex (google|yahoo|aol|search.msn) across all the websites for my company (the .com, .org, etc sites) I get these results .
This shows google accounting for the vast majority of referrals, with google france beating yahoo worldwide by a healthy margin. France!
The reasons I can think of (first two are yours):
1) Google searchers happen to be looking for what's on your site (maybe because their audience is more technical)
2) Market share numbers are wrong
3) You rank higher on Google than on Y! independent of query population (e.g. they may give higher ranking to blogs)
4) Market share for searches might be right, but share of click-throughs might be quite different (like more of Y!'s results below the fold)
5) Google might be presenting you in a more click-attractive way than Y! is.
As to why your Y! referrals are so far ahead of everyone but G: maybe a lot of your Y! referrals are Y! employees or partisans, who a) are interested in your topics, and b) use Y!'s search engine.
I had an argument with Tim Bray about this very topic (whether referral logs are a trustworthy proxy for market share). I think his argument was essentially that he would rather trust his own loosely-correlated proxy for market share than some untransparent third party's direct study of market share, no matter how loose the correlation was. (I think it's wacky to treat referrals to any one site as a market share study.) His referral numbers look a lot like yours except that you're getting more traffic from Y!, and he gets a lot from Google Image Search. (I would link if tags were allowed...)
Jeremy ate my URL:
http://www.rungie.com/~craig/gumstix-referrer.png
Oh, worth mentioning that the stats are also for October to date.
Tim,
You're getting at something I've long suspected--that market share numbers without some additional context and/or understanding of where they come from are mostly useless unless you're writing a story for Forbes (or some business pub) about the search engine industry.
Jeremy
Reader2: ( http://reader2.com )
Google 94%
MSN 1.6%
Yahoo 0.8%
MyProgs: ( http://myprogs.net )
Google 75%
Yahoo 12.7%
MSN 4.2%
(or you wanted only blog stats?)
My numbers are similar to others...
http://ultramookie.com/wayback/2005/10/22/september-referral-statistics/
Google 67%
Yahoo! 22%
Others 6.5%
Zawodny 1.5%
MSN 1.2%
AOL 0.3%
Make that 82% for Google, since AOL search is powered by Google...
I got:
Google 82.7%
Yahoo 4.8%
MSN 2.6%
Ask 0.9%
Other 9.1%
Those stats seem in line with the "publicly available market share numbers" I've seen, and my stats in general. Though sometimes MSN send the most traffic for some sites, Yahoo for others.
BTW, why Excel and not Open Office?
It's interesting that you say that the tech crowd is more prone to go to Google than the general public. I'd think that it's the other way around; Google is the only one known by everyone, and the tech crowd is aware of the alternatives. That could support your data in the sense that the tech crowd knows Yahoo! is a viable alternative and maybe AOL and MSN not so much.
My stats are showing agreement on the Google number:
http://elliottback.com/wp/archives/2005/10/22/october-web-search-statistics/
It's a bit different for photo blogs, I think. My stats are probably not representative of all photo blogs, but still...
Google (Images) 96.5%
Google 1.2%
Yahoo 0.9%
AOL 0.5%
MSN 0.1%
My photos do show up in Yahoo image search, but the cache and search results are well out of date. Slurp is slurping almost as much as Googlebot is but apparently the listing is just not getting updated as often.
My stats tells me that..
Google: 46%
Yahoo: 20%
MSN: 19%
Google images: 8%
Ask Jeeves: 3%
Seems like I have a pretty high number for MSN compared to others here, and I suppose that's because my weblog is fairly personal and non-technical..
If you look at say the office suite market, you will find that 99% is owned by MSFT and 1% by everyone else. I don't see why it will not be the same for the major search engines. It is remarkable that yahoo is doing as well as it is. Most likely because of distribution deals with SBC and the Yahoo home page. However, in the future when there is free google wifi everywhere people will no longer use the SBC home page and most of the nwq firefox downloads will not set their home page to yahoo.com
That may all be true, but when the free Google space elevator gets built I'm going to quit spending any time on the internet and get the hell off of this dirtball, so none of this will matter.
Jeremy: I don't completely agree.
I agree that some queries are more interesting and lucrative than others, but we can still ask a factual question: what proportion of overall search engine queries in a given market (say, the U.S.) does each engine currently receive?
The fact that we don't have a good way to answer that does not mean that it's a nonsensical question (even if Fortune magazine is the one asking). If I were an investor in either Y or G, I might be interested to know the Y/G query ratio within an order of magnitude.
I don't think we know the answer, but my point is that I'm impatient with any attempt to answer it based on a single site's referrer logs... That just has too many sources of skew.
Tim, I think we agree. I said the numbers didn't mean much without additional context. Also, I hope you don't think I was attempting to answer the question based on a single site's referrer logs.
I posted mine in the hopes of see others and getting an idea of how similar (or not) they are. So far, it's been working.
I happen to run a poetry and literature site and its top 4 are currently:
Google: 74.84%
Yahoo: 11.75%
MSN: 6.43%
AOL: 2.85%
So its not just technical sites that experience this.
I remember reading somewhere though, that while their market share is calculated by number of visitors, Yahoo's visitors are often there for things other than search (the various groups, my yahoo, etc), while Google's visitors are usually there to just search.
I'm heavily Google dominated (90%), followed by MSN (6%), then Yahoo (2), Google images (1%), and everybody else (1%). AOL contributed a single referrer this month.
I don't think this is a great indicator of the relative traffic driven by the engines; it's heavily influenced by the ranking of your site on each engine. For some reason Yahoo really doesn't like my site. On my top traffic driving queries I'm in the first page on Google, often nowhere to be found on Yahoo. For example, ego searching my last name has me as 5 of the top 7 hits in G, 1 of top 7 in Y.
Here, I did my stats, seperating Google by product (Reader, IG, translate, Images) and including others, like Bloglines, Memeorandum and del.icio.us, and Google trounces. Ask doesn't even show up:
http://google.blognewschannel.com/index.php/archives/2005/10/23/search-engine-referral-stats/
Check this out...
MSN... 59%
Yahoo... 27%
Google...13%
Other...1%
These are for my Work's web site, which is a non-profit agency that deal with kids who have had to be removed from their homes. Most of our traffic comes from Child Care Workers looking for jobs and Donors (Which demographically are almost universally over 50)
Anyway, I think this just verifies what everyone else has been saying, the people who would search for stuff on your blog are probably going to be bloggers who, as a technically savy group, are going to use google.
People who aren't so computer inclined are most likely going to (A) use the search engine that comes up when they open their browser or (B) use the search engine with the most recognizable name.
85% Google, 11% Yahoo, 3% MSN, 1% AOL
Excluding Google Image Search, which would add another 83%
One thing that I have noticed in my logs is that there is some crawler that fakes a Google referrer. I see a ton of referrer entries from "http://www.google.com", and I really doubt that I am on the Google main page.
My blog stats: 2005-10
Google 48.7 %
BaiDu 41.6 %
QQ 3.2 %
Yahoo 1.5 %
3721 1.4 %
YiSou 0.8 %
SoGou 0.7 %
NetEase 0.5 %
MSN 0.3 %
www.chedong.com/cgi-bin/awstats/awstats.pl?config=chedong
Compare 1 year before: 2004-10
Google 62.8 %
BaiDu 29.1 %
3721 2.5 %
Yahoo 2.4 %
YiSou 1.2 %
Our site aims at a relatively unsophisticated market, but the results are similar to yours: Google sends about 4 or 5 times the traffic as Yahoo. We are number 1 in Yahoo, and a competitor is number 1 in Google for the main search term in our market. We see the competition's raw logs due to ongoing merger due diligence investigations, so we can do a direct comparison of what number 1 for a general consumer market means in the two engines. We aim at women in their 30s to 50s in a non-technical market.
The statistics for my non-tech ONG site show the following:
Google 87
Yahoo 7
MSN 3
AOL 0
Others 3
At random looked at one site for one month. June 2005. Consumer site. Ranks no 1 organic on both Yahoo and Google for specific phrase. Of all referrals from Google and Yahoo for that phrase during June, Google 89.6%, Yahoo 10.4%.
Interesting stats.
Mine are even more bizarre.
The past few months, Google has been hovering between 80 and 90%, with Yahoo hovering between 3 and 8%. And MSN search hovering between 5 and 12%.
However, for this current month, as of now in fact, the figures have swung more towards MSN than anything else. And the Google percentage has dropped significantly. But Yahoo remaining consistent.
Google 48.8 %
MSN 44.9 %
Yahoo 4.3 %
(next one is Altavista. On 0.4%)
But last night's figures were:
Google 53.1 %
MSN 40.8 %
Yahoo 4.1 %
Did a bit more of an analysis and figured out why the figures got somewhat skewed towards MSN for October. Turns out, one of my posts about an MSN Messenger issue was the major cause for those hits. I wrote a little about my own conclusions here: http://will.id.au/blog/archive/2005/10/23/the-great-search-engine-divide
I think overall, it depends on the target audience as others have alluded to. The more tech-savvy are usually the Google users. They also tend to be the ones installing FireFox.
I work for a fairly high traffic non-technical website marketed at parents and our numbers are similar to yours. The numbers have been like that for us for the past 2 years with consistent relative growth from Google and gradual decline from Yahoo, AOL, and MSN. Because of this I've always suspected the market share numbers to be inflated for Yahoo, AOL, and MSN but perhaps they're using a different approach to count their search market share.
-Todd
Google is definitely the more popular search engine, but it doesn't help that Yahoo has approximately 19K pages from this site and Google reports aproximately 110K.
I've found incomplete indexing to be the single biggest obstacle to overcome in getting traffic from yahoo.
http://siteexplorer.search.yahoo.com/search?ei=UTF-8&p=jeremy.zawodny.com&bwm=p&bwmf=a&bwms=p
Jeremy, my numbers are even more skewed towards Google than yours:
http://thomashawk.com/2005/10/google-undisputed-search-champ.html
Haha you all stole this topic from me as I first ran a post on my referral stats back in September! Here goes: http://www.irishwonder.syndk8.co.uk/2005/09/21/latest-stats-for-this-blog/
OK just kidding, of course referral stats is something any SEO -mad person will post about sooner or later. My stats as of now (I am using Statcounter so this is only for the last 100 hits):
85.71% Google
14.29% MSN
Yahoo is not even there at all, but I guess people just happened not to use Yahoo for the last 100 visits to my blog I believe as normally I do see a few Yahoo searches leading to my blog here and there :-)
As to these tendencies in general - Google is long known for having a much larger market share - hence more referrals come from it, no surprise here to me.
Yeah, 100 is a very small sample!
I had roughly 200,000 in mine.
same with my site. I guess about 12K searches from google (including image searches), 500 for yahoo and 200 for msn... To be honest, I can't remember when I used Yahoo myselves, I think it was before AltaVista come on the market...
This has also perplexed me. Yahoo has consistently given me more hits than Google, and here is where I am right now.
Yahoo 50%
Google 37%
MSN 10%
My blog stats:
Domains Unique Reload Total
google.com 1625 929 2554
blogspot.com 1365 1902 3267
yahoo.com 688 241 929
technorati.com 461 184 645
http://bgcounter.com/?sts,subdom,,,,world2
Search Engine Unique Reload Total Share
Google 2,001 1,145 3,146 64.49%
Yahoo 853 315 1,168 27.49%
MSN search 112 43 155 3.61%
AWStats gives me similar numbers for my web server. so far for this month:
47% Google
38.6% Google (images)
6% Yahoo
3% MSN
1.8% AOL
Googlebot visits accounted for almost 25% of hits from the top 10 visiting bots, with Yahoo Slurp coming in a close 2nd at 22%. MSNBot was 4th at 14%.
There's a web analytics company in Poland that tracks virtually every Polish site and shows referral stats for those pages at
http://www.ranking.pl/rank.php?stat=domeny10AL
Google has a 73%, Yahoo 0.9%, two major Polish portal sites about 10%. What is interesting is that if you look at the market shares as measured by the search audiences (by unique users http://www.ranking.pl/rank.php?stat=months&megapanel=1) the whole search engine stats look totally different: Google 66%, two major Polish portal sites: 31% and 25%
A little off-topic: I can't really understand why did Yahoo so easily gave up the local markets worldwide to Google. It has a good search, supports many languages but for some reason doesn't provide simple localized search-only sites where ordinary Joes could start their surfing from. Now if you want to look for web pages in non-English language, you had to go through the advanced search as if it was so difficult to show the relevant language based on the browser language settings...
While I wait for the computers to finish counting the several millions of log lines, might I make one small observation?
Google is a verb.
<<<< People who use Google are more likely to be searching for content that's on my site.
That theory could be verified by analyzing the KEYWORDS that the individual Search Engines are sending to any site.
Some stats divide up their Categories and seperate the queries by each Search Engine
Also what are your Rankings for competative queries -
including more Valid, Relevant, Themed Keywords in the Title Tags may get the site Noticed on Yahoo and MSN.
My Japanese blog stats (September);
# Google 614 (58.70%)
# Yahoo!JAPAN 242 (23.14%)
# MSN Search 89 (8.51%)
# Biglobe 25 (2.39%)
# goo 13 (1.24%)
# SF Livedoor 13 (1.24%)
# InfoSeek Japan 12 (1.15%)
# OCN Seach 10 (0.96%)
# Excite Japan 9 (0.86%)
# Technorati 8 (0.76%)
# @nifty:@search 6 (0.57%)
# AOL Search 2 (0.19%)
# Goo Blog 1 (0.10%)
# FreshEYE 1 (0.10%)
# bulkfeeds.net 1 (0.10%)
So there is money in them thaaaar hills? Good article by the way.
Just to hop off topic for a second. Please can you test my site and give me some constructive feedback. Much appreciated.
Peter Peutamus
http://www.domeafavorbuddy.com/beta
