Way back in the early days of the Web (you know, before AdSense, blog spam, web mail, and dynamic HTML) were these things called hit counters. The Wikipedia article says a few things that I'll just quote outright.
On Popularity:
At one time it was common to see a hit counter on every page, but this is no longer the case for several reasons:
- They have been replaced (or augmented) by more complicated web analytics methodologies that give the webmaster a better overall picture of site traffic besides a simple, perpetually increasing number.
- As style elements, they are no longer associated with the impression of professional web design--some people consider web counters to be a "gimmicky" feature and they are thus typically found on personal pages created by individuals.
- The number of visitors to a site may be a trade secret
- Too small a number might indicate the page's lack of popularity. Removing the counter thus levels the playing field.
On Trust:
Web counters are not necessarily trustworthy. A webmaster could start the counter at a high number to give the impression that the site is more popular than it actually is.
Heh. You think?
Web 2.0 Metrics
I present this little walk down memory lane because it's related to something I've been wondering about. How the hell do we count stuff in a zero page refresh Web 2.0 buzzword compliant world?
Evan Williams recently declared Pageviews are Obsolete. Among his reasons are Ajax and RSS.
But Ajax is only part of the reason pageviews are obsolete. Another one is RSS. About half the readers of this blog do so via RSS. I can know how many subscribers I have to my feed, thanks to Feedburner. And I can know how many times my feed is downloaded, if I wanted to dig into my server logs. But I don't get to count pageviews for every view in Google Reader or Bloglines or LiveJournal or anywhere else I'm syndicated.
Ah, a sticky problem. One that should demand the attention of metrics companies like comScore and Nielsen NetRatings.
Over at Techdirt, they're asking Where Are The Web 2.0 Audits?
Want to start a business that is desperately needed? Get into the Web 2.0 auditing business (or perhaps that's Auditing 2.0). Just as with the last bubble, it's reached the point that you can't trust any of the numbers that are being floated concerning today's popular sites. Of course, we've covered repeatedly how questionable the valuations being tossed around are, but to support those bogus numbers, it seems that there are all sorts of other bogus numbers being thrown around as well.
Want an "old company" example of where this could get interesting? Yahoo! Mail. It has certainly accounted for billions and billions of "pageviews" over the years. But the fancy new version that's been rolling out is a lot more like a desktop mail reader than what most users are used to.
What's the right way to count the activity that Wall Street and advertisers care about?
- messages viewed
- ads requested
- active users
- time on site
- clicks
What subset of the possible metrics is both meaningful and not specific to a web mail application? That is, which metrics are broadly applicable to sites that employ similar technology for very different activities?
What about comparing sites that compete for users (and advertising dollars) but use radically different presentation technologies?
Beats me. But if you're starting to think this might be a little complicated, I'll leave you with a final quote from Ev's article:
In summary, there's no easy solution. There's a big opportunity (though very tough job) for someone to come up with a meaningful metric that weighs a bunch of factors. But no matter what, there will come a time when no one who wants to be taken seriously will talk about their web traffic in terms of "pageviews" any more than one would brag about their "hits" today.
Thoughts?
Posted by jzawodn at October 09, 2006 01:47 PM
>>What's the right way to count the activity that Wall Street and advertisers care about?
Advertisers I don't know... Wall Street, revenue!
Good god, I'd hate to see an accurate measure for my relationship with Bloglines.
I agree with Ev that there's no good answer; but the metric that matters to me is, "How many people really care about this site?" There are some traditional estimates for that (Q values and such), but sometimes you can just see it on the page (as with Flickr, obviously). With Bloglines or del.icio.us, it may not be as obvious (maybe that's one source of the blowup around post-acquisition del.icio.us traffic earlier this year, which seemed ridiculous to me).
Marc:
Good point. I'd hate to see my Bloglines score. Sort of. :-)
They could just turn it into a "rank" and make us feel good, right? "You're in the top 4% of Bloglines users!!!"
Thank you for bringing to the fore a problem I have been facing professionally in different facets. I would be curious to see what kind of solution is offered on the internet.
In my situation I have the luxury of controlling both ends of the stack and hence can invent tricks like special X-Headers much like cookies (or some other end to end mechanism).
Challenges galore. The YouTube deal also showcasing how irrelevant a 'page view' may become to full analysis.
I think commercial metrics will (must) trend towards firmly establishing costs per sale and/or customer aquisition. At the point where that gets good the advertiser really does not need more detail.
Site metrics are more complex since many bloggers would probably rather be read by a handful of movers and shakers who provide thoughtful commentary than by legions of regular Joes. A blog read by all G8 world leaders would be about 1000x more influential in terms of changing history than one read by American Idol fans, but would probably have limited commercial value. How do you measure that, with Yahoo's "BigWhig Rank" that pulls in personal data and assigns importance to the ... person?
Pageviews have been meaningless since 1994; clicks are only slightly more relevant. Vic's got it right: I can't think of a more reliable success metric than dollars spent for conversions received, can you?
PS: your hit counter seems to be broken. :)
Yup ! This is going to be the next chalange...
Good article !
The problem with $$/Conversion is that it doesn't apply to all sites.
Take this site for example. What's a conversion? A Comment? Surely that can't be accurate, for I read this and went 2 days before commenting - not to mention those who post viagra link comments.
Is merely reading it a converstion? How do you tell if I read it or just clicked away after the first paragraph?
Is a click on the ads a conversion?
Conversions work great for traditional websites that sell things, but how does one measure a site that provides information?
I used to waste time religously scanning the Web server logs, for my personal blog, trying to distinguish human activity and RSS readers from robots and spiders.
Now I tend to use 'Comments' as the best measure of interested and regular readers. Last time I looked, there were two (and I was one of them :-)
Don't discount the page views of a site. After all its the content of a site that brings the people to it, hence, the reason we see so many advertisers buying real estate on peoples webs.
Quite honestly, I believe the only reliable metric and honest ad sell is a true click thru on an advertisement by a real potential buyer.
I think it quite dishonest to sell "impressions" or any other gimmicky name inet ad sales go buy, kinda like masturbation without the end result.
Unfortunately, I see ad clicks being instantiated with a stopbot code that needs to be input before or after the ad is clicked because of all the bots.
What a sad state of affairs that a potential boon to human kind that the net represents, is being ruined and rendered useless by the new mafia on the block.
I also see coming in the near future ads, served with the RSS feeds. To much real estate for the business world to ignore there.
Hey Jeremy,
Any thoughts on how the value of "average" changes in a web 2.0 world? Seems like with longtail distributions, the average (pageviews, users, etc...) just ain't what it used to be.
I think the only thing worth anything today is the quality of the user base. Do you know in which context they view your site? Is this measurable? Can you get people to act on your behalf, defend your ideas/brand, or use word-of-mouth?
I would much rather have 100 focused people read my site and understand where I'm coming from than 100,000 people mindlessly strolling through. If I have a strong well-defined niche, I can advertise to it, pull advice and knowledge from it, and learn a lot in the process. If it takes one single HTTP request or 30,000, it doesn't matter.
Once this kind of thinking takes hold, then we'll see sites that cater to people as if they were in the real world. For instance, an airline company worries about the impression they are making on you even while you're waiting in the airport. If the quality of interaction is low, it doesn't matter that you're moving millions of people every day, because this traffic is volatile and if you don't give them a reason to come back, all you can brag about is some number you counted up, not the reality of the situation.
So I think the next big metric is the lack of one unified metric, because we don't have a single unified metric for the rest of our daily lives. The internet as a separate entity might as well be dead, because it's too intermixed with our day-to-day lives now.
I agree with the article that it is difficult to "measure" popularity, but isn't that really more a problem with smaller sites than for very popular sites?
Pageviews are probably pretty accurate for Yahoo or Digg. But, for smaller outfits I would say it may not be very accurate.
The purpose of wanting to know the popularity of a site is to pursue revenue flows. It is an important question for sites that are just starting up, especially if the only revenue flow is from Ads. As a site matures and becomes more popular, then, popularity can be measured in terms how much it can garner for its Ads as well as the other measures listed above.
I don't buy the "quality" vs. the “quantity” of users argument above, simply because I want my site to be self sufficient and eventually to provide my income. Hence, revenue flows are key. I think it is BOTH quality and quantity. I want the content of my site to be quality, but I am always watching the number of users.
There is no single answer for the right metric that works across sites.
Of course, that's not the answer Wall Street, reporters, analysts or the boss wants to hear. A single metric makes it easy to compare things and to put together PowerPoints. Never mind that the data is often not what it purports to be.
Every time I look into the details of comScore methodology, I'm stunned by how bad it is. For every comScore report I look at, I can tell you why Yahoo, Google, AOL or whoever is doing better or worse than the data show. (And I'm not talking about sampling versus actual measurement.) Of course, these numbers are quoted far and wide, both in the press and in internal company presentations.
The biggest problem with a single metric is that it can easily be manipulated to achieve the desired outcomes.
For a fun, non industry read on manipulating metrics, the Post had an interesting story:
"To eliminate unneeded cars, the county established a minimum annual mileage -- 4,500 -- and told its 11,500 employees and supervisors that any cars with odometers that did not meet that figure would be taken away."
See what happened:
http://www.washingtonpost.com/wp-dyn/content/article/2006/09/23/AR2006092300995.html
HINT: The headline was "Fairfax Employees Run Up Odometers To Keep Their Cars"
"One that should demand the attention of metrics companies like comScore and Nielsen NetRatings"
the reason these companies aren't doing anything revolutionary to track this kind of stuff is because their technology ground to a halt years ago. their current business model is promotion and patent enforcement lawsuits. their data is crap and the "panel based" reporting they do is full of holes. Nielson tried to shore up their crumbling, inefficient model by buying RedSheriff. Oops, more of the same crap.
my company tracks AJAX just fine. customers can control whether a click represents a page view or a click or any other defined event. they can also tie this to source of visitor and conversion. "sure, if you company is so great why don't we know anything about you?" because we put our money into R&D and our technology instead of into marketing and legal fees.
There is an alternative to today's measurement panels. The fact is, no panel will ever accurately measure small sites.
Quantcast provides free direct measurement of website audiences using a measurement pixel. We also provide a panel-based estimate for sites without pixels.
