In Google's embarrassing mistake, which I thought was going to be about Google Web Accelerator, Dylan calls nofollow a failure.
Since its enthusiastic adoption a year and a half ago, by Google, Six Apart, Wordpress, and of course the eminent Dave Winer, I think we can all agree that nofollow has done … nothing. Comment spam? Thicker than ever. It’s had absolutely no effect on the volume of spam. That’s probably because comment spammers don’t give a crap, because the marginal cost of spamming is so low.
But it's worse than that, he claims.
Worse, nofollow has another, more pernicious effect, which is that it reduces the value of legitimate comments. Here’s how:
Why should I bother entering a comment on your blog, after all? Well, I might comment because you’re my friend. But I might also want some tiny little reward for participating in a discussion, contributing to the content on your site, and generally enhancing the value of the conversational Web. That reward? PageRank, baby. But if your blog uses the nofollow tag, you’ve just eliminated that tiny little bit of reciprocity. Thanks, but no thanks. I’d rather just comment on my own blog. And maybe, if you’re lucky, I’ll link back to you.
I've seen that first hand. The "psychology of linking" did change in a fairly obvious way after nofollow started. Unfortunately, that was a downside that none of us saw coming back when we announced our support. I'm not sure any of us expected people to ration their links as if they were somehow in limited supply. But it happened anyway.
Introducing even a fairly subtle and indirect "economic" model into a system always changes behavior. I know that I forget (or at least underestimate) that more often than I should.
Look. Linking is part of what makes the web work. If you're actually concerned about every link you make being counted in some global database of site endorsements, you're probably over-thinking just a bit. Life's too short for that, ya know? Link and be linked to. Let the search engines sort it out.
Posted by jzawodn at May 30, 2006 06:59 AM
I'm actually surprised to see comment spam still a problem. Ever since I switched to Wordpress's Akismet, comment spam's gone down to practically zero.
I agree that linking has changed, some because of the infiltration of spam comments and trackbacks that abuse linking but also because in the early days of the web a link was a manually, hard-coded piece of HTML. With the tools we have today a link isn't quite as special as it once was.
As far as getting rid of spam links, I think it's my responsibility to keep my weblog free of spam links (and extend that to other bloggers as well as providers of blogging tools). I used to get tons of spam comments and trackbacks, so I turned both of them off. That was no good either so I installed the SpamLookup plugin to MT (now included by default) and also get almost nothing now, and when I do it rarely slips past the approval process.
I implemented a wordpress plugin to do a Jeremy-style spam guard ("type X below"), and found that it blocked 100% of the spam comments on my blog in the last 6 months. There were 254 spam attempts blocked yesterday.
As for nofollow not working, I guess I'm just a dufas because I always thought to leave a comment because I wanted to express something. Heck, Jeremy's blog doesn't even link to the poster's site, yet gets a lot of comments, so perhaps I'm not alone.....
I'm not surprised that comment spamming is still thick. It's probably because the vast majority of spammers are too stupid to know about it. They're script kiddies running farms of sites that simply spray links where ever they can. Considering I dropped my spam content by 75% just by slightly renaming the comment and trackback links, I'm willing to bet that the bulk of spammers simply don't care. It's not like they're even interested in making their process more efficient or updated. If they did, the "Jeremy" hack would have been resolved months ago.
As for the "NOFOLLOW" in comments debate, I have a whitelist of "blessed" sites that's fairly generous. I don't know your site, it gets the NOFOLLOW, I do, it doesn't. Granted, considering that my spam blocking pretty much eliminated spammers from hitting my site, I'm tempted to turn it off.
Still, it's one of the benefits of being at the "Used Slurpee Cup" level of bloggers. I'm just not that attractive a target.
Well... I use the nofollow thing from time to time to link people I don't want to give Pagerank for linking them.
Jeremy,
I just posted about a fascinating study on Splogs and Spings by the ebiquity group at The University of Maryland. Great stuff; they look at weblogs data and conclude some fascinating things about authentic Blogs versus Splogs and also propose a solution.
Peter Abilla
For comment spam I installed the Spam Karma plugin for wordpress and I've been incredibly satisfied with it. Only twice did it make a human enter a captcha so that the comment could be posted (usually the person can post their comment without interference) and Spam Karma has caught all the comment spam so far. It's really a great amalgamation of a bunch of spam identification techniques.
Blogs aren't adding 'nofollow' to the commenter's weblog link are they? It might make sense to add them to links in the comments, but I should still be getting all that glorious page rank to my weblog... right?
You took the words right out of my mouth. I wrote this back in February raising the same concerns:
http://gabrito.com/post/nofollow-hurts-bloggers-more-than-spammers
nofollow was doomed from the start because it makes an unreasonable assumption.
In order for it to work it needs to be taken up by 100% of sites which is never going to happen. Even if just a small percentage don't implement it then it still pays for the spammer to do automated spamming. The returns are less but not reduced completely.
I don't know if anyone specifically predicted how linking would change when nofollow was announced, but plenty of us believed that it would do nothing to stop, slow down or ease the flow of spam. From 17 Jan 2005 on Google implementing nofollow:
"If blogs implement this it will be great, for Google, not for blogs. Because Google will be able to refine their PageRank by avoiding links in comment spam, but it will be useless for blogs because they will still get bombarded by comment spam."
http://joseph.randomnetworks.com/archives/2005/01/17/googles-attempt-to-reduce-comment-spam/
In response to your Comment Spammers Have Blogs of Their Own post:
"Just like the fact that filtering spam won’t stop it from being sent to you, fixing pagerank and other search engine calculations won’t stop spammers from hitting blogs and hitting them hard."
And from time to time since nofollow was put into action I've written about how it has made things worse. One way is the changing of link behaviour, but even if that didn't happen nofollow is still bad for search engines because they ignore the good with the bad. The only way I've seen nofollow implemented is to apply it to all comments. They have a saying for this, throwing the baby out with the bath water.
I'm hoping that at some point nofollow will be turned off by the same group of companies/people that turned it on in the first place. Unfortunately I think that the chances of that aren't very good. Who wants to be the first to admit they were wrong?
Jeremy, thanks for your comments on my post. I don't mean to suggest that every time I post a comment, I consider the ever so slight economic value of linking or not linking. After all, I'm not even checking to find out if your site uses nofollow or not. But what is significant, I think, is that nofollow adds a bit of friction to free discourse on the web. That amount of friction might be tiny (it's certainly less than disabling links altogether, or shutting down comments), but I think it's not so small as to be insignificant. Combined with the fact that nofollow has no appreciable effect on comment spam, I think that's reason enough to drop it -- as I did.
There are cases, where nofollow is good, like this one: http://vrypan.net/log/archives/2005/02/01/why-nofollow-is-useful/
*You* might not think that way, but a surprising number of people claim to.
Akimset doesn't actually stop comment spam. It just puts it into a queue for you to delete later so it doesn't show up on your site. IMO, that still waste my time when I have to swift through the spam and look for valid comments (which has happened on several occasions ).
When nofollow came out, it seemed like a fairly decent thing to have. But most definitely not for stopping spammers. You have to admit that a link is a link is a link. And if a spammer gets it onto your site, then they'll have a slight chance of someone clicking on it and making an extra penny.
You put out a post and in less than one hour you have 15 comments, and you say commenting has slowed down since NOFOLLOW? Yikes, I and many other bloggers would love to get that kind of a comment response.
Uhm, no. Actually what I said is this:
The "psychology of linking" did change in a fairly obvious way after nofollow started.
I didn't say anything about slowing down.
>> Link and be linked to.
Good Advice! Link on, dudes!
>> Let the search engines sort it out.
Ummmm - this part is not working very well ...
Nofollow is just the tip of the iceberg. Until/unless several quality and anti-spam measures improve significantly at the SEs, webmasters will continue to spend too much time trying to optimize sites rather than produce quality content.
Nofollow deflects some comment spam but it also devalues some links that should be considered. As you note people often treat links more as a commercial rather than an idea endorsement. Ironically Google's concern about link manipulation has led to a web where "optimizing" sites and links has become far more important to companies than quality concerns.
The market is helping to shake this out as niche search gets better (e.g. Technorati is better than Google for searching blogs).
I love nofollow. We use it a ton on Slashdot. If you have the karma bonus, you don't get it, so good users can not have it. But more importantly: if the reason you are posting is to get Google Juice, then I do not want you posting. Seriously, just go away. You ARE the noise.
"I'm not sure any of us expected people to ration their links as if they were somehow in limited supply. But it happened anyway."
Maybe it's because I was researching the search-engine optimizers, but I was concerned at the time that nofollow might have rank solution">unintended side effects: "Overnight, a handful of weblog companies have implemented a change that touches the entire Web: How people trade the most valuable unit of currency in the attention economy, the hyperlink."
Sorry, my self-celebratory link was munged because I overlooked your "No HTML" rule:
One other bad thing about "nofollow" is its name. There are lots of people who think (not unreasonably) that it's supposed to tell the search engine spiders not to follow the link. So there are "nofollow"s out there that people are adding for completely confused reasons. If it had to exist, it should have been called something else, like "noendorse".
Typepad user here, I have seen spam plunge in the past 6-12 months. I suspect what's going on here is not a failure of nofollow but rather a failure of amateur blog hosters who don't realize that you need several blades of a swiss army knife to fight spam.
I don't buy into the concept of page rank as "payment" for comments. If page rank is someone's motivation for posting, then their contributions are inherenly less valuable to me.
I for one as a webmaster, find the NOFOLLOW tag very, very useful. The web is NOT made of blogs alone (contrary to some people beliefs)... So, taking aside blogs and bloggers, "ordinary" sites (for a lack of a better word), do benefit greatly. As a lot of you know, search engines place value on who links to you, calculated by Google as PageRank amongst other factors. Since everyone and anyone is free to link to your site, and there's no way to control how they do it, there are no penalties that can be attributed to your site or pagerank if "bad", "spam" or "evil" sites link to yours.
However in the past, you needed to be extra, EXTRA careful on who you linked to, because it could actually HURT your search rankings badly... Suppose you went and done a study on spammers, viagra rings, and the like. If you placed some links there, your site used to went down the results lists as if you had pulled the toilet chain... Now, one can safely link to anywhere, without worrying if the site your are linking to is "good" or "evil", simply by adding a NOFOLLOW tag.
So, to challenge the title for your post Jeremy, I would say what goes wrong doesn't lie on the NOFOLLOW tag, it lies on the way the search engines value links. The NOFOLLOW is actually a way of saving yourself a lot of grief and trouble, in a market where Yahoo and Google dictate the rules for webmasters to follow, if they want to be in their top positions.
I own a number of forum based websites. Most of the links my users post are not spam but references to various products and suppliers. As an example, you can see www.lincolnvscadillac.com - which is an automobile related, forum based website. Many of my users link to a tire supplier, or a parts supplier, etc.
To me, this is EXACTLY where search engines want to be getting linking popularity information from. These are being posted by the general internet public and are, IMO, the best measure of popularity.
Unfortunately many people believe in things such as pagerank leakage and think their site will be weaker the more outgoing links there are, and they seek to stop this leakage.
The other reason, we never know where users are linking to. I cant read every post, I get over 1000 a day across my sites, and I cant check every link. If someone links to a spammy site, or one in a 'bad neighborhood', I may never see it. So I end up in a dilemma. To nofollow or not to nofollow.
Someone should better explain how outgoing links affect a website (especially forums) so webmasters are not so fearful of things like PR leak and being banned for linking to a 'bad neighborhood'.
From the comments above, we could derive a conclusion:
/NOFOLLOW and comment moderation should be mutually exclusive./
Some websites are more valuable than others because their creators enforce quality standards. Weblogs that moderate their comments are more valuable than ones that don't: specifically, their links are more valuable than the links on unmoderated weblogs. Unmoderated weblogs are a perfect match for NOFOLLOW because their links are not required to meet quality standards before being published.
Don't put much effort into spam-proofing that old wiki or blog? Just turn on NOFOLLOW. Releasing a product to be used by people who might feel the same way, or have no position on the matter? Please, turn on NOFOLLOW by default. In the end, it's not a question of NOFOLLOW=GOOD or NOFOLLOW=BAD, it's a matter of using it intelligently.
You might ask, why should we make the effort to use NOFOLLOW intelligently? As Brian Duffy said above:
> What's good for Google is good for us all.
At least until Google turns evil...
Jeremy, I think the way you summed up your post was the exact solution for nofollow:
"Link and be linked to. Let the search engines sort it out."
I can think of at least 10 different ways where search engines' crawlers can take nofollow as a suggestion and add some heuristics that will let them decide whether to take the nofollow as it is or not.
This would essentially make nofollow a suggestion search engine's crawlers which they will either take or not.
I was quite surprised that Google implemented such a stoopid idea.
I pretty much predicted when nofollow was introduced that it wouldn't have any effect at all on the comment spammers. I'm still using an anti-spam plugin for MT. :)
The only thing that is going to stop spammers is legal action. There will always be sites they can spam, and there will always be webhosts they can host their sites on.
yeah, the nofollow thing seems to be a bit counterintuitive
> Unfortunately, that was a downside that none of saw coming back when we announced our support. I'm not sure any of us expected people to ration their links as if they were somehow in limited supply. But it happened anyway.
Actually, it was pointed out almost instantly by Ben Hammersley:
http://www.benhammersley.com/weblog/2005/01/20/let_no_fellow_nofollow_lest_we_all_lie_fallow.html
Similar sentiments were echoed by several other posts written around the same time.
Some of us *did* see it coming.
I think the nofollow attribute will always be usefull. It's better with him than without him. It gives us a chance to select and order what we websites we want to trust with our pagerank.
There's a contention between search engines and website owners since linking became such a major factor in conferring relevance score impact. Once that became a factor, website owners felt that they had the right to control their voting abilities, even to the point of being able to sell votes.
Search engines don't like this because it either seems "unnatural" to them, or because it doesn't seem egalitarian -- page relevance would be impacted increasingly by the weight of money more than the weight of information relevance.
I think that so long as non-linking factors are also used in assessing relevance, you should allow sites to sell links, and still respect the link weighting. We site owners should be able to own our site importance to some degree, and should be able to choose importance conference.
One prime way for you to turn around the loss of value caused by introduction of nofollow tags would be to find a way to give a site a portion of importance weighting for how many valuable outbound links they use. If the site is in a "good neighborhood", isn't using spammish/black-hat practices, and has healthy/nonsuspect linking schemes, give them a bit of a "reward" for the number and quality of outbound links.
This might turn around your problem introduced by nofollow.
Speaking of, I've been opening links from my BlogLines account all day and lots of them have nofollow on the end, making me go up a folder to get to the link. I know BlogLines just did some tweaking, I hope this gets fixed fast.
Then again, maybe it has something to do with WP & FF updates on the same day lol.
To me, nofollow has always been a token attempt to deal with the issue of information pollution of spam comments and a real solution should be able to discern that a link is spam without a blogger adding it to his blogs. If I am a spammer able to hit 100K or more blogs in one go on something like viagra, I make enough money just off clickthroughs that I am going to do it regardless of the PageRank benefits. As many have also pointed out, if you don't lock down 100 percent of the web then the pages that aren't using nofollow throw off the usefulness of the tag. The search engines need to be better at UNDERSTANDING a comment and seeing that something is spam. I can, you can, and everyone who is mad at spam can, so why can't Google/Yahoo/MSN and the other billion dollar search companies?
It also seems fair to me that a commenter who leaves a valuable comment SHOULD be given some link juice for being an active member of another person's blog.
Why?
1. Keywords Bring the Blogger More Traffic: When a commenter adds real discussion to a page, they are enhancing its unique text and possible keywords for the page to be discovered. This means more traffic, visitors, and potential $$$ for the blogger who is allowing comments. Why shouldn't good users be rewarded for their contribution to someone else's site?
2. Blog Owners Need to Police Their Own Land: A blog is a living thing. If you don't want a comment, delete. A blog filled with comment spam (automatic or manual) just shows the blogger isn't paying enough attention. We must all tend our own garden.
3. Blogs Deserve MORE Juice: Some have speculated Google et al wanted nofollow to help push down the rankings of blogs and blog owners in order to allow more "mainstream" sites to appear. Not sure if its true or not, but to me, blogging is the next wave of the web. I find more great things through blog posts then most sites. Everyone is an expert at something, and I think the SE's shouldn't be hindering blog participation and link value.
From what I see yahoo, msn and even google still follow nofollow's and while google may not pass pr, googlebot is definitely following the link.
"I'm not sure any of us expected people to ration their links as if they were somehow in limited supply."
I find this statement hard to believe. People have been hoarding PR for ever; this just made it easier. What I dont think anyone expected was for Google to subsequently come out and say to nofollow paid links. That was an abuse of the trust the other SEs had when signing on to this idea to combat blog spam. Google BUSHwhacked MSN and Yahoo with that stunt.
For the record I said this ...... I pointed out that people wouldn't include links in comments as much since they're punished for doing so.
The other point that nofollow does work on is preventing search spam. While the comment spam still exists it makes building a memetracker or search engine easier.
*The "psychology of linking" did change in a fairly obvious way after nofollow started.*
I'm not so sure, the use of JS on OBLs was just beginning to mushroom when the nofollow was introduced...
I think the "nofollow" gig is a drag for legitimate bloggers, as full participation in the blogoshpere won't help your blog ranking in the same way.
But....due to the spammers and the advent of business blogs, I suppose it was needed. Just look at this page for instance. As of today the PR of this page is a 7. Could you imagine how many Viagra spammers would be hitting this page?
Terry
Not Following ;-)
Hi Jeremy
I was actually just remonded about this post when reading some research someone carried out on how search engines are currently treating nofollow.
http://www.macalua.com/2006/11/03/nofollow-yahoo-still-doesnt-follow/
It seems Yahoo is still indexing nofollow links.
In addition during my research into sites that have links to shared feeds (linkblogs) created with Google Reader, Yahoo seemed to be indexing links to these shared full feeds, whereas Google and MSN were not.
Does yahoo value sans-nofollow links inside of rss feeds?
I've been putting my links into my comments (making them more blatantly spammy) instead of the "weblog url" field, and it seems to be working.
Amazing how such a small element can cause such havok. People are becoming greedier with their links, taking less effort to contribute and build a better internet, and also shifting reponsibility into the hands of "Authority" sites without giving credence that perhaps we are the best ones to determine relevance.
A free web goes much further than letting go of controls, but also being happy and excited to communicate and share with one another.
Yes, "the 'psychology of linking' did change in a fairly obvious way after nofollow started.", and now we've a trend which seems to result in web-wide-nofollow insane. I don't think intended misuse of rel=nofollow (e.g. Wikipedia) is a big deal. Geeks can perfectly apply it to paid links, ads, biased editorial linkage and other cases fitting nofollow's ever morphing semantics. Nofollow-confusion OTOH is a big problem, and it's getting out of control. Gazillions of site owners, publishers and editors out there don't use it where they should, or use it for the wrong reasons, just because they don't understand it. I believe it's not too late for a major redesign of the hapless 'nofollow' crawler directive.
I think the 'psychology of linking' is taken over the web. There is an entire industry growing around the 'psychology of linking.' I think it's quite distracting and, overall, taking away part of what was good about the web in the first place. It seems to me that if people have problems with spam on their blog they will either a) decide they don't care and let it go or b) take steps to stop it. If they go with a) others will see it's not a place for serious conversation and stop coming. If they go with b) they will probably be rewarded for their efforts. Either way, it is sorted out by the web community. No need for these search-engine-imposed rules. Almost feels like BIG BROTHER - IMHO.
Jeremy
Your thoughts on nofollow are interesting. Yes, I agree that commenters must have an incentive to comment. Using nofollow across the board for comments is like bombing a building to kill a mosquito.
Will see more comments from me on your blog. Lets kill nofollow.
BTW, did you notice the 'partial treatment' or shd I say, preferential treatment given to wikia links in wikipedia?
I decided to remove it from my blogs after reading this.
The question is whether you want to support spammers or not.
There are always going to be some.
I use nofollow to not reward the spammers.
You are not really penalizing other webmasters
because only people who care about the subject join the discussion
I'm not sure how rel=nofloow is working in terms of achieving the page rank on google searches. owever, I have seen that if one puts adsense ads on the blog, google ends up ranking your page higher than other pages with higher page rank or even relevance.
I love the article. Some people don't think about the other side of using "nofollow" tags. I am glad you are getting the word out there.
The official claim is that links with the rel=nofollow attribute do not influence the search engine rankings of the target page. In addition to Google, Yahoo and MSN also support the rel=nofollow attribute.
i think it helps indexing
tough topic. i think really, they should just stick with captcha to hold off spambots.
Yea well I pondered about the idea of turning it off and today I decided to do it. My blog now supports No Follow :) Happy linking FTW :P
Didn't have the time to get to the bottom of the comments, but I just wanted to throw in a stir I've recently noticed that "nofollow" links may actually be followed to some extent...
Rand Fishkin (seomoz) touches on it here: http://www.seomoz.org/blog/seven-short-seo-scribbles
Its been a REALLY long time sense anyone commented on this article but I have a question that no one seems to be able to answer.
If you check into blog search software (software that searches for wordpress blogs, tells you PR, tells you if you've commented before, etc.) or automated blog comment software (same as above only it automatically posts comments for you) they all do exactly the same thing. They bring back lists of blog sites that have nofollow enabled....
Now if you are using auto comment software what exactly is the point in making comments on blogs that have nofollow enabled? That would seem counter productive as a spammer. Why spam the sites that AREN'T getting indexed? I really just don't understand that.
I emailed a comment spam software company and they said this: "There are still a lot of blogs with dofollow links. Most search engines ignore such tags. Even if nofollow links are given less weight, search engine bots still follow these links and crawl your website!"
Is this true? If it is true then whats the point of nofollow at all? So it weighs less on those links? So what, that means they just need twice as many links. And if they are spamming that should be easy to get.
I say this because it seems to me the only persons nofollow hurts are the honest ones. Can someone explain this more to me or point me in the right direction?
This site is very helpful for me. This is very attractive for me. *You* might not think that way, but a surprising number of people claim to
Great post, and I agree 100% seems everything is a no follow link now a days. I use to contribute to blogs on a daily basis, 1 it was fun to read about things, I like putting my opinion out there, but at the same time was helping my site. Now with the no follow tag on every blog, I just don't bother wasting my time, and instead use it to enhance my own site, and find good quality links.
