I'm a bit confused by what Jarrett is claiming about Yahoo's RSS stuff:
George has discovered the hidden cost of Yahoo’s embrace of RSS. It only refreshes the RSS feeds it allows you to put on your My Yahoo! page if the source of the feed pings Yahoo’s ping server. (Details in George’s post, from Yahoo! tech support, or on their page for publishers.)
Well, that's simply not true. The word "only" is what makes it false. The pinger simply accelerates the refresh process. There's no conspiracy here.
In reading the note that George got back from Yahoo's Customer Care, I was pleasantly surprised at how they accurately described the crawler refresh. But I must have missed the part about sites being required to ping Yahoo.
Can someone else enlighten me?
If we need to adjust the documentation or clarify things with the CC group, that's easy enough to do. But I first need to understand how someone could come to that conclusion in the first place.
Dave Winer calls this "Yahoo's first attempt at a gated community built around RSS" which I find equally confusing. Maybe the gates are made of one-way glass? I don't see them. Please help me see them.
Posted by jzawodn at July 06, 2004 10:40 AM
Jeremy, thanks for the clarification. I will definitely post a follow up on my blog pointing to this post shortly.
Still, the reason I felt frustrated in the first place is that this is what ping services like weblogs.com were originally intended for--a central way to notify services anywhere that a blog's content had changed, so the blogger wouldn't have to do it himself. The vision was that by setting up a central community resource, the content publisher could perform one simple step and anyone who wanted fresh content could come and get it. With 12 ping services now around, each serving different needs and different applications, it seems that model is being changed and now a blog author is expected to individually ping every consumer of his or her feed if he/she wants the changes refreshed in a timely basis.
I understand that without the pings the crawlers will still come back and refresh the feeds. The problem is that, if you look at George's entire complaint (including some of the previous posts), it looks like the crawlers' unpredictable frequency means that some feeds go stale on Yahoo when their sources have merrily updated away in the meantime. This kind of works against the value prop for incorporating RSS in MyYahoo.
I don't want to make too big a deal of this, and it may be that what George and his commenters are seeing is just a bug or teething pains in the feed crawling process. But the conclusion that these practitioners are reaching is that they do need to ping Yahoo directly to verify that Yahoo reads their feeds.
Spidering (be it webpages or rss or whatever) should not be thought of as a real-time thing. I'm sure Yahoo's many-headed RSSBot has a queue a mile long. I'm sure if it was real-time it would be abused immediately, if not sooner.
Not to put too fine a point on it, but when weblogs.com started to become slow, or timing out entirely on pings due to the growth in the number of publishers using the service, I think the need for a more stable and scalable service became apparent. Whether that means 12 services -- I dunno, but definitely alternatives should be needed and welcome if the solution is becoming more problematic than helpful.
Sorry for the confusion.
And the weblogs.com slowness is being taken care of, slowly.
It would be great if Yahoo would periodically read changes.xml so that people who ping weblogs.com can get the benefit of fast updates from Yahoo.
Over at Ping-O-Matic we've been having periodic troubles with Yahoo's ping interface. I've searched high and low, could you point me in the direction of the best person to talk to about this?
and i'll point out that yahoo! could hook up their service to blo.gs's cloud service (http://blo.gs/cloud.php) and get the pings to blo.gs passed on to them automatically.
Jeremy, see Jim's note, and now do you see why I called it a gated community?
There's been a lot of work done to create open networks for flowing changes. If I'm not mistaken, Jim's service includes weblogs.com pings in addition to pings he's received at blo.gs. So even competitive services are working together. And so far all we've done is spend money (correct me if I'm wrong Jim) no one's making a profit here.
Another question, and please don't take this as criticism -- does Yahoo provide a feed of pings its received, in other words, do you have the equivalent of changes.xml? That would be the really friendly thing to do. I honestly don't know if you do it or not, so consider this a question.
I'm getting a tad tired of this "not invented here" syndrome. Just because someone else is doing the same thing you're doing, doesn't make it bad. Nor does it make it a gated community. If Yahoo is building a gated community, weblogs.com is also a gated community.
Also, weblogs.com should stop telling me to "get a life" when I ping it more than once in an hour. There are plenty of legitimate reasons to blog more than once in an hour, including edits, so stop insulting the users of your service.
Scott, the message was changed a couple of months ago. It now says "Can't accept the ping because the weblog hasn't changed." Hopefully you like that better, but if you don't, be sure to speak up.
I don't know about this problem but I have been seeing the wrong RSS feed on My Yahoo! intermittently the last week or so--I only have one, mine, just to keep an eye on things and sometimes I see someone else's! When I clicked the feedback link, I got nothing, no way to submit a problem report.
I noticed that my two feeds (my blog and our news) were both showing 2 month old headlines.
I tried pingomatic. Didn't update. I pinged Yahoo! directly...still no update a week later and as reported above the My Yahoo page was showing a feed I hadn't subscribed to.
re: http://www.weblogs.com/changes.xml
any reason that doesn't contain the RSS feed URL? Doing so would make it directly useful to an RSSbot, especially given the flaky uptake of feed auto-discovery.
blo.gs also collects updates from weblogs.com, blogger.com, antville.org, and pitas.com (the last two are scraped from html).
the only other service i know of that accepts pings that makes its list of updated sites available is blogrolling.com, but the validity of its xml is very hit-and-miss.
blo.gs does not make as much on its google ads as it costs for hosting, although the gap is fairly narrow, and the cost is modest enough that i've never really worried about it.
eric: the blo.gs version of changes.xml includes feed information. (since weblogs.com does not collect that information, it may not be in a position to provide it. but it does have a second ping service for rss feeds.)
What about dynamic feeds (like Feedster's) that effectively can't hit a ping server? The test feeds that I run through MyYahoo RSS don't update as one would expect from the prior comments/replies and of course we would like them to. Suggestions welcome.
