After spending most of the day on the problem of "how to find the RSS feed (if there is one) given a URL" I've come to the conclusion that it's a pain in the ass.

I have a much better appreciation for the pain that aggregators developers have been thru. And that includes Feedster, Technorati, and others who have the same (or similar) problems.

Ugh.

Posted by jzawodn at August 20, 2003 11:59 PM

Reader Comments
# Scott Johnson said:

Actually sucks is the understatement here. Try it sometime on a Russian or Hebrew blog and you're in for a real treat. God bless the little orange icon.

on August 21, 2003 02:32 AM
# Janne said:

Every blog platform should support the RSS autodiscovery standard:.

on August 21, 2003 02:37 AM
# Francois Schiettecatte said:

Yes they should support the RSS autodiscovery standard, but the majority dont, and when they do they dont always stick to the standard, I have run into three different alternatives. You can also probe for file names, but there is no standard there either, and we have identified 100+ file names for rss feeds. Finally some sites use a specially constructed URL (blogspot), those are good because they are easy to put together.

Finally the fun part is that some sites just return an HTML error page with a 200 status if you specify a bad url (when you probe for file names), so you have to check that the returned page is actually RSS/RDF, which requires a regex from hell.

Expect to put about a week or two of work into this if you want to do this reliably.

on August 21, 2003 03:54 AM
# Matt Croydon said:

...and then sometimes, you just KNOW that the feed is at backend.php :)

on August 21, 2003 04:40 AM
# steve minutillo said:

I've had very good luck with this bit of PHP written by Keith Devens. I use it in Feed on Feeds, and it works great. Enough blogs support auto-discovery now that this works almost all the time, in my experience. Maybe I'm just subscribing to extra-geeky blogs, though.

on August 21, 2003 07:34 AM
# Wes Felter said:

If Webmasters make their feeds impossible to find, that's their problem. I don't see why it should be the aggregator's responsibility.

on August 21, 2003 08:30 AM
# Mikel said:

was this for personal or work purposes? ;)

on August 21, 2003 09:34 AM
# Jeremy Zawodny said:

Yes.

It was. :-)

on August 21, 2003 10:03 AM
# Francois Schiettecatte said:

I really cant agree with Wes Felter, a lot of weblog users have no idea whether they produce an RSS feed or not, but they do want to see their weblog indexed. It is an education issue and as more and more people enter the weblog space, you will find that you need to do more and more educating as these people are not early adopters (read 'geeky'), but users of the technology who dont necessarily understand all the ins and outs of weblogging.

on August 21, 2003 10:59 AM
# Mihai Parparita said:

There's also Mark's ultra-liberal RSS Finder, which seems like it tries a variety of methods:

http://diveintomark.org/projects/rss_finder/index.html

A description of it is here:

http://diveintomark.org/archives/2002/08/15/ultraliberal_rss_locator

on August 21, 2003 11:50 AM
# zombie said:

jeremy, you know what i do? i select part of the blog posting text, copy it to clipboard, paste it into feedster search box, and look for the rss feed url.

unfortunately, feedster isn't as good as google, so this never works from first search. i have to delete some words, and try at least 5 or 6 times, but eventually, i find it ;)

on August 21, 2003 02:33 PM
# Srijith said:
on August 21, 2003 08:04 PM
# Harsh said:

Hi,

I too have tried BlogStreets : RSS Discovery and it works pretty well. Why dont you ask them what approach they have taken towards it

Regards
Harsh

on August 21, 2003 11:30 PM
# Jeff Barr said:

Syndic8 has a search function which will do exactly what Jeremy wants.

More info in my blog entry.

It can be accessed from the site and via the XML-RPC interface.

on August 22, 2003 08:12 AM
# Jan said:

You are 100% right!!!
God bless the little orange icon

on August 22, 2003 10:25 AM
# Carpanta said:

No, it is not hard to discover the RSS feed, just look at the source code of the page and try to find:
-rss
-rdf
-xml or
-backend

It is my personal formula ;D cheers :)

on August 22, 2003 12:07 PM
Disclaimer: The opinions expressed here are mine and mine alone. My current, past, or previous employers are not responsible for what I write here, the comments left by others, or the photos I may share. If you have questions, please contact me. Also, I am not a journalist or reporter. Don't "pitch" me.

 

Privacy: I do not share or publish the email addresses or IP addresses of anyone posting a comment here without consent. However, I do reserve the right to remove comments that are spammy, off-topic, or otherwise unsuitable based on my comment policy. In a few cases, I may leave spammy comments but remove any URLs they contain.