Reading about the pending closure of the Yahoo! Photos and Yahoo! Auctions (in US, at least) services leads to an interesting question for those of us in the business of providing free APIs to our services.
What's the right way to decommission a Web Service API?
I suspect that since this is still a fairly new area, there aren't many "best practices" (I hate that term) established yet. And I further suspect that it's a bit more tricky that it might seem at first.
Consider the possibility that your REST-like service offers output in several formats (as some of Yahoo's do): XML, Serialized PHP, RSS, and JSON. And further assume that since the barriers to consumption are so low (no registration required), we can't possibly notify everyone in advance.
Clearly we ought to strive to shut things down in such a way that breaks people's code the least (code that we've never seen and can't influence). One line of thinking is that we pick a date and start returning the equivalent of an "empty set" for each response type. That means anyone using the results in a web site or application will have a chance to notice that something is wrong. But at some point we probably need to shutdown the actual endpoint. And when that happens, it'll generate a 404 error (or something similar) and possibly start to break things.
What do you think?
And, more importantly, is there any consensus on how to future proof libraries and code samples such that they can detect this in the future and fail more gracefully?
Oh, and I have a story about Yahoo! Auctions I should tell sometime soon...
Posted by jzawodn at May 10, 2007 01:23 PM
This raises the question, what will become of the Yahoo! Photos TiVo HME App. Does the company have any plans to integrate that app into viewing Flickr photos?
Honestly? You tell as many people as you can personally. Then you deprecate new users. Then you shut off all users. Takes about 2 years to do it right. If you do it right, there will only be a minor firestorm, if you do it wrong, major.
Just sayin...
Chris DiBona
Is there a mechanism in place for changing versions of the API? If so, this would seem to be a special case - you're "upgrading" the server-side API to a version that, unfortunately, no available client recognizes.
I'd expect a "best practices" server to support two versions (N, N+1) of the client API, with a way to warn the version N clients that their time is limited (when the server moves to N+1, N+2).
Thats what 410 Gone is for, hopefully people are checking for it.
Right. But "hopefully" is pretty easy to refute based on the evidence I've seen so far. :-(
But that's where the future-proofing comes in I guess.
rss feeds? unless they're on a hostname you can simply retire, you just feed 404 or 410 responses forever. there are a surprising number of rss-consuming applications that are too stupid to ever go away.
REST deals with this in a sense. I would consider using a 501 Not Implement instead of a 410 gone perhaps.
Perhaps something on top of it would be good. In the case of photos something that could point to Flickr perhaps. That's almost like a service migration, or major change to the API rather than a shut down. Obviously a 301 wouldn't be appropriate though.
Should probably coopt two status codes - one for deprecating and one for deprecated. Establish those consistently across all properties and work to establish it as a standard.
Presumably, you can come up with some default data to return in answer to the request that gets the massage across. Things like "Sorry, we've gone out of business" if they expect to receive a string, or a PNG image with the same message graphically if they're expecting an image. In other cases, I suspect the 501 or 410 may be the right response. My point here is to create a complete set of data that the users are trying to access with the API that communicates your message and have all calls to the API return that data. People will eventually figure it out.
I have a similar problem at work. I publish scientific papers with URLs to data, code, whatever. A couple of years later the agency Web-Nazis show up and shut down my site because it does not (can not) comply with their rules and mandates. While not exactly the same, it does point to the need to archive snapshots of the web and the technologies in use at the time.
I'd hate to be a historian living 200 years in the future. We'll, I'd be thrilled to still be alive 200 years from now and would probably be happy to be a historian if that is what kept me alive that long - but I digress and I think you know what I really meant.
I think you will need to post a note in the same format the api currently works while still using the status code 200 for a while. I know when del.icio.us moved all of their api's over to secure it took me along time to notice as my script cached the response and tried to get a new version every half-hour. If it was unsuccessful it just used the cache. I only noticed when it was pointed out to me that I hadn't been posting links in a while.
One approach would be to start responding more slowly, increasing the delay by n% a day. With good fortune there will be enough time available between now and "the application is gone" for most to notice.
A related question is whether you can trust web APIs for longer-lived applications (ROI lifetimes between a year and ten years plus). Yahoo may provide the first really good answer to that one with this move and how it's handled - just how far can Yahoo be trusted as a platform to build on?
Why not ease into the blank result sets over a few weeks? 1 out of every 10 blank would alert most app-masters that something is wrong. increase the proportion of blanks over time and then let it run 100% blank for a few months before killing it totally.
I think a combination of the above points might be a good way to go.
I also agree that using a HTTP response code is a good idea. I expect that most people would at least be checking for a 200 response code. I'd be inclined to return a different response code whilst still returning the expected content for a period of time.
After that, I'd probably start injecting some additional data into the responses. If the API will tolerate adding a whole new 'section' then great, if not prepending a warning to the start of every content block is going to be very obvious to everyone.
As soon as consumers start seeing 'this web service is being phased out and on it will cease to function.' in the response - it is going to be pretty self explanatory.
The 'how' of turning it off is probably pretty straight forward, there are more than one set of options available. The thing I'd be more interested is how long you'd be willing to let the service run for whilst being decommissioned. I can only imagine the number of people consuming various Yahoo! web services, so I would have thought that the volume of traffic would play a part in that decision?
Al.
When we killed our old schema API, we went with Chris's 18 month deprecation window, along with a hard core messaging plan. In some cases, we even called people on the phone.
Then again, we actually cared about keeping our customers. This has also been a big internal argument against allowing "anonymous" sign-ups, as there's no way for us to contact then when issues such as this come up.
As to 404 vs 410 vs 501. The users who care about those types of things, and are actually checking return codes and doing things about it, are the users that are going to already know you're deprecating the API. :)
In an utopian web, we could expect every major photo site to have an API. Now,
1) Ask all users to migrate to a photo service like they are doing now.
2) There definitely will be quite a few people who won't do so after the deadline, which is 3 months in this case. Simply migrate these people's photos to Flickr, since you own it anyway, and redirect their Yahoo! Photos login to Flickr.
3) Maintain the API service afterwards and redirect every API request to a the API service of the other services. This will definitely cost less than what it takes to run Yahoo! Photos today, so you could maintain it forever till demand drops to zero. You'd be maintaining your good name for keeping your word at the same time.
Is "possibly start to brake things" a clever reference to stopping the service or simply a misspelling?
Oh gosh! You've become an apostrophe abuser too?
s/Yahoo's/Yahoos/
TIA
Uh, yeah, it is. I registered a false positive. Sorry for the irrelevant noise.
