I've had a tab open in my browser pointing at Tom's Native to a Web of Data slide for a few days now. It's been nagging me. I needed to do something with the ideas encapsulated in that one slide, and tonight it struck me while I was doing something completely unrelated.
Though I wasn't at his presentation and haven't talked with Tom about this, I've decided to annotate (some may say "translate") them for the benefit of the typical MBA laden, non-engineering focused Product Manager that you might find at a large Internet company... or anyone else who cares, really. :-)
Look to add value to the Aggregate Web of data
As a company with infrastructure that can scale to scan, retrieve, and analyze a significant portion of all the public on-line information in the world, think about how you can use those capabilities to improve the world. What can you do that someone looking at a much smaller set of the data cannot? What patterns can be found? What connections can be made? What can you simplify for people?
Build for normal users, developers, and machines
Make whatever you build easy to use, easy to hack, and make it emit useful data in a structured form. That means you need a usability geek, an API geek, and probably an XML/RSS/JSON geek.
Start designing with data, not pages
Figure out what data is important, how it will be stored, represented, and transferred. Think about the generic services that one can build on top of that repository. Only then should you get the wireframe geeks and/or the photshop geeks involved.
This is scary, because you won't have a mock-up right away. Your PowerPoint presentations will look as if they're missing something. But that's okay. This is about doing some engineering style design before product design and interface mocks.
Identify your first order objects and make them addressable
Figure out what your service is fundamentally about. If it's a social shopping application, you're probably dealing with people, items, and lists of items. Nail those before going farther. And make sure there's a way to access each object type from the outside world. That means there's a URL for fetching information about an item, a list, etc.
These are the building blocks that you'll use to make more complex things later on. Hopefully others will too.
Use readable, reliable, and hackable URLs
If the URL is hard to read over the phone or wraps in email, you're not there yet. Simplicity and predictability rule here. Consider something like http://socialshopping.com/item/12345. You can guess what that URL does, can't you?
You may not grasp how important this is, but don't let that stop you from worry about it. This stuff really does matter. Look at how most URLs in del.icio.us are guessable and simple. Mimic that.
Correlate with external identifier schemes
Don't go inventing complete new ways to represent and/or structure things if there's already an established mechanism that'd work. Not only is such effort wasteful, it significantly lowers the chance that others will adopt it and help to strengthen the platform you're building.
You are building a platform, whether you believe it or not.
Build list views and batch manipulation interfaces
Make it easy to see all items of a given type and make it possible to edit them as a group. Flickr does this when you upload a batch of photos. Search, in its many forms, is the classic example of a "list view."
Create parallel data services using standards
Developers (and the code they write) will want to consume your data. Do not make this an afterthought. Get your engineers thinking about how they might use the data, and make sure they design the product to support those fantasies. Again, always default to using an existing standard or extending one when necessary. Look at how flexible RSS and Atom are.
Don't re-invent the wheel.
Make your data as discoverable as possible
The names and attributes you use should be descriptive to users and developers, not merely a byproduct of the proprietary internal system upon which they're built. This means thinking like an outsider and doing a bit of extra work.
Posted by jzawodn at February 16, 2006 09:14 PM
some of the accepted canon i universally reject, starting with this: web APIs.
they are a waste of time and money beyond maybe helping other internal developers work with your code. otherwise, a waste of time and money.
no one is making real money with APIs. no one even really knows *how* to make money with APIs. in fact, most sites, including Y!, will shut you down should your use of APIs create too much load. so what is the point? no one knows. unless your site is arstechnica or oreilly.com, your core audience is not developers, currying favor with this crowd is not going to make you money. if you want to tell me its some good PR, fine, but i offer that even that has worked itself out now.
contrary to belief in the web2 world, you can build a wildly successful web business without engaging any of the core principles of the web2 crowd. case in point: myspace. $400 million dollars for a site that has some of the most laughably ass-tastic design principles on the web and from what i can tell, no API support whatsoever.
You can build wildly successful businesses without any APIs, but I can list a number of top 10 sites that make a lot more money today because they have APIs.
1) eBay: 45% of eBay.com listings go through tools hooked up to eBay using Web Services.
2) PayPal: They do a heck of a lot of external partner commerce using their payments APIs. For example, Dell, Apple iTunes, Go Daddy, etc.
3) Salesforce: There are a number of key applications that integrate with their CRM. I saw a figure like 50% of traffic, but I can't find the interview right now.
4) Google: Serious AdWords buyers use the API.
5) Yahoo: s/AdWords/Overture/
Please notice that I did not say "maps" or "widgets" or "affiliates" or "rss".
Of those, I find affiliates to be the one that's probably next in line to join the list. I know eBay derives a lot of values of API-enabled affiliates, and I bet Amazon does too.
I will let others jump in and defend "maps" and "widgets" and "rss". :)
I think you're *seriously* underestimating the internal value of having sane, supported APIs so that other groups can use your services.
Is there anything I wrote that implies external use?
adam trachtenberg, i concede your points, those are good examples. i still hold my points though with regards to many of the y! properties that i feel pointlessly deploy apis.
jeremy, in my post i specifically mentioned internal use as a viable target for apis, in fact in y!'s case i think they are the only viable use.
Nah, I just didn't have time to write the long version. :-)
Jeremy, it's great to see your annotations. Like a few other people (including one from Yahoo!) I saw TOm's slides as a great presentation of some of the key Semantic Web ideas, without going into the exotica of specific technologies.
In response to grumpY!'s comments, Web APIs (or rather exposed services and data) are *critical* to the future progress of the web.
"no one even really knows *how* to make money with APIs" - sure, but no-one knew how to make money with regular web pages until the web came along. Provision of data and service endpoints means that the network effect can kick in, but business models will have to change from a centralised data/service model to one that is distributed. A lot of the mechanics and data can become commodity, with individual companies offering facilities built on top of the Web as Platform, not just mimicking traditional desktop/LAN architecture.
The Web of Data Tom describes is what enables the web to be treated as a platform on which services can be built (and going back to my first point, Semantic Web tech is designed for exactly this context).
I was at the conference and liked Tom's presentation a lot. However, I disagree with the point of "Start designing with data, not pages."
The first goal of any web application is to gain users, not to deliver content to search crawlers and data access to plug-in / add-on developers.
From Google's or Yahoo's point of view, it's great if there's a website out there with easily accessible, addressable data items that one could crawl. In the long term, this will give your web app more traffic. However, delivering data to Google and Yahoo should not be your first concern.
Likewise, it's great if you have a good API, but developers will only use it if you can prove that:
(a) whatever you provide will profit them in terms of traffic or usability (Google Maps API model), or
(b) whatever you provide will help them spend or make money (Google Adwords API, PayPal solutions)
While it's good to create goodwill, (a) doesn't help you much in generating profits. (b) is substantially better, as it also helps add to your bottom line. However, before anyone starts using the APIs that fall under category (b), you have to have users who are willing to spend money or eyeballs.
Therefore, you have to focus on the user and the user experience right from the start.
I see a lot of value in thinking about your first order objects very, very early. But Tom put it as thinking about data only, and "adding on" the user experience later. In my opinion, that's wrong. You've got to think about the user right from the start. And besides, what would I do without my beloved mock-ups?
"Start designing with data, not pages"
Oh God YES!!! First understand the schema.
Keep the "artistes" out of the mix until you know how and why it does what it does. Graphic designers are great at solving graphical problems, generally the are terrible at solving logical problems.
Ex-engineers make MUCH better PM's than MBA's. (Plus they know when engineering is blowing smoke up their asses and padding the schedule).
there seems to be an implicit conclusion in some of the comments i am seeing here.
there are web services and there are web sites. services truly benefit from apis and a data-driven approach, since their primary interface may be through an api. in adam trachtenberg's example we see payapl. likely very few users of paypal care what the paypal site looks like as long as their cash arrives, so it is a service first and site second.
in my own example, myspace is clearly an instance where it is a site, not a service, apis of are of little value because users derive utility from the specific interface that myspace provides.
perhaps reasoning like this will help developers understand in the future where apis and data-driven approaches really add value, but as gabor says, it would be incorrect to apply the same design principles to entities with vastly different goals and uses.
Tom Coates' presentation brings up some interesting thinking, but I'm having trouble establishing if there's any genuine new wisdom to it, or who it's directed at. Looking at the first three items I'm surprised that his presentation generated such noteriety (maybe it was all in the delivery):
1) LOOK TO ADD VALUE TO THE AGGREGATE WEB OF DATA
Is there a rule of karma in the new Web 2.0 paradigm that if you make your data more addressable to external sources good fortune will surely come to you? (revenues will make it worth the extra effort to implement)
This seems to present an unnecessarily narrow mindset for entrepreneurs looking for new web based business ideas. Are all the data generation opportunities now taken so the only good opportunities remaining are metadata analysis? Perhaps this is the mindset of someone who lives on planet Yahoo! where the world may revolve around building on Yahoo!s IP and userbase. However the suggestons seem more limited in value for the wide world beyond.
2) BUILD FOR NORMAL USERS, DEVELOPERS, DEVELOPERS AND MACHINES
Do they still give brownie points out for stating the obvious? Are there really web professionals still around who haven't discovered and recognized usability?
3) IDENTIFY YOUR FIRST ORDER OBJECTS AND MAKE THEM ADDRESSABLE
Again this seems to be the Yahoo! centric world. There are surely many holes in the market to sell products and services where breakeven isn't dependent on making your product or service addressable. It's cream on the cake worth considering once higher order priorities are achieved.
This all seems like convenient preaching from an employee of a company who spends their days working out new ways to make money from data-mining, and who finds their ideas jinxed by a tower of meta-Babel where there is no convenient uniform API.
USE READABLE RELIABLE AND HACKABLE URLS
Yes - that's right, distract your development team and delay deployment by having designers think through how other people can data mine your website.
Is there something new to learn here? I was hoping to find some gems of wisdom from the brains at Yahoo! useful to web services product managers in the world outside.
IMHO What's important is get to market first and fast, ship with the bare minimum (and carefully research what this is), prove your concept, stake your claim, win mindshare and build your brand. Later once you're established raise more capital and reinvest to build out from the minimum. Maybe building APIs and addressability is top of the list at Yahoo!, in the rest of the world of web services it can be at best a dangerous distraction and at worst the start of a precarious slide into unproductive effort and financial oblivion.
When I said "think about the user experience, too", I didn't mean "hire artistes and let them go crazy with Photoshop". :-)
I meant "Think about what your pages should look like and how the user will interact with them". This can be done on paper, if you are so inclined.
Understanding what you want to display, where, and how, will help you design your data model.
That being said, I think my argument only applies to "surface innovation" websites, as defined by William Grosso. There are basically "better event calendars and better blog aggregators," not new, deeply innovative websites or types of applications. Tom's suggestions may help you creating deep innovation, as thinking about data only liberates you from having to think about how to present it.
"myspace is clearly an instance where it is a site, not a service, apis of are of little value because users derive utility from the specific interface that myspace provides."
Ha. A while back, MySpace had no feeds. I made a tool that scraped MySpace's HTML and produced feeds. Now MySpace has (crippled) feeds, but meanwhile hundreds of people started using my scraping tool, and asking me for things like the ability to put their LiveJournal or Blogger feed into their MySpace blog, the ability to do the opposite, comment feeds, and pretty much everything else that would belong in a MySpace API.
Users don't care about APIs until they want to do something you didn't consider. So as long as you're omniscient, you're right that APIs are unnecessary.
Simon also has notes
from the Coates pres (and others).
Bleh. No HTML.
Coates: http://simon.incutio.com/notes/2006/summit/coates.txt
Others: http://simon.incutio.com/archive/2006/02/08/summit
Hey Jeremy, thanks for the annotation! There are some bits that I think I'd respond to, or want to clarify, but generally they're only bits where I think my slides don't represent strongly enough what I said on the day. I might have to write the damn thing up. Very annoying.
I've been really enjoying this thread too though - particularly in the slight collision between people who disagree with great chunks and those who say that it's all completely obvious. That alone made me smile a little!
But really - as I said in my thread - I'm not claiming to have said anything revolutionary. I think it's more about giving developers an argument and some ammunition to use to convince other people about stuff that they already know (at least mostly) to be true.
"Build for normal users, developers, and machines" is possibly the most obvious, yet missing part of any Web development. Great presentation, great added value. You Yahoo! guys really get it!
Well, ol' Tom is really layin' down the "Web 2.0" version of the law! I'm impressed that he did it in 2006, given that this stuff's been around since dirt was invented:
"Universal Resource Identifiers -- Axioms of Web Architecture"
by Tim Berners-Lee (inventor of the WWW):
http://www.w3.org/DesignIssues/Axioms.html
and REST (REpresentational State Transfer):
http://rest.blueoxen.net/cgi-bin/wiki.pl
And he gives it to us in our choice of favorite standard wWW formats: either a honkin' big 16MB PDF file or 60-odd giant .PNGs, each which has about 30 bytes of useful information. Ol' Tom's really big at makin' things _discoverable_, y'ah see! Nothin' quite as discoverable as a 600Kb .PNG of some letters scrawled on a blackboard, eh?
Yes, I agree. If this were an auditory blog, you would have surely heard the "dry humor" tone in my voice as I spoke that in after the fact jest.
