One of the most enlightening experiences I've had in my 5+ years at Yahoo was sitting in on some usability tests. Being on the "watching" side of the one way glass is fun. But it can be particularly frustrating when the application or service users are attempting to use is your own.
Back in 2000 or 2001, I watched a few such tests on specific areas of the Yahoo! Finance site, which I worked on at the time. I walked away from those tests thinking differently about how we build stuff for "normal people" and have been doing so ever since.
I highly recommend doing this if you're involved in building software. You may well be very surprised by what you learn.
I was reminded of this by reading the story of Mac Word 6.0. Under the subheading Learning the Meaning of “Mac-Like” the following appears:
Moreover, while people complained about the performance, the biggest complaint we kept hearing about Mac Word 6.0 was that it wasn’t “Mac-like.” So, we spent a lot of time drilling down into what people meant when they said it wasn’t “Mac-like.” We did focus groups. Some of us hung out in various Usenet newsgroups. We talked to product reviewers. We talked to friends who used the product. It turns out that “Mac-like” meant Mac Word 5.0.
We spent so much time, and put so much effort into, solving all the technical problems of Mac Word 6.0 that we failed to make the UI of Mac Word 6.0 behave like Mac Word 5.0.
I remember the trainsition from Mac Word 5.1 to 6.0. It was, indeed, a painful one for most Mac users. This is worth thinking about every time someone suggests radical UI changes.
Posted by jzawodn at June 22, 2005 04:10 PM
With the latest release of REALbasic (an easy-to-use, cross-platform, object-oriented RAD environment), we completely redesigned our UI based on both new and experienced user testing. The new UI yields quite a bit more productivity (although since it's completely new, there are still a few kinks). Despite being originally designed by several Mac users, and then molded to be more cross-platform, and having direct input throughout development by one of Apple's UI evangelists, we got complaints about not being Mac-like anymore.
I tend to think that a radical UI change often is perceived as a bad thing, until someone really tries to use it. Until they sit down with an open mind, it strikes them the wrong way. For Mac users, this often makes them think, "This software just isn't Mac-like." I've done it myself, so I'm not really blaming anyone. :)
Ah yes, the pain of Word 6.0, I remember it well! I can't remember how many Macs we reverted back to 5.1 on. 6.0 was a hog too, so on our older Macs 5.1 was zippy and an all around comfortable user experience.
Pete's right. What we hated most was how much memory it ate up considering how few of the enhancements we actually cared about. Of course, we all installed it, if only to verify that we could indeed cover the screen with toolbars.
I still have the 13 or so floppies that Word 5.1 for Mac OS live on. Those red floppies with ITS on the metal cover. :)
Ah yes, the pain of Word 6.0, I remember it well!
Ah yes, the pain of Word 5.0, I remember it well!
Ah yes, the pain of Word 4.0, I remember it well!
Ah yes, the pain of Word 3.0, I remember it well!
Etc. etc etc. I'm probably one of the few people who have used every single version of MSWord since 1.0 on both platforms. And they all sucked.
I am astonished at the bass-ackward conclusion of the MS Mac guy. He actually thinks his busted 5.0 version defined Mac-like software. I guess he never saw a real Mac word processor like MacWrite Pro. What he witnessed in user testing was something he only obliquely mentioned, not conforming to your previous version's GUI standards. And users of the old v5 spent so much time learning nonstandard nonmaclike things, and let's face it, if it was truly Mac-like, they wouldn't HAVE to learn the features because their use would be obvious. The v5 users spent so much time learning the non-Maclike features of the existing v5, they stuck with it even when offered "improvements" (that could merely be different, not better).
Heh.
My favorite war story about Usability testing came from a top level property I worked on back in 2000.
"I don't know how I got here, or if I'll ever get back, but if I do, I'll find this information incredibly useful!"
I really need to get that engraved on a plaque to hang in my cube someday and look at it whenever I get the feeling that I've done all I can with something.
All that usability testing on Yahoo! Finance and you still didn't make it so that when you fail to find a ticker symbol you don't automatically do a lookup to help them find the right one? This page is very redundant:
http://finance.yahoo.com/q?s=salesforce
Also, you should work on being able to easily switch users in Yahoo! Mail. Another extra page to navigate what could be done in a single page.
I was once told that in a useability test, when you're behind the one-way mirror getting frustrated at your users, "They can't hear you talk, but they CAN hear you scream!"
Hm, do big companies also offer tours into their Usability labs? I would love to see one one day. :))
My favorite quote from developpers when I show them how their things are broken "User will never use it this way. And they will never complain about it." Exactly. They will just go away.
Hm. A new goal in life. Make a usability lab, get devs in, make them cry. ;))
Hi Jeremy - thanks for the plug for the value of usability testing.
For others who have never been in a usability testing lab and would like to see one, November 3 may be your day.
November 3 is World Usability Day and all sorts of activities are being planned around the world. Many of the events will take place at usability labs and include tours.
See http://worldusabilityday.net/ for mroe info - still growing as more groups announce their plans. About 10 of the 40 locations have blogged.
If you cannot wait that long, then I suggest you track down your local UPA chapter - http://upassoc.org/chapters/ - or local SIGCHI chapter - http://sigchi.org/local-sigs/ - usually the leaders of those local chapters have meetings near usability labs and will give tours.
Keith
"We spent so much time, and put so much effort into, solving all the technical problems of Mac Word 6.0 that we failed to make the UI of Mac Word 6.0 behave like Mac Word 5.0."
Exactly right. I was one of a small number of Mac users who actually loved Word 6 -- and I think that was because I didn't really expect it to be like Word 5, but rather tried to evaluate and learn it on its own merits. It also helped that I actually used the advanced features that it offered.
