In Search as a Dialogue, Greg says:
For example, let's say I'm trying to find discussions some of the topics covered at Foo Camp. I might start by searching for "foo camp". Not satisfied with those results, I might change it to "foo camp blogs". That doesn't get me what I want. I try "foo camp web feeds". And so on. I'm repeatedly refining my search query, trying to find the information I need.
But current search engines ignore this stream of related queries, this dialogue, instead treating each search as independent. There is an opportunity for techniques that focus explicitly on this kind of refinement process, using all the information to help you find what you need more efficiently and reliably. Personalized search is one of these techniques.
No kidding.
I'm sure we've all had that experience of haphazardly refining a query. It can be frustrating. But what was even more frustrating for me back when I worked on building related search a year and a half ago.
Part of that involved reading search query logs. What I quickly noticed is that you can clearly see when that's happening. While the data was interesting in an academic sense, it was more frustrating than anything else!
Why?
Because as a human I usually could tell exactly what someone was looking for by the second or third query. But there were times when it took them far longer than that to find the result they wanted (or they gave up).
Heck, I even see this in the logs for the little search box on the front page of my blog. And I've seen less experienced users fumbling to find something in Google or whatnot, often running half a dozen queries before realizing they might be doing something incorrectly. But at no point does the software really help them.
What this all tells me is that search is a skill but it really shouldn't be. The Microsoft research is shining a light on this fact. Our software needs to work harder to pay attention to and react to what we're doing--especially when we're failing!
If only the search engine could stop after a few tries and say, "hey, I'm guessing that you're looking for something like..." You know, just like any reasonably bright librarian might. (You do remember libraries, don't you?) Yeah, it'd probably freak some people out, but what if it actually was helpful?
Amazon's A9 is an interesting step in evolving search, but it really seems to be going in a different direction. Rather than making search a "lean and mean" operation the way that Google had, A9 is trying to make searching the web a different kind of experience. They're encouraging exploration while also trying to tie in your previous behavior (past queries).
Posted by jzawodn at September 17, 2004 12:44 PM
AlltheWeb does this to some extent. I don't think that it follows the sequence of search terms, but it does suggest refinements. It used to do clustering which was also pretty useful (although it was hidden at the bottom of the page where most people wouldn't find it).
Hmm... I wonder who owns that search engine...
Kind of like that MS Word paperclip helper but for search.
An interesting observation. I was thinking of something similar to address the inability of search engines to incorporate direct user feedback. For example: Did the user skip some of the search results - perhaps those results are less relevant? Did the user stay at one of the results for a prolonged period of time, actively reading the content (indicated by mouse/keyboard activity) - perhaps that result is more relevant? Shouldn't that information be propagated back into the search indices, along with all the key word and inbound and outbound link reference counts?
There is a strong need to incorporate these indicators and others to enable search engines to understand the user before they even enter a search term. This is especially true given the exponential growth in available information - growth which is quickly rendering most search results useless (search engines, one must add, that fail to reach the deep web that stretches into corporate databases with localized web search interfaces). I wonder: could a user's social network, ethnic, cultural or educational background assist the search engine understand the context surrounding a set of search terms? If, for example, a user is searching for the term "Java" and the search engine knew that the user's friends had all executed searches for Java and eventually settled on results with a limited set of terms related to the Java computer languages, wouldn't it make sense to use that information in some way?
The distributed nature of the information required by a search engine to truly understand the context behind a user's search query, coupled with the need to diffuse the risk of having such a critical tool in the hands of a few companies, suggests to me that there might be a need to shift towards a P2P-based search engine approach.
For me, the perfect personalized search comes down to this: Software guesses what I'm searching for based on the last interactions with it in a certain timeframe (that is, the last 10 searches max, or all searches from today).
But NOT based on what I searched for last year (as A9's method kind of implies).
I think this approach is (1) better and (2) much less intrusive when it comes to privacy.
(Full post on my blog at: http://www.buzzhit.com/2004/09/search-engines-that-talk-back-jeremy.html)
Jeremy's thoughts on search as a "skill" are interesting. His conclusion, based on his considerable knowledge, is that search shouldn't have to be a 'skill'; it should just work.
This is an interesting observation, especially in light of a recent post on Yahoo!'s Search blog regarding the daughter of a Yahoo employee using search to find her first car. I commented on that here (http://www.buzzhit.com/2004/09/required-education-for-todays-youth.html), essentially saying that search is a skill that all youngsters (heck, everyone) will need to have in the future...
take a look at Dowser. "It clusters results from major search engines, associates words that appear in previous searches, and keeps a local cache of all the results you click on in a searchable database."
