Saturday, January 03, 2009

Review of Surf Canyon, a Semantic Search Add-on for Firefox and Internet Explorer  

View Comments

     I have previously discussed several services which attempt to solve some of the limitations of the dominant, generic search engines which most people use. Two of the main limitations are presentation of results, and the difficulty of understanding semantic relationships among related search terms. Surely you have noticed the sort of message which usually appears alongside a generic search engine result page, eg "Results 1 - 30 of about 3,450,000." Perhaps you, like myself have wondered what possible significance or utility could there be in having millions of potential results? After all, you usually just want ONE result; the right one. Research has shown that very few people look beyond the first result page. The rest of those results are meaningless. If the answer isn't found on the first result page, most people will instead begin to experiment with a different keyword or combination of keywords.

     What if the information you want is simply not on the first page? What if it is on the fifth page that you never looked at? In most cases you would never be the wiser. This is what I mean when I talk about limitations in the presentation of results. Technically, your query returned the information you were seeking, but you never saw it. Wouldn't it be nice to find a way to solve that problem?

     Semantic relationship and meaning is an even more complex problem. Take one of the simplest types of semantic challenge, a word which has a number of different meanings. Depending on the context, the word "plant" can mean factory, or a photosynthetic organism, or it can be a verb. Search results should be very different for each of these meanings. Instead, you often get all of them lumped together in one result. This in turn compounds the problem of presentation. As humans, we can quickly grasp meanings of words from their semantic context. Machines have great difficulty doing that.

     Surf Canyon is an add-on for Firefox and Internet Explorer which aims to solve the dual challenge of semantics and presentation. It does this by reconfiguring results from Google, Live Search, Yahoo Search, and Craigslist. Surf Canyon works by a combination of preview, content folding, and machine learning to help you reach deeper and more relevant search result content. By preview, I mean that Surf Canyon looks ahead into subsequent results pages. By content folding, I mean that Surf Canyon provides you with a bull's-eye button which you can click in order to show or hide these deeper results. By machine learning, I mean that Surf Canyon observes and remembers which search results you decide to click. Next time you run the query, Surf Canyon will give precedence to similar results. This should lead to more relevant results over time.

     I decided to test Surf Canyon on Google with the query "solar plant growth." I chose this because such a query could easily be interpreted to mean either the biological effect of solar radiation on plant growth, or the growth of industrial solar power generating plants. Google on its own will give a mix of both results. I wanted to see how well Surf Canyon would perform with it. The first result page included a bull's-eye which expanded like so:


surf canyon solar plant growth subquery1.png


     You can see that in this case I have expanded subqueries which pertain to biological plants. I clicked on one such result, then reloaded the original query. Subsequently, I got this subquery result:


surf canyon solar plant growth subquery2.png


     As you can see, it's still a mixed bag. However, it does contain results of the type which I want (biological). So Surf Canyon has indeed delivered to some degree. I tried another query, this time "nuclear plant growth." Again, this could be taken to mean growth in the nuclear power industry, or it could refer to the molecular biology of growth in the nucleus of a plant cell. I'm looking for the biology result. The first result page gives me:


surf canyon nuclear plant growth query1.png


     As you can see, it's heavily weighted toward the industrial power meaning. But there is one molecular biology result. I click it, re-run the query, and this time my results are much more relevant:


Surf Canyon nuclear plant growth subquery.png


     I have demonstrated to my satisfaction that Surf Canyon does indeed substanially perform as claimed. Additionally, there are a few configuration options that you can tweak to your preference. However, I do also have some criticisms and reservations:
  • Surf Canyon requires a download and installation. It only supports Firefox and Internet Explorer. This approach is a barrier to adoption. In general, I prefer solutions which are Web-based, and as cross-platform as possible.
  • Surf Canyon does not work on Google News, Google Books, etc. I was surprised and disappointed to discover this. Google News will return significantly different results for the same query than the generic Google Web Search will, so for current events Google News would be preferable.
  • Surf Canyon is not as dynamically interactive as some other services. It requires reloading the query to further refine its results. In many circumstances, that will not be much better than simply using Google and visiting page 2, 3, etc or just retyping a different query.
  • Surf Canyon will still return a lot of duplicate or irrelevant results, in my experience.
  • What if my semantic preference changes? How quickly will Surf Canyon adjust? It's not clear to me. I would prefer a way to save a semantic search, and also a way to clear/start over. Update: Mark Cramer from Surf Canyon was kind enough to respond to my question:

         So that you know, Surf Canyon uses semantic “real-time” implicit personalization, which means that it does not learn over time. Therefore, the semantic preferences are very short term. Additionally, we insert a “reset recommendations” link at the top of the page should you wish to erase and start over.
         One of the reasons that we don’t build long-term user profiles is because that’s already being done by Google. If you’re logged in, Google will record your search history and make adjustments to the results. This is completely compatible and complementary to what we’re doing: http://blog.surfcanyon.com/2007/09/19/hold-the-pickles-hold-the-lettuce/.

     In summary, Surf Canyon shows promise as a search tool. It's a beta product which needs improvement. It has a lot of competitors, so my advice to you as a prospective user is to try it and see if you like it. If you like it, great. If you don't like it, keep on looking elsewhere because you have a lot of other options.

Links to this post

What next?

You can also bookmark this post using your favorite bookmarking service:

Related Posts by Categories



blog comments powered by Disqus