Challenges of Search
I just returned from a conference in Rome where I presented a session on search. The basic premise is this: Search is not a utility. Search is an application. Search needs to be thought through and integrated into the process that it is meant to support.This does not mean that there is no place for basic search – the plug and play utility model that tools like Google Search Appliance leverage. In that case, search provides a valuable function in helping people access large stores of unorganized content.As much Google bashing as I do, I am a frequent user of Google Desktop. Hypocritical? I don’t think so. GSA is appropriate for what I use it for – searching through email messages and my hard drive for certain types of information. Sometimes I find what I am looking for and sometimes I don’t. But this is because of the relative effort I place on organizing my content versus the time it takes to do so. It’s easier for me to search as I do and risk not finding something than it is for me to organize all of my email. On the other hand, I have a more structured method for the information that I place higher value on – proposals, SOW’s, client project documents and conference presentations.
This is the key – the relative value of information dictates how much energy should be put into organizing it.
This slide illustrates that for certain content, the business value of that content dictates how much energy we put into tagging, editing and reviewing it. This adds cost to the content. In other words, there is no free lunch.
Specialized Search ApplicationsThis leads to the question of how search can support access to high value content. One example of this relates to product data. On e commerce web sites, faceted search or guided navigation gives users tremendous discriminating power to find what they want. Sites like PC Connection and Wineaccess.com illustrate examples of faceted search – the leveraging of metadata. In the first case, one can segment product listings by type of product (accessories, laptops, desktops, servers), speed of processor, size of hard drive, amount of ram, etc. In the second, facets allow for precise access to wines of a certain grape type, winery, vintage, price, region, style and country. One of the powerful aspects of faceted search is the fact that at each node, the number of results are presented so a user knows what to expect. This is the old “advanced search” that no one would ever use – but now it actually appears as navigation and so people use it readily and prefer it because of the power it affords for users to find precisely what they need.So why doesn’t everyone use faceted search? The problem is one of tagging. Going back to our continuum of content value, better tagged content is more expensive. In an e commerce example we can justify this tagging since users will likely buy more stuff when they can get to what they want. But in some cases, content may have been created over a longer period of time that cannot be “retrospectively indexed”. In this case, some of the newer search tools can perform “entity extraction” and present facets to users. But this requires that unstructured content be at least somewhat structured. Is this another contradiction? Not really. Think of the difference between an email message and a news story. Email is notoriously poorly written – it can be about many different topics and is frequently dashed off without a second look.A newspaper article on the other hand, adheres to strict standards for content and structure. Since the article contains more structure than the email message, it will be easier to derive metadata and therefore extract meaningful entities.The challenge of fully leveraging search lies in correctly applying and exposing metadata – the metatadata explicitly applied to content through structured or unstructured tagging – as well as metadata that is implied or derived from the content source and inherent structure.
All search tools leverage one or both of these types of metadata. The key is to understand context and user tasks. The same key to any application. Search is not an appliance, though it is treated as one. It has to be considered as part of a well thought out application.