The Good, The Bad, The Ugly of Internal Search

At Semphonic’s X Change conference within the internal search huddle someone raised the question, “Why would I not want people to search?”  The question was asked in a puzzled tone as if she sure the question itself was an immediate      end.  How could there ever be an instance where an internal search could be bad?

There’s a commonly held misconception that an internal search is always a good thing.  That it’s a positive step on the way to a success.  People blindly accept this notion and move onto analyzing internal search terms, where they suspect the real substance is.  And who could blame them with industry gurus telling them that this single action alone with tack on 10K to their salary within a year (pardon my acronym but -- wtf?). 

This blind faith can be incredibly detrimental to the success of your internal search analyses.  The set of internal search terms your tool collects contains limited information.  There are many instances where the same term is one site users key to success and another site users plea for help.  Understanding which is which and when you don’t want people to be searching on your site you need to get down and dirty in your tool.

First things first you need to establish a baseline of how many visits include a search and how many searches there are a visit.  At this point, regardless of if 2% or 98% of visits include a search, you should have no inclination whether a search is “good” or “bad.”   

Next you should gain understanding of where in a visit a search fits (this paragraph, and most of the remainder of the blog assumes that your internal search tool is available on all or most site pages).  A page from and page to report only shows you one step in each direction, but that alone should be revealing.  Things to notice are if people are mostly coming from home, key site routers, other products, etc. and if people are going to products, routers, exits, etc.  Often grouping pages by content categories will make understanding where search fits into your site easier.

Getting a baseline for how all your site visitors as a whole are utilizing search is the easy part.  Where you have to stretch your mind and your tool is with segmentation.  More often than not different groups will use internal search very differently. A common dichotomy is between first time visits and repeat visits.  Sometimes first time visits try to avoid internal search like the plague, while repeat visitors are drawn to it like a magnet.  Sometimes its visa versa.  Usually it’s way more complicated. 

To try to determine if you want your first time visitors to be searching start by building a segment.  Then utilize that segment to pull the same data you just did for all site visitors.  Key things to compare are the percent coming from the home page and the percent of exits from the search results page but really you should compare it all and see if anything sticks out.  Then take it one step further and build segments of first time visitor who searched and those who did not search and check out conversion rates of each.  You might want each segment to be only of those who saw 2+ pages or some other basic engagement metric filter out single access visits and balance out the fact that someone who is searching is in some way engaged with your site.

You will also want to go through a similar process with 2+, 10+, 20+ returning visits visitors to compare behavior the searching behavior of multiple groups.  This (hopefully) will give you some perspective of if you want everyone to be searching right away, if its more effective to have your new visitors to navigate via links while returners utilize search immediately or some other variation.   

You, knowing your site better than I do, might see similar population splits as new visitors vs. returning visitors.  These might deserve a higher priority than understanding what role in the conversion process search plays for new and returners and in that case apply a similar method as above to the two, three, four segments you have in mind.

And if your shaking your head and mumbling that you don’t have the time for a sizable analysis proposed above try a single short one. Although building a segment that breaks out a group of users who had an internal search and a site success is self selecting, this is sometimes and good way to get suggestions for what might be working.  By seeing when and why this group is searching you can prevent the potential mistake of saying internal search is always good.

So why would you not want people to search?  I bet you can come up with a reason or two.

Comments

[Daniel Shields]

I'm absolutely loving this. Great topic, awesome timing, and graceful contribution.

There is so much you can get from search. Avinash had a point in saying what he stated. I think the appeal for readership was made to the ego.

Its really important for people to realize the contextual implications as part of their analysis of this data. Its that thing that Judah always pounds the table about: Data without context. What good is it if you can't boil the fluff out of it.

So...about a month ago Google decided to make the announcement, and rightly so at the eMetrics Summit in DC, that they were adding search measurement functionality to the analytics solution. Now, everyone in the world has the ability to get good data to feed out. How many other huge, universal areas of site analytics will it take before people pierce the veil of silence and start talking about what they find? Will they heed warning after warning about a cautious and calculated approach to making changes? How many huge ugly search buttons can we expect to see? Will engaged search become a new consideration?

I'd love to hear thoughts from you or anyone on this.

I can't figure out if people aren't getting valuable insights to contribute to a larger understanding or if practitioners actively choose not to participate in enriching the practice pool.

[MitchellT]

Well, its been six months now since this post. I'd like to know why we wouldn't want visitors to search on our sites?

The only reason I can think of why you wouldn't want people to search is if your onsite search engine was poor (and that's the case on many, many sites!).

I'm amazed this topic hasn't generated more traffic, as internal search data can be a goldmine on many sites.

[Jesse]

MitchellT,

Thanks for your question; they are always appreciated – especially the good ones.

Thinking up hypothetical scenarios where you would not want people to search isn’t that difficult of a task, the challenge is figuring out what scenarios apply to your site.

It’s (almost) never the case on sites with average or above average internal search tools that an analyst can make the sweeping generalization that internal search should never be performed by anyone. When you segment your traffic , however, behavioral differences sometimes reveal instances where the act of searching by a specific population segment is clearly not a good thing. A real world example that comes to mind is a commerce site with a large number of products. On this site returning customers were more likely to have a purchase if they utilized internal search, but prospects were a different story. Prospects with two or less visits who utilized internal search had a significantly worse conversion rate than a comparable group of prospects that didn’t touch internal search.

Other examples of when a specific population utilizing internal search is a negative thing are seemingly limitless. It is conceivable for internal search to make certain population segments consume more content, less content, sign up for newsletters less frequently, buy less, send few lead gen emails, etc.

For people with the tools and resources its important to take a step back from their direct internal search data and consider if all internal searches are helping people progress toward a site success.