Web Analytics Tool Time

Connecting the Dots or Coloring by Number

Whenever we take on a new employee, an internal battle of epic proportions ensues within me.  This battle does not feature Custer, Germans or Russell Crowe, but instead two pre-school concepts.  Do I give a new employee the skill to connect the dots in order to see the whole picture or just have them color by number? 

In the second scenario the newbie doesn't need to know what the picture is of to be productive, but you can also expect their learning process to be significantly slower. So when you take on a new employee what training technique or mix of techniques should you undertake? The easy answer seems to be to connect the dots before setting them free, but that is easier said than done. 

Calling the process of teaching a newcomer to web analytics all the pieces necessary to do a good analysis is time consuming would be a massive understatement.  And, I'll go out on a limb here, your probably already busy with your own web analytics responsibility.  I will not claim to have perfected a seamless training plan, but in my experience I have found that starting with a simple, quick connecting the dots exercise can have great benefits.  This exercise is best suited for a computer with a multiple monitor setup (the more the better), but its worth doing even if that is not available. 

Start by pulling up instances of the source code of a tagged web page, the web page itself (possibly with a data overlay if your analytics tool has one), the online interface of your web analytics tool and the Excel reporting tool (if your tool has one).  For bonus points you can also pull up an instance of the site with a packet sniffer tool overlay, like Page Engage or Charles.

To begin go to the site window and physically do something on the site.  Start simple with a something like a link click or search.  Then work backwards to the source code and painstakingly slowly point out what in the source code is pushing out the web analytics data.  Especially if your trainee is completely new to web analytics, don't expect much to about source coding to stick.  Even if only 5% of the words you say regarding source coding actually resonate, it is still worth both of your time to lay the foundation. 

Next leap forward to your web analytics tool and show exactly where the data you just generated is located.  Presumably you have already introduced the interface of the web analytics tool to your trainee, but if not this is a good point to walk them through the basics of navigation.  When showing him/her where the metric you generated is located go slow, step by step through the interface hierarchy.  Don't just click through to the metric, but explain why the metric is hidden behind each layer of the hierarchy. You'll probably sound silly to yourself and possibly your trainee as you go layer by layer, but this should not only make the metrics location stick but should allow the trainee to reason through the location of future metrics.

Finish off by building an Excel data block that pulls the metric that you initially generated with your on site action.  Obviously if this is your trainees first exposure to the Excel integration tool that should be an introduction to how to build data blocks as it is how to get the specific metric. Focus on explaining the difference and similarities of navigation to the metric in the context of the web interface. After going through this process with one metric, step up the difficulty ever so slightly do all the same steps again with a prop/evar/event/custom metric or something along those lines.  As you go through the process make special notes about the significance of the prop/evar/custom metric and why the metric you're using was tagged that way.  Metric three should be something like a campaign or product purchase.  I like to conclude the lesson by letting the trainee go through the steps on his/her own.  Like fiber, appropriately timed self guided learning has strong digestive powers. Sure this exercise does nothing teach a newbie the basics of doing an analysis or building a meaningful report, but even da Vinci leaned how to connect the dots before painting the Mona Lisa or something like that.

The Good, The Bad, The Ugly of Internal Search

At Semphonic’s X Change conference within the internal search huddle someone raised the question, “Why would I not want people to search?”  The question was asked in a puzzled tone as if she sure the question itself was an immediate      end.  How could there ever be an instance where an internal search could be bad?

There’s a commonly held misconception that an internal search is always a good thing.  That it’s a positive step on the way to a success.  People blindly accept this notion and move onto analyzing internal search terms, where they suspect the real substance is.  And who could blame them with industry gurus telling them that this single action alone with tack on 10K to their salary within a year (pardon my acronym but -- wtf?). 

This blind faith can be incredibly detrimental to the success of your internal search analyses.  The set of internal search terms your tool collects contains limited information.  There are many instances where the same term is one site users key to success and another site users plea for help.  Understanding which is which and when you don’t want people to be searching on your site you need to get down and dirty in your tool.

First things first you need to establish a baseline of how many visits include a search and how many searches there are a visit.  At this point, regardless of if 2% or 98% of visits include a search, you should have no inclination whether a search is “good” or “bad.”   

Next you should gain understanding of where in a visit a search fits (this paragraph, and most of the remainder of the blog assumes that your internal search tool is available on all or most site pages).  A page from and page to report only shows you one step in each direction, but that alone should be revealing.  Things to notice are if people are mostly coming from home, key site routers, other products, etc. and if people are going to products, routers, exits, etc.  Often grouping pages by content categories will make understanding where search fits into your site easier.

Getting a baseline for how all your site visitors as a whole are utilizing search is the easy part.  Where you have to stretch your mind and your tool is with segmentation.  More often than not different groups will use internal search very differently. A common dichotomy is between first time visits and repeat visits.  Sometimes first time visits try to avoid internal search like the plague, while repeat visitors are drawn to it like a magnet.  Sometimes its visa versa.  Usually it’s way more complicated. 

To try to determine if you want your first time visitors to be searching start by building a segment.  Then utilize that segment to pull the same data you just did for all site visitors.  Key things to compare are the percent coming from the home page and the percent of exits from the search results page but really you should compare it all and see if anything sticks out.  Then take it one step further and build segments of first time visitor who searched and those who did not search and check out conversion rates of each.  You might want each segment to be only of those who saw 2+ pages or some other basic engagement metric filter out single access visits and balance out the fact that someone who is searching is in some way engaged with your site.

You will also want to go through a similar process with 2+, 10+, 20+ returning visits visitors to compare behavior the searching behavior of multiple groups.  This (hopefully) will give you some perspective of if you want everyone to be searching right away, if its more effective to have your new visitors to navigate via links while returners utilize search immediately or some other variation.   

You, knowing your site better than I do, might see similar population splits as new visitors vs. returning visitors.  These might deserve a higher priority than understanding what role in the conversion process search plays for new and returners and in that case apply a similar method as above to the two, three, four segments you have in mind.

And if your shaking your head and mumbling that you don’t have the time for a sizable analysis proposed above try a single short one. Although building a segment that breaks out a group of users who had an internal search and a site success is self selecting, this is sometimes and good way to get suggestions for what might be working.  By seeing when and why this group is searching you can prevent the potential mistake of saying internal search is always good.

So why would you not want people to search?  I bet you can come up with a reason or two.