Web Analytics Tool Time

Where Big Meet Small- The Perfect Training Analysis

In my intro Economics classes in college we always talked about the world as if it were a perfect place. No we didn’t hold hands and discuss rainbows, bunnies and sunshine, but we did discuss the widget industry as if it existed in a vacuum. I knew that concepts like the widget industries adjusting perfectly to a decrease in widget pricing were far from the real world, but for the purpose of education I was willing to suspend my disbelief.

So, in true Econ 101 I present to you a best case scenario for a training analysis. Chances are small that you have a task like this to pass along to your developing analyst, but anything even slightly resembling this analysis should be the catalyst for leaps in growth.

In the perfect world your analysis should be of big proportions with few boundaries. Forget classifying your campaigns or building a report on those who enter via an email, see 2-4 pages and exit on a product page, they are small peanuts. This analysis should be of an entire web site. But let’s not forget how fragile your new analyst is. You cannot pull up www.ebay.com and say GO. In this perfect world, the site you put in front of your analyst should be of small proportions, let’s say a site with 30-100 total pages, 3-4 page templates, 5-10 primary routers and 1,000 visits per day.

The hardest part and possibly least real world piece of this analysis is that optimally the analysis should be done one to three months into an analysts career or just at the point where they are becoming a (mostly) self sustaining productive force. I know the temptation to use your analyst, who recently was graduated from “burden for now” to “they only ask five questions a day!” for all those little housekeeping tasks will be immense, but try to postpone the housekeeping. 

So you now have allocated plenty of time for your young analyst to perform a boundary less analysis of a small site. What kind of direction should you give them? Not much. Tell them the end result should be the combination of a site overview and highlights of the most interesting insights and trends the site has to offer. Beyond that I like to rattle off a few specifics to look at, like site router effectiveness and exit pages reports. Also, I make sure to outline a few basic segments like first time visits, clients, etc, and make sure to leave it open to the new analyst to build a few of their own.

Once you laid that foundation, sit back and watch the growth. Like a Chia Pet, you should be able to watch the growth almost daily. You should only get concerned about the progress of the analysis if the flow of questions halts.

Sometime in the middle of this you should sit down with the young analyst and run through all the data they have collected. Ask them to specifically point out the most interesting findings. With each finding explain why it is valuable or why it wouldn’t make the cut for a presentation. Also, make sure that the forming analysis will be complete and tell a story when it is complete. If the final analysis appears like it will have loose strings get your young analyst back on track.

In the end, this works as a training tool because for the first few months you have given your young analyst all the skills to analyze a whole site. Given them this assignment at the opportune time during their growth helps little leanings fit into a much bigger picture. Conversely, be careful not to spring this analysis too early as it could easily cause confusing and retard growth. By allowing your young analyst to perform this analysis you should in turn expect to watch his/her comprehension of how to think about, build and use population segments skyrocket. It’s hard to understand how segments work when you’re not starting with a complete pie, but this analysis hands your trainee a full pie to break up on their own. By comparing many comparable pages your developing analyst will get a better sense of how to analyze based on data as opposed to project personal judgment. This will be helpful in the future when they might be asked to analyze a single page. They should also more efficiently find the interesting nuggets in smaller analyses and be able to better mentally plan out a future analysis from end to end.

It has been in moments while discussing a big and small analysis with fresh analysts that I have seen true comprehension. I specifically recall an instance where I was corrected on how to define an “engaged” segment. For a brief instant my ego was wounded, but quickly I came to my senses and remembered that while training nothing is more satisfying than being corrected.

This big and small analysis could be nothing more than a training tool, but by keeping an open mind it might actually turn up new leanings or new areas of analytic focus. If done properly the final product of this analysis could be a perfect brainstorming tool for laying out an analytic roadmap. Additionally, if your considering a site rebuild, about to undertake a site rebuild or have recently rebuilt your site then use this analysis to baseline or check for improvements. My favorite part about new analysts is that they always bring a fresh prospective; it’s your job to let them.

Connecting the Dots or Coloring by Number

Whenever we take on a new employee, an internal battle of epic proportions ensues within me.  This battle does not feature Custer, Germans or Russell Crowe, but instead two pre-school concepts.  Do I give a new employee the skill to connect the dots in order to see the whole picture or just have them color by number? 

In the second scenario the newbie doesn't need to know what the picture is of to be productive, but you can also expect their learning process to be significantly slower. So when you take on a new employee what training technique or mix of techniques should you undertake? The easy answer seems to be to connect the dots before setting them free, but that is easier said than done. 

Calling the process of teaching a newcomer to web analytics all the pieces necessary to do a good analysis is time consuming would be a massive understatement.  And, I'll go out on a limb here, your probably already busy with your own web analytics responsibility.  I will not claim to have perfected a seamless training plan, but in my experience I have found that starting with a simple, quick connecting the dots exercise can have great benefits.  This exercise is best suited for a computer with a multiple monitor setup (the more the better), but its worth doing even if that is not available. 

Start by pulling up instances of the source code of a tagged web page, the web page itself (possibly with a data overlay if your analytics tool has one), the online interface of your web analytics tool and the Excel reporting tool (if your tool has one).  For bonus points you can also pull up an instance of the site with a packet sniffer tool overlay, like Page Engage or Charles.

To begin go to the site window and physically do something on the site.  Start simple with a something like a link click or search.  Then work backwards to the source code and painstakingly slowly point out what in the source code is pushing out the web analytics data.  Especially if your trainee is completely new to web analytics, don't expect much to about source coding to stick.  Even if only 5% of the words you say regarding source coding actually resonate, it is still worth both of your time to lay the foundation. 

Next leap forward to your web analytics tool and show exactly where the data you just generated is located.  Presumably you have already introduced the interface of the web analytics tool to your trainee, but if not this is a good point to walk them through the basics of navigation.  When showing him/her where the metric you generated is located go slow, step by step through the interface hierarchy.  Don't just click through to the metric, but explain why the metric is hidden behind each layer of the hierarchy. You'll probably sound silly to yourself and possibly your trainee as you go layer by layer, but this should not only make the metrics location stick but should allow the trainee to reason through the location of future metrics.

Finish off by building an Excel data block that pulls the metric that you initially generated with your on site action.  Obviously if this is your trainees first exposure to the Excel integration tool that should be an introduction to how to build data blocks as it is how to get the specific metric. Focus on explaining the difference and similarities of navigation to the metric in the context of the web interface. After going through this process with one metric, step up the difficulty ever so slightly do all the same steps again with a prop/evar/event/custom metric or something along those lines.  As you go through the process make special notes about the significance of the prop/evar/custom metric and why the metric you're using was tagged that way.  Metric three should be something like a campaign or product purchase.  I like to conclude the lesson by letting the trainee go through the steps on his/her own.  Like fiber, appropriately timed self guided learning has strong digestive powers. Sure this exercise does nothing teach a newbie the basics of doing an analysis or building a meaningful report, but even da Vinci leaned how to connect the dots before painting the Mona Lisa or something like that.

Interview with the (Web) Analyst

When Gary Angel first introduced the five new Semphonic bloggers he provided his own thoughts on each of us and what we might write about.  When it came to me he wrote,

“…Given his blog title and constant heavy tool usage, I’d expect his approach to be hands-on, practical and down-in-the-weeds. But given his mentoring experience, perhaps he’ll be talking about training and helping new analysts as well.”

Until reading Gary’s words I had never considered blogging on the topic of training new analysts.  But come to think of it, it's not a bad idea.

Semphonic is in the process of growing and with that comes new analysts.  Right now we are in the process of hiring one or two new analysts who I will undoubtedly be (partially to mostly) responsible for building him/her/them from the ground up. Given the position I’m sitting it, this seems like the perfect time to start blogging in real time about the process, challenges, tips and ideas for training new web analysts.

Since this is the beginning it seems reasonable to start where an analyst’s career begins, with an interview.  I’m lucky enough to get to grill all prospective employees once Gary and Joel have had their way with them.  When I sit down with an interviewee here is what is running though my mind and the web analytics skills (I didn’t think people here would care to read about how interpersonal and presentation skills are important) I’m looking for. 

•    All prospective analysts come to Semphonic offices having prepared a basic analytics exercise.  Most frequently this is a comparison between two months of data for a small to medium trafficked site using HBX, Omniture or a similar tool.  Having someone run through their analysis in front of me is probably the single most telling ten or fifteen minutes of the interview.
•    When someone is running through their presentation it is usually readily apparent if they understand, in general, what they should be looking for.  I can teach tool usage in a few days, a variety of analyses in a few weeks, but teaching what to look for when there are thousands of numbers in front of you will take months, if ever.  I don’t ask the inexperienced to go strait to the heart of a problem and create an elegant solution, just for their thinking to be “on track.”  This could look like a lot of things, but usually manifests as someone spending time discussing several relevant KPIs (i.e. single page access rate, views/visit, visits/visitor, etc.) and connecting them to each other to create conclusions.
•    A big part of web analytics is story telling.  I want to hear an interviewee tell me an interesting, believable story of what users are doing on a site.
•    I do not expect someone with no web analytics experience to fully grasp the concept of traffic segmentation, but I like to hear people hinting at bits and pieces of it.  Also, just as problematic, if not more so, than someone not understanding segmentation, is a person thinking about it in terms of a previous job.  Both a complete lack of understanding about segmentation and a strong non web analytics segmentation foundation are big red flags for me.
•    General tech, Internet and computer application savviness counts.  The more web sites someone sees and uses, the better I expect them to be at understanding what works and be able to generate fresh ideas.  And, the more tricky computer apps someone has mastered the faster I expect them to pick up the ins and outs of an analytics tool.   
•    The ability for people to understand the difference between correlation and causation (in terms of real world application, not Webster’s Dictionary) is essential. 
•    Someone who is willing to look past his or her opinion of a site and directly at the numbers.  A skilled, experienced analyst can and should inject plenty of informed opinions into their thinking, but just as Picasso perfected the human body before cubing it up, a new analyst must first go solely with the numbers.  All too often I have heard interviewees lead into their presentation by explaining that a clients site is “very good” or “very bad” (usually the former, not the later—they are trying to make a good impression!).  As soon as someone says this all I can see is a ticking time bomb of an analyst, who, when they have acquired the web analytics skills, will selectively seek out data to support an opinion rather than reference the data that truly tells the story.
•    I want the interviewee to have loved doing the analytic exercise required of all applications.  I’ll accept “liked a lot,” but anything less makes me worry.  When I was doing the analysis leading up to my interview I remember seeing it as a break from my job hunt, a truly interesting and enjoyable real world puzzle.  I want the person in front of me to have felt the same way.  The more someone likes the work the faster they will progress.  It’s not easy finding an employee who will go to sleep at night pondering techniques to try out the next day, but it’s also not a bad thing to strive for.