Web Analytics Tool Time

Where Big Meet Small- The Perfect Training Analysis

In my intro Economics classes in college we always talked about the world as if it were a perfect place. No we didn’t hold hands and discuss rainbows, bunnies and sunshine, but we did discuss the widget industry as if it existed in a vacuum. I knew that concepts like the widget industries adjusting perfectly to a decrease in widget pricing were far from the real world, but for the purpose of education I was willing to suspend my disbelief.

So, in true Econ 101 I present to you a best case scenario for a training analysis. Chances are small that you have a task like this to pass along to your developing analyst, but anything even slightly resembling this analysis should be the catalyst for leaps in growth.

In the perfect world your analysis should be of big proportions with few boundaries. Forget classifying your campaigns or building a report on those who enter via an email, see 2-4 pages and exit on a product page, they are small peanuts. This analysis should be of an entire web site. But let’s not forget how fragile your new analyst is. You cannot pull up www.ebay.com and say GO. In this perfect world, the site you put in front of your analyst should be of small proportions, let’s say a site with 30-100 total pages, 3-4 page templates, 5-10 primary routers and 1,000 visits per day.

The hardest part and possibly least real world piece of this analysis is that optimally the analysis should be done one to three months into an analysts career or just at the point where they are becoming a (mostly) self sustaining productive force. I know the temptation to use your analyst, who recently was graduated from “burden for now” to “they only ask five questions a day!” for all those little housekeeping tasks will be immense, but try to postpone the housekeeping. 

So you now have allocated plenty of time for your young analyst to perform a boundary less analysis of a small site. What kind of direction should you give them? Not much. Tell them the end result should be the combination of a site overview and highlights of the most interesting insights and trends the site has to offer. Beyond that I like to rattle off a few specifics to look at, like site router effectiveness and exit pages reports. Also, I make sure to outline a few basic segments like first time visits, clients, etc, and make sure to leave it open to the new analyst to build a few of their own.

Once you laid that foundation, sit back and watch the growth. Like a Chia Pet, you should be able to watch the growth almost daily. You should only get concerned about the progress of the analysis if the flow of questions halts.

Sometime in the middle of this you should sit down with the young analyst and run through all the data they have collected. Ask them to specifically point out the most interesting findings. With each finding explain why it is valuable or why it wouldn’t make the cut for a presentation. Also, make sure that the forming analysis will be complete and tell a story when it is complete. If the final analysis appears like it will have loose strings get your young analyst back on track.

In the end, this works as a training tool because for the first few months you have given your young analyst all the skills to analyze a whole site. Given them this assignment at the opportune time during their growth helps little leanings fit into a much bigger picture. Conversely, be careful not to spring this analysis too early as it could easily cause confusing and retard growth. By allowing your young analyst to perform this analysis you should in turn expect to watch his/her comprehension of how to think about, build and use population segments skyrocket. It’s hard to understand how segments work when you’re not starting with a complete pie, but this analysis hands your trainee a full pie to break up on their own. By comparing many comparable pages your developing analyst will get a better sense of how to analyze based on data as opposed to project personal judgment. This will be helpful in the future when they might be asked to analyze a single page. They should also more efficiently find the interesting nuggets in smaller analyses and be able to better mentally plan out a future analysis from end to end.

It has been in moments while discussing a big and small analysis with fresh analysts that I have seen true comprehension. I specifically recall an instance where I was corrected on how to define an “engaged” segment. For a brief instant my ego was wounded, but quickly I came to my senses and remembered that while training nothing is more satisfying than being corrected.

This big and small analysis could be nothing more than a training tool, but by keeping an open mind it might actually turn up new leanings or new areas of analytic focus. If done properly the final product of this analysis could be a perfect brainstorming tool for laying out an analytic roadmap. Additionally, if your considering a site rebuild, about to undertake a site rebuild or have recently rebuilt your site then use this analysis to baseline or check for improvements. My favorite part about new analysts is that they always bring a fresh prospective; it’s your job to let them.

Top 10 Benefits of Migrating to Omniture’s SiteCatalyst? Part II

Without further ado here are my thoughts on "benefits" 6-10.  In case you just tuned in click here to start with my first post on the "Top 10 Benefits of Migrating to Omniture's SiteCatalyst?"

6. Standard reporting-

In this section the narrator claims that SiteCatalyst can be responsible for producing "board room ready reporting" that is "clear and compelling."  Sigh.  If only it were that easy. I admit Omniture does produce some of the prettiest interface and exportable reports.  And no matter what decision makers at your company might say, prettiness does count for something.  But you aren't going to get out of the box reports that you can put in front of a board room and tell a complete story.  Often when I'm pulling together an overview report or analysis, I will find that I can start off by using prefab or easy to access reports in Omniture, but by the middle or end I am needing to build my own customized reporting in Excel.  At that point I sometimes end up rebuilding the reporting I pulled from Omniture in Excel for the sake of continuity. 

For the record Omniture reports do come out looking flashier than what you can build in Excel 2003, but if you invest in Excel 2007 you can easily build reports that trump both.

7. Personalized Interface-

Personalized interface reporting can be an important feature to your company (although it is only mentioned as an afterthought in this section).  Interface reporting is a perfect way to get a quick overview of what is important to you and your business.  It can be a dangerous tool if it is all people at your company utilize or if it makes you reliant on arbitrary little dials that point to green, yellow or red (which obviously translate to bling bling spend money like it ain't nothing, the Magic 8 ball says "Ask again later" or Chapter 11).

Those currently using HBX are probably no stranger to interface reporting…or are you?  I'm not exactly sure why, but in an informal study of Semphonic clients’ accounts I find that those using SiteCatalyst are far more likely to be actively using interface reports than those with HBX. A safe bet why is that in Omniture interface reports are much more flexible.  If you’re used to the complete freedom of Excel, they will make you feel like you’re trapped in a Columbian jail, but compared to other tools they are one of the biggest backyards in the neighborhood. 

If you, the heavy tool user at your company, are assigned to build reporting for others, you will notice several weaknesses in how reports are shared.  You can build a dashboard or singular report and allow other users to access it, but if you want it to be easily accessible for other users or appear for them upon login you'll have to login to their account and set those things up yourself.  No big deal if you’re setting up three accounts, but headache city if that number is double digits.    

To sum five and six up, Omniture will give you more interface reporting, exporting and scheduling options than HXB.  It's your call whether that's important to you or not (be aware that the above thoughts are relevant to things you can get immediately out of HBX and SiteCatalyst; to see my thoughts on how both tools Excel integration tools stack up click here).

8. Web 2.0-

This section was perplexing to me.  It is titled "Web 2.0" but really just discusses how Omniture can correlate data.  Last time I checked, even if your site doesn't contain blogs, social networking or rich internet applications, correlating data was still an important feature.

The main takeaway current HBX clients should have from this section is that when using SiteCatalyst you’re going to have to think about your data and extracting relevant things from it differently.  In HBX you might have built an Active Segment to understand the engagement level of people coming from a specific email campaign but with SiteCatalyst you’re going to have to rely on Subrelations/Correlations.  Unfortunately, Subrelations/Correlations are much less flexible than Active Segments.  Once you build an Active Segment you essentially can correlate it with anything else you collect data on, but with Subrelations/Correlations you have a finite number of things that you can correlate your data to.  If you anticipate your needs when setting up your account, then you should have no problem; but if the Subrelations/Correlations you set up don’t cut it for reporting or an analysis, you’ll have to pick up the phone and your wallet.

9. Genesis-

Numbers nine and ten are essentially strategically placed selfless plugs.  I'm sure when Omniture purchased VS they intended to convert the VS customer base to being not only SiteCatalyst clients but clients of the full spectrum of all things Omniture.  And Genesis would be one of those things.  Although I have limited experience with it, I do like the idea of being able to easily send out emails to specific segments of your site users.  I have heard some complaints the Genesis based paid campaign optimization technology isn't quite "there" yet, but I lack hands on experience to support that claim.

10. Platform-

Ten is the kicker.  The point that will undoubtedly linger with you.  The words that will leave you drooling over not only SiteCatalyst, but everything Omniture has to offer.  But Discover, SearchCenter, Offermatica and Touch Clarity and Genesis aren't the same as the pack of gum you add in the checkout isle.  They’re damn expensive.  If you’re debating between SiteCatalyst and Google Analytics, you almost certainly bother with these add ons, but for big budgets these will all be tempting in their own ways.  If you’re considering add ons just be careful not to out buy your bandwidth (ALERT: That last sentence was a Gross oversimplification).

I understand why Omniture lumped these things together, but you should mentally break them out.  Discover sticks out of that list as the only thing that doesn’t do something completely different than SiteCatalyst.  If you currently have active HBX users responsible for analysis and reporting in place, they should be able to understand and utilize Discover from (almost) day one.  In fact they might need to utilize the tool to extract data that you previously were able to get directly out of HBX.  The other tools are the outliers:  the ones that integrate (although not necessarily easily) with SiteCatayst, but do entirely separate things than SiteCatalyst.  These should all be evaluated individually with consideration to their benefit to your company and if you have resources that could not only handle but extract value from them.

Top 10 Benefits of Migrating to Omniture’s SiteCatalyst?

Despite the finalization of the Omniture acquisition of Visual Sciences, details of the future of the product line and migration process are as uncertain as ever. On Omniture.com the ridiculous “Learn about the migration program” link routes to a page of quotes from those currently benefiting from Omniture. Not only does this link route to a page of irrelevant information its salt in a wound. When I’m approaching a traffic jam the last thing I want is a call from my coworker, who left early, letting me know he was pulling into his driveway.
                                    

One of the most substantial pieces of “new” information is the narrated presentation of the top 10 reasons why HBX customers should switch to SiteCatalyst. To help HBX customers wade through this list of spoon fed Omniture rhetoric I figured I would provide a heavy tool users perspective on each reason why to switch. Here are my thoughts on 1-5. Expect 5-10 to be posted in a day or two.

Omniture’s Top 10

1. Admin Console

Omniture describes their admin console as a powerful tool for maintaining and sustaining all things SiteCatalyst and largely I agree with this description. The admin consol is fairly easy to use and gives you a lot of control over creating logins, specifying what content users can access and manipulating report suites. Its not quite as intuitive as the drag, drop, bada bing, bada boom interface the friendly narrator suggests, but the heavy tool users (who should be the only ones with access) should have no trouble using it. And once heavy users get a taste of it, it will be hard to remember how they ever lived without it.

If anything, my biggest issue with the admin console it that it allows users to change too much. In two clicks you can remove every page in a report suite or do numerous other actions that will do irreparably damage to your report suite or wallet (usually Omniture can fix your f*** ups, but it’s gonna cost ya). Additionally, it’s surprisingly easy to create undeletable entities, such as report suites and data categories here. It is common for higher up, light users to expect access to something called the “Admin Console,” but in interest of keeping your report suite looking clean and full of numbers this should be discouraged.

Conversely, there are areas of the admin console where you will reach a dead end and have to call Omniture Live Support. Sometimes it will make sense why you have to call, but other times, like when your building a rollup report suite, you’ll be a bit annoyed.

2. International Capabilities-

It’s nice that Omniture supports a lot of currencies. You still have to pick a single currency per report suite, so it’s not revolutionary system for handling currency but its still better than nothing. In reality no web analytics tools handle multiple currencies as well as they should, and this is a common area to run into implementation and reporting issues.

I can’t say much in regards to the fact that Omniture supports a slew of languages, but I was surprised that Omniture didn’t mention here (or anywhere in this list) the extensive amount of online help and documentation they have available. Heavy users will often find holes in the manual and Knowledge Base, but it’s still the best online help of any enterprise web analytics solutions. The value of good online support should be taken into consideration when you are evaluating your tool options. Omniture Live Support licenses must be purchased on a person to person basis and are NOT cheap. If you plan on having SiteCatalyst users in your organization that do not have Live Support access they will most certainly be digging through what is help is available online.

3. Version Support-

Omniture promises to support older versions of SiteCatalyst and I promise not to care. The biggest difference I notice from one generation to the next is the color scheme. Users would have to be VERY stuck in their ways or not be able to adapt to the minor changes of a new version of SiteCatalyst. Omiture’s willingness to support old versions makes then enablers, and it might actually benefit your business if they let older versions sail into the sunset when their time comes.

4. Custom Metrics-

I tend not to use custom metrics very often, but I do think they are cool. If your into stretching the limits of in interface reporting to the max custom metrics will help. You do have to be careful not to get too carried away building metrics, though, or your reports could become more confusing than it is helpful.

5. Segmentation-

Telling a group of people that are currently using HBX Active Segments that ASIs are a reason to switch to SiteCatalyst is just ludicrous. Powerful segmentation is an essential feature of any enterprise solution, so I understand why Omniture feels the need to brag about their capabilities, but they are barking up the wrong tree. ASIs are a massive step back from Active Segments. ASIs are sluggish and limited to 5 (which at times feels like it might as well be 0). In principal these things segmentation tools are nearly identical, but if you’re expecting ASIs equal to or better than Active Segmentation you’re going to be sorely mistaken.

The narrator also mentions SAINT as a segmentation tool. Although I don’t really agree that most of the uses for SAINT are related to segmentation, I do like SAINT. It’s a handy and fairly easy to use classification tool that allows you to easily change things like product numbers into product names or rollup products into relevant groupings. When rolling up you only get summations and not deduping, so look to the tool for page view data and not visit/visitor data. SAINT will give you a slightly easier development, since you won’t need to spend hours figuring out how to send Omniture an image request with the product name instead of the product number. Also SAINT will keep your report suites looking cleaner and easier to use.

WOWA

As the champaign glasses clinked and lips smacked on the night of December 31st 2007 I made a resolution to blog more.  It might have taken me until the 16th day of January to begin my resolution, but from here on out in 2008 expect a lot from Web Analytics Tool Time. 

Throughout the year this space will be filled with content to help you, the web analytics practitioner, but today this space is for me to turn reflective.  Read if you like.  Pass if you like. Just don’t expect to gain any knowledge except that of how I view the World of Web Analytics…

I am incalculably a nerd.  I’ve been in science fairs and essay contests. I attended a college built by nerds for nerds, where only people who did not embrace the notion that they were a nerd where having an identity crisis.  I picked my major because out of my freshman year class pool including drama, photography and psychology, economics was the most fun. I even find ways to put nerdy twists on my non nerdy interests.  At Oakland A’s games I might cheer with all my testosterone induced might for a late game rally but I also could probably tell you both starters pitch count, approximate lefty/righty success ratios of the guy warming up in the pen and the on deck batter’s on base percentage over the previous road trip.  About the only time I won’t strike you as a complete and utter nerd is when I’m atop a surf or snowboard, and that’s only because I won’t be saying much of anything.

So it doesn’t really surprise me that when I think of my first two years in web analytics I consider it parallel to something quintessentially nerdy: fantasy books.  Without fail fantasy books always seem to begin in some small, rural, seemingly normal town.  Novato, California, the town where my web analytics career was born, wouldn’t fit that description, but in the World of Web Analytics (WOWA) Semphonic would.

During those first couple of months the newness of the work acted as blinders to the greater web analytics world.  As far as I was concerned Semphonic consultants were the only people in the world who built Active Segments, implemented Omniture, did deep dives on conversion rates or made site rebuild recommendations. 

I began opening up my eyes to the WOWA via blogs.  Not coincidentally I started by reading the closest blog to home, Gary Angel’s.  Gary’s writing led me to the likes of Eric Peterson (who I found he mostly agreed with) and Avinash Kaushik (who I found he not only disagreed with but on occasion shredded to pieces).  The blogs of Peterson and Kaushik were portals to the writing of many others.  Chipping away each day I slowly climbed the mountain of blogs and gaining perspective with each step.   

Armed with the perspective that the WOWA doesn’t only exist in the second floor of an office building in Northern California on my first major business trip I traveled to the offices of Omniture in Oram, Utah.  Looking up at the wall of Omniture clients I realized that blogs just scratched the surface, the surreal parallel universe of WOWA extended much further. 

My view of the WOWA was almost entirely conceptual until last year's San Francisco Emetrics Summit.  The WOWA I had come to undertand existed as tiny islands amongst oceans of other industries, but Emetrics swelled our land mass and allowed me to attached bodies to floating minds.  Eric Peterson announced his departure from VS, I met Avinash and as a result of listening to a series of presentations my web analytics ego swelled.  Everything made sense.  Everything was easy.  I had figured out the WOWA.

Then a few months later came a reality check in the form of Semphonic’s Xchange Conference.  Throughout the course of two days, intimate conversations pealed back layer after layer of the WOWA.  Up until that point my WOWA only involved analyzing individual web sites, but concepts like behavioral targeting opened up my view to include the entirety of the World Wide Web.   

My journeying around to the corner of the WOWA in one sense has shrunk Semphonic into a tiny island; an entity that is relegated to a line in a Forrester report.  But on the other hand my perspective has made it bigger than ever.  I didn’t understand my island’s uniqueness and progressiveness until I saw how others within the WOWA operated.   

I read fantasy books because they take seemingly normal words and twist and turn them into something extraordinary.  But by the second or third book in a series they always lose my interest as the author runs out of nuances about the world and has to turn to crazy plot twists.  So far the WOWA I have come to know has never failed to reveal something new with each passing page.  And considering the rapid industry and intellectual growth within out WOWA I can’t imagine it ever becoming stagnant.  The more I live and learn in this odd parallel world the more I like it.  .