Web Analytics Tool Time

Speculating on the Future of Omniture's Excel Client

Usually in the blogosphere someone will write a blog and other people will comment.  I am responding to my own blog, "The Future of Omniture's Excel Client" because:

A. I have more to say that didn't seem appropriate in the more journalistic style blog (couldn't you tell I once wrote microscopic blurbs for a 200,000 copy a day paper?).

B. I fear though that the overwhelming popularity of my blog (Google Page Rank 4/10!) has labeled my writing as untouchable.

My interview with Ray Rauch left a lot to be desired.  Extracting exact details from him was a bit like pulling teeth with no Novocain.  I tugged, he made some noise, but nothing substantial came out.  

On one hand his silence gave me no clear sense of the product line, but on the other hand he left me plenty of room to speculate.  Emphasis in the last sentence should be on "speculate."  Other key words to keep in mind as you read this blog are "guess," "hypothesize" and "don't sue me."

From the interview I truly got the sense that Omniture has done and will continue to do their client needs homework (or they simply read my "Open Letter to Omniture"). In promotional materials and conference speeches, preaching about product development based on client requests sounds great, but I have fears that the process isn't as democratic as it sounds.  The interview left me with worries that Omniture has unequally weighted the needs of the most engaged Report Builder users.  This is perfect for me and the other supertool users, but I have worries that this direction will create substantial barriers to entry for infrequent users.  In addition to committing to adding complex features it probably would have been wise for Omniture to focus on making Excel Client more smooth and intuitive.

The main takeaway clients or potential clients should have about the client induced direction of the tool is be vocal about your desires.  Omniture has promised that if they get enough momentum behind a request it will be put in place (assuming it is possible).  Make them put their money (actually YOUR money) where their mouth is.

The additions that will come first in the "sooner than you think" update are anyone's best guess.  Based on some of the emphasis in the conversation I would think the ability to copy and paste request, with both absolute and relative references, is what we should expect.  Rouch made it very clear that copy and paste was a request they had received from many, many clients so it seems logical it the capability would be in the initial update.  The update also makes sense because Excel Client already has limited copy and paste functionality, so it should not be a stretch to enhance it.

It seems very possible that copy and paste functionality will be integrated into an Excel Client without complementing additions, but I'm going to give Omniture more credit than that.  I trust that Omniture will recognize that without adding the ability to add data blocks without accompanying text, the ability to copy and paste them is significantly less valuable.  The Omniture mandate that all data blocks come with text showing the date range, metric type, etc. in relatively inflexible formats make data blocks big and clunky, difficult to reference and nearly pointless to copy and paste.  

The strength of Report Builders copy and paste functionality come from the ability to set up the cells data blocks will references, whether they be with page names, dates, segments, etc. in advance of building a single data request.  Then being able to build one data request, copy and paste it over and over and have a neat, tidy and compact report before you know it.  

In order to get this type of speed and flexibility with Report Builder my guess is Omniture will install a check boxes or toggle button to allow users to select non-data, written items they would like included in each data request.  Or at very least Omniture will provide an all or nothing button so users can either include all the usual text or none of it.      

In addition to adding robust copy and paste functionally, I would guess the next generation of Excel Client will have one other change, the look and feel.  Although, Omniture acknowledge they were aware Report Builder users loved the interface and process of building data blocks, I have serious doubts an updated Excel Client will resemble anything close to Report Builder.  Given the direction of the Omniture family of products I would guess the look and feel changes would mostly be used to make the tool more inline with the new Omniture Suite.  

This partially makes sense because Report Builder has promised to add functionality to extract Discover data in the short run and probably will add it for other products in the long run.  But unlike the Omniture Suite, Report Builder would probably be better suited for seamless integration of other Omniture products where users are only conscious of the data they want, not whether it is coming from Discover, Search Center, etc.

The look and feel changes that I'm projecting might slightly improve the flow and usability of the tool, but not bring it anywhere close to that of Report Builder.  In my perfect world an updated Excel Client interface would be 80% Report Builder, 10% old Excel Client and 10% Omniture Suite, but I would expect a distribution more along the lines of 50% old Excel Client, 40% Omniture Suite and 10% Report Builder.

When the dust settles on the next round of Excel Client updates I expect to see a prettier tool that is no easier to use but has one significant HBX/client inspired addition.  It will be one step in the right direction, but no giant leap for webanalyticskind.  Omniture will probably still need two more years and two more updates to catch the Excel Client product up to and (maybe) surpass Report Builder.  Sigh.

Author's note- The correct answer was B. 

The Future of Omniture’s Excel Client

Click here to read the introduction to this blog

Out of this year’s Omniture Summit came word that Omniture’s Excel integration product, Excel Client, would be evolved to integrate features of HBXs Report Builder

In late March I had the pleasure of speaking with Ray Rauch, Omniture’s Vice President, Customer Support, regarding the future of Excel Client, specifically regarding the Report Builder additions.  Rauch spent four years with Visual Sciences prior to the acquisition and now and now works with customers currently engaged with or transitioning from HBX.  His team is constantly listening to the thoughts, ideas and needs of clients.      

In the beginning of our conversation, Rauch was surprisingly honest and quick to admit some of the superiorities of the HBX Report Builder tool over Excel Client.  He noted that numerous clients he had conversed with touted the speed and flexibility of Report Builder.  In both those respects Rauch explained that Excel Client has a “poor competitive advantage” and it was something they were aware of and working on.

Trying to get to the details of the tool convergence, I asked some very specific functionality questions during the interview.  They included:

• Will copy and past using absolute or relative cells be added?
• Will the Excel Client interface remain in its current form or will it take no the look and feel of Report Builder?
• Will Excel Client adopt technology that will allow for the full range of Excel formatting capabilities of data blocks?
• Will it be possible to pull single numbers without any text as data blocks?
• Will functionality to Report Builder’s matching and trending be added to Excel Client?
In various ways Rauch acknowledged that just about all my questions were one that had come up in the past, but chose not to elaborate on any in particular.  He suggested that additions along these lines are what Omniture is looking into for the next Excel Client update.

Despite mostly talking in generalities, what Rauch was willing to discuss did in fact turn out to be quite revealing.  He said that a “…tremendous amount of really positive feedback and really great ideas…” has already been collected from clients to help dictate the product direction.   

“Were really hoping…to look into how we can integrate those features of Report Builder into Excel Client.  It’s pretty safe to say that we did not run across anything the customers said that they wanted, that they enjoyed in the Report Builder interface that we couldn’t do under Excel [Client] it’s just a matter of the timing of those specific features and how they’ll be integrated so were keenly interested… that ReportBuild has the best...features”

The convergence of the two product lines will not happen overnight.  Omniture plans to
“…get the most important things out in the first iteration and then evolve the product.”
For the first step in converging the products Omniture appears to be more focused on speed than a comprehensive update as it, “may require a step back…so we can get it out quickly.”  After the first update Rauch explained Omniture will, “Continue to develop that line and mask sure we work with customers over time.” Additional Report Builder functionality will be added, “If we get enough momentum and critical mass behind it…”      

But talk is cheap, when can you expect to see Report Builder functionality in Excel Client?  During our conversation Rauch pointed out that until very recently the two products were developed for many years in separate vacuums and have stark differences that go much deeper than functionality.  Despite the challenges ahead of the Excel Client development team, Rauch sounded confident Report Builder functionality would be found in Excel Client “sooner than you think.” 

Up Next:

Early next week- Thoughts, reaction and speculation for the future of Excel Client
Shortly after- Transitioning your HBX Excel reporting to Omniture Excel Client   

 

An Excuse and An Introduction

Click here to go directly to “The Future of Omniture’s Excel Client,” otherwise go ahead and wade through the following (pathetic) excuse.

A. Stagnant. Neglect.
B. Holding pattern. Pause.

Pick either A or B and you have described my treatment of this blog.  If you look back and see my last post, which promised an immediate follow up, was on February 21st you might pick A.  But let me make a case for B. 

I was planning on writing about transitioning Excel based reporting based on the assumption that HBX’s Report Builder and Omniture’s Excel Client would remain as two (completely) separate tools.  Then flowing down from the mountains of Utah came word that my assumption isn’t quite correct.  The two tools will converge with time. 

At that point I would have put on the breaks, admitted I knew nothing of what Omniture meant by “converge,” and blogged about something else if it wasn’t for one June Dershewitz.  Using her rare combination of web analytics skills AND social skills, she made acquaintance with “someone, who knows some people” (or a fellow in Omniture marketing, but is there really a difference?). 

Apparently Omniture, for better or worse, had been following the Semphonic blogs since they were launched.  In spite of, or as a result of, my previous blogs, Omniture was willing to put me in contact with someone close to the source of the Report Builder- Excel Client tool convergence. 

After an interview with, Ray Rauch, Omniture’s Vice President, Customer Support it no longer seemed appropriate to turn around and immediately blog about converting Report Builder report to Excel Client.  Instead, it made more sense to share my newfound wealth of knowledge about the future of Excel Client.

Making the Switch – Transitioning Your Excel Reporting From HBX to Omniture

Part I: Implementation and Interface

So you’ve looked (hopefully), leaped (bravely) and landed on your feet (hopefully).  You’re now a proud Omniture client, with data coming out of your ears, new platforms to play on and an endless ocean of analytic possibilities.  But before you even begin contemplating your first Omniture based deep dives a series of email pour in, all with the same gist – where’s my daily/weekly/monthly Excel report?  These come in from people who didn’t know about the Omniture acquisition of HBX, that your company has transitioned from tools or exactly how to define a single access visit, the just want their freakin numbers. 

If you haven’t considered how to rebuild existing reporting until the “where’s my report?” emails roll in, you’re already too late.  You have already failed those dependent on you for reporting.  After “looked” and before “leaped” is when you should begin laying the framework for rebuilding existing reporting in Omniture’s Excel Client.  Here is a list (in time relevant order) of steps and considerations for transitioning your HBX reports, especially those of the automated variety, to Omniture.

1.    Its important to realize that many of the HBX interface tools that you can utilize to get reporting data (campaigns, active segments) won’t exist in their same form.  In general, Omniture reports require more forethought and planning because they are more dependent on tagging and less dependent on an interface customization.  Without active segments or campaigns (the HBX kind) to lean on, you’re going to have to start thinking about how they can be replaced with a combination of custom metrics and correlations/subrelations.  It will require a major in shift in thinking going from having a hand full of custom metrics to a boat load of them. And it’s your job to interweave important reporting metrics in the implementation of custom metrics. Often custom metrics for reporting would have been tagged as props or evars anyway, but not always.  Metrics tagged as props are slightly (or vastly, depending on your implementation) different from evars, so be careful when deciding if your metric will be a prop or an evar.  If you have the slots to spare considering doing custom metrics as both, so you get the best of both words.

2.    The other main tool you must consider (A.D. active segments) is correlations and subrelations. Although, these are enabled by calling Omniture, you need to consider them before tagging.  Prop correlations must be set on the same page or a correlation simply isn’t possible.  Subrelations, however, are more flexible since they do not need to be set on the same page and will most likely be integral to your future reporting.  Obviously subrelations/correlations cannot exist in vacuum so as your stretching your mind thinking about your new found wealth of custom variables you have to stretch it further to consider how you want them correlated/subleased and with what.  Personally I’ve always liked puzzles, I hope you do too.

3.    Anyone who spends a lot of time with Omniture will find the words “turned on” and “enabled” seep into his or her frequent vocabulary.  Once your implementation is in place you should make sure to enable visits and visitors for metrics where they are necessary for reporting or might be necessary in the future.  Doing this in advance of needing them will save you a ton of grief.

4.    Once your implementation is in place, correlations/subrelations are connected and visits/visitors have been turned on, spend some time with SAINT.  With Report Builder you might have been using Excel equations, like vlookups, to sum groups of pages or translate page names, but with SAINT this no longer has to be the case.  If your page groupings or lookup tables change more often than each month it's probably easier to continue using Excel equations, but if they are more static take advantage of SAINT.

5.    Even though you lack active segments in SiteCatalyst, Omniture throws you a bone of five ASIs.  You may or may not choose to use these for reporting.  If you do be aware that the bigger the segments population or the more complex the segment the more lag you’ll get on the ASI data.  In general, it’s a bad idea to rely on any ASI for time sensitive reporting in any form.  However, ASI are a powerful tool to help recreate HBX reports in Omniture and if you plan on using them in delayed reporting they should be built as early as possible.

Stay tuned for Part II where differences between HBX Report Builder and Omniture’s Excel Client will be discussed along with the changes you will need to make in your Excel report building technique.  Additionally in II I’ll discuss what the best practices are when you cannot fully automate a report with the Omniture tools available.

Top 10 Benefits of Migrating to Omniture’s SiteCatalyst? Part II

Without further ado here are my thoughts on "benefits" 6-10.  In case you just tuned in click here to start with my first post on the "Top 10 Benefits of Migrating to Omniture's SiteCatalyst?"

6. Standard reporting-

In this section the narrator claims that SiteCatalyst can be responsible for producing "board room ready reporting" that is "clear and compelling."  Sigh.  If only it were that easy. I admit Omniture does produce some of the prettiest interface and exportable reports.  And no matter what decision makers at your company might say, prettiness does count for something.  But you aren't going to get out of the box reports that you can put in front of a board room and tell a complete story.  Often when I'm pulling together an overview report or analysis, I will find that I can start off by using prefab or easy to access reports in Omniture, but by the middle or end I am needing to build my own customized reporting in Excel.  At that point I sometimes end up rebuilding the reporting I pulled from Omniture in Excel for the sake of continuity. 

For the record Omniture reports do come out looking flashier than what you can build in Excel 2003, but if you invest in Excel 2007 you can easily build reports that trump both.

7. Personalized Interface-

Personalized interface reporting can be an important feature to your company (although it is only mentioned as an afterthought in this section).  Interface reporting is a perfect way to get a quick overview of what is important to you and your business.  It can be a dangerous tool if it is all people at your company utilize or if it makes you reliant on arbitrary little dials that point to green, yellow or red (which obviously translate to bling bling spend money like it ain't nothing, the Magic 8 ball says "Ask again later" or Chapter 11).

Those currently using HBX are probably no stranger to interface reporting…or are you?  I'm not exactly sure why, but in an informal study of Semphonic clients’ accounts I find that those using SiteCatalyst are far more likely to be actively using interface reports than those with HBX. A safe bet why is that in Omniture interface reports are much more flexible.  If you’re used to the complete freedom of Excel, they will make you feel like you’re trapped in a Columbian jail, but compared to other tools they are one of the biggest backyards in the neighborhood. 

If you, the heavy tool user at your company, are assigned to build reporting for others, you will notice several weaknesses in how reports are shared.  You can build a dashboard or singular report and allow other users to access it, but if you want it to be easily accessible for other users or appear for them upon login you'll have to login to their account and set those things up yourself.  No big deal if you’re setting up three accounts, but headache city if that number is double digits.    

To sum five and six up, Omniture will give you more interface reporting, exporting and scheduling options than HXB.  It's your call whether that's important to you or not (be aware that the above thoughts are relevant to things you can get immediately out of HBX and SiteCatalyst; to see my thoughts on how both tools Excel integration tools stack up click here).

8. Web 2.0-

This section was perplexing to me.  It is titled "Web 2.0" but really just discusses how Omniture can correlate data.  Last time I checked, even if your site doesn't contain blogs, social networking or rich internet applications, correlating data was still an important feature.

The main takeaway current HBX clients should have from this section is that when using SiteCatalyst you’re going to have to think about your data and extracting relevant things from it differently.  In HBX you might have built an Active Segment to understand the engagement level of people coming from a specific email campaign but with SiteCatalyst you’re going to have to rely on Subrelations/Correlations.  Unfortunately, Subrelations/Correlations are much less flexible than Active Segments.  Once you build an Active Segment you essentially can correlate it with anything else you collect data on, but with Subrelations/Correlations you have a finite number of things that you can correlate your data to.  If you anticipate your needs when setting up your account, then you should have no problem; but if the Subrelations/Correlations you set up don’t cut it for reporting or an analysis, you’ll have to pick up the phone and your wallet.

9. Genesis-

Numbers nine and ten are essentially strategically placed selfless plugs.  I'm sure when Omniture purchased VS they intended to convert the VS customer base to being not only SiteCatalyst clients but clients of the full spectrum of all things Omniture.  And Genesis would be one of those things.  Although I have limited experience with it, I do like the idea of being able to easily send out emails to specific segments of your site users.  I have heard some complaints the Genesis based paid campaign optimization technology isn't quite "there" yet, but I lack hands on experience to support that claim.

10. Platform-

Ten is the kicker.  The point that will undoubtedly linger with you.  The words that will leave you drooling over not only SiteCatalyst, but everything Omniture has to offer.  But Discover, SearchCenter, Offermatica and Touch Clarity and Genesis aren't the same as the pack of gum you add in the checkout isle.  They’re damn expensive.  If you’re debating between SiteCatalyst and Google Analytics, you almost certainly bother with these add ons, but for big budgets these will all be tempting in their own ways.  If you’re considering add ons just be careful not to out buy your bandwidth (ALERT: That last sentence was a Gross oversimplification).

I understand why Omniture lumped these things together, but you should mentally break them out.  Discover sticks out of that list as the only thing that doesn’t do something completely different than SiteCatalyst.  If you currently have active HBX users responsible for analysis and reporting in place, they should be able to understand and utilize Discover from (almost) day one.  In fact they might need to utilize the tool to extract data that you previously were able to get directly out of HBX.  The other tools are the outliers:  the ones that integrate (although not necessarily easily) with SiteCatayst, but do entirely separate things than SiteCatalyst.  These should all be evaluated individually with consideration to their benefit to your company and if you have resources that could not only handle but extract value from them.

Top 10 Benefits of Migrating to Omniture’s SiteCatalyst?

Despite the finalization of the Omniture acquisition of Visual Sciences, details of the future of the product line and migration process are as uncertain as ever. On Omniture.com the ridiculous “Learn about the migration program” link routes to a page of quotes from those currently benefiting from Omniture. Not only does this link route to a page of irrelevant information its salt in a wound. When I’m approaching a traffic jam the last thing I want is a call from my coworker, who left early, letting me know he was pulling into his driveway.
                                    

One of the most substantial pieces of “new” information is the narrated presentation of the top 10 reasons why HBX customers should switch to SiteCatalyst. To help HBX customers wade through this list of spoon fed Omniture rhetoric I figured I would provide a heavy tool users perspective on each reason why to switch. Here are my thoughts on 1-5. Expect 5-10 to be posted in a day or two.

Omniture’s Top 10

1. Admin Console

Omniture describes their admin console as a powerful tool for maintaining and sustaining all things SiteCatalyst and largely I agree with this description. The admin consol is fairly easy to use and gives you a lot of control over creating logins, specifying what content users can access and manipulating report suites. Its not quite as intuitive as the drag, drop, bada bing, bada boom interface the friendly narrator suggests, but the heavy tool users (who should be the only ones with access) should have no trouble using it. And once heavy users get a taste of it, it will be hard to remember how they ever lived without it.

If anything, my biggest issue with the admin console it that it allows users to change too much. In two clicks you can remove every page in a report suite or do numerous other actions that will do irreparably damage to your report suite or wallet (usually Omniture can fix your f*** ups, but it’s gonna cost ya). Additionally, it’s surprisingly easy to create undeletable entities, such as report suites and data categories here. It is common for higher up, light users to expect access to something called the “Admin Console,” but in interest of keeping your report suite looking clean and full of numbers this should be discouraged.

Conversely, there are areas of the admin console where you will reach a dead end and have to call Omniture Live Support. Sometimes it will make sense why you have to call, but other times, like when your building a rollup report suite, you’ll be a bit annoyed.

2. International Capabilities-

It’s nice that Omniture supports a lot of currencies. You still have to pick a single currency per report suite, so it’s not revolutionary system for handling currency but its still better than nothing. In reality no web analytics tools handle multiple currencies as well as they should, and this is a common area to run into implementation and reporting issues.

I can’t say much in regards to the fact that Omniture supports a slew of languages, but I was surprised that Omniture didn’t mention here (or anywhere in this list) the extensive amount of online help and documentation they have available. Heavy users will often find holes in the manual and Knowledge Base, but it’s still the best online help of any enterprise web analytics solutions. The value of good online support should be taken into consideration when you are evaluating your tool options. Omniture Live Support licenses must be purchased on a person to person basis and are NOT cheap. If you plan on having SiteCatalyst users in your organization that do not have Live Support access they will most certainly be digging through what is help is available online.

3. Version Support-

Omniture promises to support older versions of SiteCatalyst and I promise not to care. The biggest difference I notice from one generation to the next is the color scheme. Users would have to be VERY stuck in their ways or not be able to adapt to the minor changes of a new version of SiteCatalyst. Omiture’s willingness to support old versions makes then enablers, and it might actually benefit your business if they let older versions sail into the sunset when their time comes.

4. Custom Metrics-

I tend not to use custom metrics very often, but I do think they are cool. If your into stretching the limits of in interface reporting to the max custom metrics will help. You do have to be careful not to get too carried away building metrics, though, or your reports could become more confusing than it is helpful.

5. Segmentation-

Telling a group of people that are currently using HBX Active Segments that ASIs are a reason to switch to SiteCatalyst is just ludicrous. Powerful segmentation is an essential feature of any enterprise solution, so I understand why Omniture feels the need to brag about their capabilities, but they are barking up the wrong tree. ASIs are a massive step back from Active Segments. ASIs are sluggish and limited to 5 (which at times feels like it might as well be 0). In principal these things segmentation tools are nearly identical, but if you’re expecting ASIs equal to or better than Active Segmentation you’re going to be sorely mistaken.

The narrator also mentions SAINT as a segmentation tool. Although I don’t really agree that most of the uses for SAINT are related to segmentation, I do like SAINT. It’s a handy and fairly easy to use classification tool that allows you to easily change things like product numbers into product names or rollup products into relevant groupings. When rolling up you only get summations and not deduping, so look to the tool for page view data and not visit/visitor data. SAINT will give you a slightly easier development, since you won’t need to spend hours figuring out how to send Omniture an image request with the product name instead of the product number. Also SAINT will keep your report suites looking cleaner and easier to use.

The Good, The Bad, The Ugly of Internal Search

At Semphonic’s X Change conference within the internal search huddle someone raised the question, “Why would I not want people to search?”  The question was asked in a puzzled tone as if she sure the question itself was an immediate      end.  How could there ever be an instance where an internal search could be bad?

There’s a commonly held misconception that an internal search is always a good thing.  That it’s a positive step on the way to a success.  People blindly accept this notion and move onto analyzing internal search terms, where they suspect the real substance is.  And who could blame them with industry gurus telling them that this single action alone with tack on 10K to their salary within a year (pardon my acronym but -- wtf?). 

This blind faith can be incredibly detrimental to the success of your internal search analyses.  The set of internal search terms your tool collects contains limited information.  There are many instances where the same term is one site users key to success and another site users plea for help.  Understanding which is which and when you don’t want people to be searching on your site you need to get down and dirty in your tool.

First things first you need to establish a baseline of how many visits include a search and how many searches there are a visit.  At this point, regardless of if 2% or 98% of visits include a search, you should have no inclination whether a search is “good” or “bad.”   

Next you should gain understanding of where in a visit a search fits (this paragraph, and most of the remainder of the blog assumes that your internal search tool is available on all or most site pages).  A page from and page to report only shows you one step in each direction, but that alone should be revealing.  Things to notice are if people are mostly coming from home, key site routers, other products, etc. and if people are going to products, routers, exits, etc.  Often grouping pages by content categories will make understanding where search fits into your site easier.

Getting a baseline for how all your site visitors as a whole are utilizing search is the easy part.  Where you have to stretch your mind and your tool is with segmentation.  More often than not different groups will use internal search very differently. A common dichotomy is between first time visits and repeat visits.  Sometimes first time visits try to avoid internal search like the plague, while repeat visitors are drawn to it like a magnet.  Sometimes its visa versa.  Usually it’s way more complicated. 

To try to determine if you want your first time visitors to be searching start by building a segment.  Then utilize that segment to pull the same data you just did for all site visitors.  Key things to compare are the percent coming from the home page and the percent of exits from the search results page but really you should compare it all and see if anything sticks out.  Then take it one step further and build segments of first time visitor who searched and those who did not search and check out conversion rates of each.  You might want each segment to be only of those who saw 2+ pages or some other basic engagement metric filter out single access visits and balance out the fact that someone who is searching is in some way engaged with your site.

You will also want to go through a similar process with 2+, 10+, 20+ returning visits visitors to compare behavior the searching behavior of multiple groups.  This (hopefully) will give you some perspective of if you want everyone to be searching right away, if its more effective to have your new visitors to navigate via links while returners utilize search immediately or some other variation.   

You, knowing your site better than I do, might see similar population splits as new visitors vs. returning visitors.  These might deserve a higher priority than understanding what role in the conversion process search plays for new and returners and in that case apply a similar method as above to the two, three, four segments you have in mind.

And if your shaking your head and mumbling that you don’t have the time for a sizable analysis proposed above try a single short one. Although building a segment that breaks out a group of users who had an internal search and a site success is self selecting, this is sometimes and good way to get suggestions for what might be working.  By seeing when and why this group is searching you can prevent the potential mistake of saying internal search is always good.

So why would you not want people to search?  I bet you can come up with a reason or two.

An Open Letter to Omniture

Dear Omniture,

Congratulations on expanding your client list.  The wall in the entry way of your Orem office just got all the more impressive.  In fact you might need to let it spill over onto a new wall.  Luckily, it does not appear that new walls will be hard to come by.

As you dismantle a company with equally good products (this is a gross over implication, but lets call it better for some worse for others), backed by significantly less successful salesmen and ineffective management please consider retaining at least one piece.  HBX Report Builder is better than SiteCatalyst’s Excel Client, no if ands or buts about it. 

With Report Builder I have more formatting flexibility so it is easier to make my reports come out looking the way I (and Semphonic’s clients) want them to.  Report Builder also has fewer bugs.  Although I occasionally lose my data requests with Report Builder, never populates incorrect data or words instead of data or fails to populate new data altogether.

Report Builder also has cooler features.  It lets me copy and paste requests with respect to absolute or relative reference cells.  This feature combined with proper planning when setting up my requests saves me hours a week.  The ability to group and mass edit requests by account, date range and granularity also has saved me incalculable hours.  Matching and trending in Report Builder is very nice for building one request off another.  And Report Builder makes it so easy to include Active Segment data, which is invaluable (I realize this isn’t really an issue with the report building tools more of an AS vs. ASI, but I will still miss the capability).

And most importantly of all when I’m using Report Builder I produce reports fast, much faster than in Excel Client.  So fast in fact I almost always pull up Report Builder instead of HBX when I’m grabbing data.  In contrast I’m only using Excel Client when I absolutely have to be.  As an analyst the amount of time it takes to get large volumes of data counts in front of me counts for a lot.  If gives me all that much more time in my day to make the numbers meaningful.

Report builder is simply more linear, intuitive and moves faster from one step to the next.  If you do end up integrating the best features of the two tools it would be a mistake to integrate them into Excel Client.  Pride aside, make the right business decision and use the technology you just bough. 

Thanks and I look forward to working with you a lot in the near and distant future.

Regards,
Jesse

Stop Wineing- Start Understanding: Musings on Web Analytics Engagement

In 1976 a now infamous wine tasting was held in Paris, France.  The “Judgment of Paris” was a blind taste test of French and California wines held on French soil and judged by a panel of French wine aficionados designed to prove once and for all that France produced the finest wines in the world.  Seemingly unbeatable French Chardonnays and Cabernet Sauvignons were pitted against the same varietals from California.  And, when all the votes were tabulated and all the Frenchmen drunk with arrogance, the unanimous results were read: California produced the best wine in the world.

In 2007 a debate is raging over the relevance of and the use for “engagement” in web analytics reporting and analysis.  A week and a half ago Avinash argued that we should delete the word “engagement” from our reporting and replace it with “excuse.”  This caused many bloggers including Anil Batra, Captain Blackbeak, Eric Peterson and Jacquis Warren to pick apart Avinash’s argument and Gary Angel to tear it apart.  Engagement itself has become such a hot topic Manoj Jasra posted a blog with little more than 19 links to engagement related blogs and articles.  And despite (or more likely as a result of) all the content many questions about when, how and if you should use engagement remain unanswered.

So you ask what is a recap of wine tasting that predates me (see picture to upper left) and a link library to the ramblings on engagement doing in the same blog.  Well, let me ask a question in Freakenomics format to pull it all together…

Q: What does the Judgment in Paris and engagement have in common?
A: They both could have different outcomes depending on what you chose to measure.

In 1976 French judges were trying to get their ballots back, claiming the competition results were falsified or refusing to comment to the press because it was “too painful” because they had selected a Chardonnay from Chateau Monelena and a Cab from Stags Leap (both located in the Napa Valley—now aren’t you disappointed you didn’t book an extra night or two after the X Change) as their top wines.  For those involved, it was enough that, according to the competitions rules, California wines beat French wines.  Never mind that three out of the top four reds were all from France, or the fact that if numerous other methods had been used for choosing the winner France would have come out on top (given the same scores). 

Engagement isn’t all so different than the blind wine tasting.  Depending on what you define engagement as you might see a variety of winners.  It occurs quite frequently that the switching of your engagement definition will switch the rankings of campaigns that lead to the highest percent of engagement. 

To me, this neither shows the strength nor weakness of including engagement in a report or presentation, instead, it is juncture that separates a good analysis from the ones that react like angry judges.  Understanding the implications of your engagement definitions is critical before it is ever used in a report or presentation.  A proper understanding can only come about by seeing the effects of many different engagement definitions upon your metrics.  Even if you end up using “two or more site page views,” as your measure for engagement, you comprehend why that definition is better for your business than “four clicks of X, two clicks of Y…”

The only way to gain understanding is getting down and dirty in your web analytics tool, preferable, but not limited to, one built with an analysis in mind.  Omniture Discover or Visual Sciences can very quickly show you the affect of changing your engaged definition five, ten, fifteen times.  Now you can compare one form of engagement to another, the significance of your engagement percent compared to other site metrics and intelligently communicate why engagement has real estate in your reports.  My only caution is make sure you always step back and thoughtfully analyze your choices, picking the engagement definition that works best for your business and not simply what looks best.

Engagement is dangerous when you give it weight without understanding.  That’s a slippery road that can lead to misallocation of campaign spending, inappropriate assessments of successes and a laundry list of others.  But when you spend time thinking about engagement in the context of your website you usually come out with a valuable metric for your website. 

   

Building Excel Based Web Analytics Reports

As promised, here are some tips and tricks for building Excel based web analytics reports.  In the future, I will actually use paragraphs in my blogging, but until then you'll get my thoughts in numbered form.   

All Tools   

1.    Build data requests based on other data requests. This gives you the ability to always get visits, visitors, pages to, etc. for the top pages, no matter what the top pages happen to be.  It can also give you complete flexibility over reporting date range.   
2.    Anything that can be referenced in a cell should be referenced.  Referencing will make it possible to copy and paste the same request (if your tool has this feature) and get back different data.  Even if you're not trying to build a copy and pastable report, referencing everything will keep your reporting cleaner, easier to change/update and be much faster to trouble shoot (Disclaimer: I stick to this when building reports within HBX’s Report Builder, but not necessarily all of the all tools.  I have never seen HBX lose a cell reference, but other tools like Excel Client, are less reliable and can lose or misuse references.)
3.    Excel Vlookups are your friend…for several reasons.
a.     When pulling data for specific site elements (pages, links, etc.) you can either build a request that pulls data for only that element or you can pull a bunch elements all at once.  For tools that allow you to copy and paste requests, it is often better to get individual element data with individual requests, but for those where that is impossible the vlookup is usually your fastest bet.  It is almost always the case when pulling a large block of site elements, like pages, that their order will change from time period to time period.  Usually you can be pretty confident that your order confirmation will be amongst the top 200 most viewed pages each month, but you can bet the farm it won’t be rank 67 every month.  The solution for this is to insert a vlookup to seek out the order confirmation page data. 
b.     If you would like to show how a data element has changed over time its usually helpful if those elements line up.  Since vlookups need fixed values to look up against, you’ll have to decide which period you will use to fix the values.  Sometimes it is best to use the most recent period of data as your fixed vlookup values so that the most recent period will be in ascending order.  However, there are also instances where building a request to span the entirety of the time period you're looking at be at a more granular level and use the names within that as your fixed values.  This is your best bet.  If you're looking at campaigns, referrers or anything that might have only had significant volume during a sliver of the time you're looking at then it's best to use a vlooup list connecting to data that spans the length of time your seeing.
4.    Dates and date ranges visible to viewers of reports should all be requests or references of requests. Doing this will make it impossible to forget to update the dates in a report and give you an extra check to ensure your report has refreshed.

HBX Report Builder

1.    As much as you can build cookie cutter reports.  Cutting and pasting ten requests five times is much easier than building fifty individual requests.  There are many strategies behind doing this.
a.    Report Builder lets you edit multiple requests at once.  Your limited to changing the report date range, granularity or account – but within those limitations you can still do a lot.  It quickly converts a generic report for one account to a report for an entirely different account or makes a weekly report a monthly report nearly instantly.
b.    Anything that can be referenced in a cell should be referenced (aside from the mass edit possibilities mentioned above). Referencing will make it possible to copy and paste the same request and get back different data.  Even if you're not trying to build a cookie cutter style report referencing everything, it will keep your reporting cleaner, easier to change/update and be much faster to trouble shoot (Note: this tip is under the HBX section and not all of the tools sections because I have never seen HBX lose a cell reference, but other tools like Excel Client, have had past bugs where do not properly refresh
c.     References can either be absolute or relative, which works perfect when your referencing one cell but gets dicey when your referencing two cells, one absolute and one relative.  When that is the case the trick is to make both relative.  To make both relative you can add in extra rows or columns, copy the absolute values into them and then hide them.   
d.    HBX tends to lose your active segments within requests when you pick it from the drop down list. It cannot lose the segment if you write it into an Excel cell and reference it.  Even if you don’t reference anything else, always reference your active segments. It will save you a ton of headaches.

Omniture’s Excel Client

1.    When building two or more similar data blocks do not press “Done” as soon as you have pressed “Insert Now.”  Once you have inserted your data block you can go back a step, make the necessary changes and then insert another new block (An easy one, yes, but I didn’t want to leave Omniture out).