In 1976 a now infamous wine tasting was held in Paris, France. The “Judgment of Paris” was a blind taste test of French and California wines held on French soil and judged by a panel of French wine aficionados designed to prove once and for all that France produced the finest wines in the world. Seemingly unbeatable French Chardonnays and Cabernet Sauvignons were pitted against the same varietals from California. And, when all the votes were tabulated and all the Frenchmen drunk with arrogance, the unanimous results were read: California produced the best wine in the world.
In 2007 a debate is raging over the relevance of and the use for “engagement” in web analytics reporting and analysis. A week and a half ago Avinash argued that we should delete the word “engagement” from our reporting and replace it with “excuse.” This caused many bloggers including Anil Batra, Captain Blackbeak, Eric Peterson and Jacquis Warren to pick apart Avinash’s argument and Gary Angel to tear it apart. Engagement itself has become such a hot topic Manoj Jasra posted a blog with little more than 19 links to engagement related blogs and articles. And despite (or more likely as a result of) all the content many questions about when, how and if you should use engagement remain unanswered.
So you ask what is a recap of wine tasting that predates me (see picture to upper left) and a link library to the ramblings on engagement doing in the same blog. Well, let me ask a question in Freakenomics format to pull it all together…
Q: What does the Judgment in Paris and engagement have in common?
A: They both could have different outcomes depending on what you chose to measure.
In 1976 French judges were trying to get their ballots back, claiming the competition results were falsified or refusing to comment to the press because it was “too painful” because they had selected a Chardonnay from Chateau Monelena and a Cab from Stags Leap (both located in the Napa Valley—now aren’t you disappointed you didn’t book an extra night or two after the X Change) as their top wines. For those involved, it was enough that, according to the competitions rules, California wines beat French wines. Never mind that three out of the top four reds were all from France, or the fact that if numerous other methods had been used for choosing the winner France would have come out on top (given the same scores).
Engagement isn’t all so different than the blind wine tasting. Depending on what you define engagement as you might see a variety of winners. It occurs quite frequently that the switching of your engagement definition will switch the rankings of campaigns that lead to the highest percent of engagement.
To me, this neither shows the strength nor weakness of including engagement in a report or presentation, instead, it is juncture that separates a good analysis from the ones that react like angry judges. Understanding the implications of your engagement definitions is critical before it is ever used in a report or presentation. A proper understanding can only come about by seeing the effects of many different engagement definitions upon your metrics. Even if you end up using “two or more site page views,” as your measure for engagement, you comprehend why that definition is better for your business than “four clicks of X, two clicks of Y…”
The only way to gain understanding is getting down and dirty in your web analytics tool, preferable, but not limited to, one built with an analysis in mind. Omniture Discover or Visual Sciences can very quickly show you the affect of changing your engaged definition five, ten, fifteen times. Now you can compare one form of engagement to another, the significance of your engagement percent compared to other site metrics and intelligently communicate why engagement has real estate in your reports. My only caution is make sure you always step back and thoughtfully analyze your choices, picking the engagement definition that works best for your business and not simply what looks best.
Engagement is dangerous when you give it weight without understanding. That’s a slippery road that can lead to misallocation of campaign spending, inappropriate assessments of successes and a laundry list of others. But when you spend time thinking about engagement in the context of your website you usually come out with a valuable metric for your website.