You've probably audited your Google Analytics setup and validated the data roughly matches data in your CRM etc (bonus points if you perform this QA process regularly).
How often do you audit tracking for Optimizely, VWO, Convert.com or other SaaS testing tools? Once a year? Just at implementation? Never?! It's no wonder we find the data in these tools trackers to be rather wonky.
And, how do your testing tool's data collection features compare to GA's?
NB. This assumes you've set up your GA properly.
Report on experiments with strongly vetted & maintained data#
Google Analytics data is fundamentally better for reporting on experiments than SaaS tools' built-in trackers. Don't just rely on SaaS testing vendors' black box analytics solutions to work as you expect.
SaaS testing tools perform data collection and enrichment inside a black box for simplicity. Source: Krauss on Wikipedia
But if you're reading our blog, you likely already track experiments into GA via Events/Custom dimensions. But perhaps you're not tracking them into Google Optimize. With Optimize, you get reports that are actionable, easy to use and built on top of the same data you tend to every day:
Clear and easy-to-use reports based on your Google Analytics data. Source: HappyIdiots.nl
Are you happy with session-based testing?#
I was surprised to learn Google opted for session-based units for testing. For longer-term buying cycles that may take weeks or months, a user-level unit of assignment is likely to be more useful.
If this doesn't matter to you, read on!
How to integrate Optimizely / Convert.com / VWO with Google Optimize#
Start by creating an Optimize account... but don't install any tags yet! Yours is already installed. It'll be either your Optimizely, Convert.com, VWO or Mojito tag)... no need for more split testing tools to slow down your page load times.
When you have an account, create a new experience:
In the fly-out menu that opens, create an "A/B test" or "Multivariate test" and set your URL to something arbitrary, e.g.:
http://BANANAS
Most of the time, you'll just want a simple A/B test.
Next, set up your experiment with variants and objectives. Then, grab the Experiment ID from under the "Measurement and objectives" section:
This experiment ID will come in handy shortly.
Your variant IDs will simply be 0 through n variants - see below as we tag your variants up.
Tag your variants in Optimizely/VWO with the Google experiment & variant IDs#
You'll need to push the Experiment ID and variant ID from each variant in Optimizely / Convert.com / VWO (or whatever you're using), into Google Tag Manager via the dataLayer.
For each variant in your testing tool, code like this can be used to track the experiment & variant IDs:
// ControldataLayer.push({ 'event': 'mojito_exposure 1-0-0', 'mojito': { // Standard experiment tracking 'waveId': obj.options.id, 'waveName': obj.options.name, 'recipe': obj.chosenRecipe.name },
// Add your Optimize keys in your assignment/exposure tracking 'gaExpId': 'Eostf1IbTje3Zjcx1vfZng', // Add the GA experiment ID 'gaExpVar': '0' // Add the variant ID here (0=control, 1=treatment one and so on)});// Treatment 1dataLayer.push({ 'event': 'mojito_exposure 1-0-0', 'mojito': { 'waveId': obj.options.id, 'waveName': obj.options.name, 'recipe': obj.chosenRecipe.name },
// Add your Optimize keys 'gaExpId': 'Eostf1IbTje3Zjcx1vfZng', 'gaExpVar': '1'});Variants are 0-indexed - So '0' = 'Original' and '1' = 'Treatment 1' and so on. Meanwhile, multivariate experiments follow this same format, but each factor is dash-separated. i.e. 0-1-0 (Factor 1: control, Factor 2: treatment 1 and Factor 3: control).
Next, collect those new variables in GTM/GA...#
With your variants tagged in Optimizely/VWO, configure GA to collect the experiment and variant IDs in an event like so:
Make sure these are set to "non-interaction" hits so you don't hide your true bounce rate. But understand this means you won't see "active users" while your experiment is running.
You'll need this event to carry the experiment ID and variant IDs to Google Optimize as custom fields, you'll need to:
- Hit "Enable overriding settings in this tag"
- Browse to "More settings" -> "Fields to set"
- Add two new fields:
- expId: Point to the the Google Optimize experiment ID variable
- expVar: Pointing to the 0-indexed variant ID
Setting these custom fields mimics the standard Optimize tracking and allows you to piggy back off the existing hits you're pushing into GA.
Publishing the experiment & starting reports#
When you're ready to publish, you'll need to first hit "Launch" on Google Optimize, and then on VWO / Convert.com / Optimizely. Pretty soon you'll see reports in Google Optimize with performance of your variants against the metrics you picked.
Optimize's reports are surely not ugly.
And if that's not enough, add each variant as a segment in GA! Segment the data as you like:
This will create a segment on the fly.
Now you can explore your experiments in greater detail, using the data you know and trust... at least more than the SaaS tools.