The DataHero Blog

Measuring A/B Testing with Google Analytics

June 25th, 2015


Tools like Optimizely are great for setting up and running A/B tests, but don’t provide much in-depth analysis for other metrics. They also provide the data in a silo. One test may yield more sign ups, but lead to lower revenue or less engagement. By integrating Optimizely and Google Analytics, you can pull Optimizely’s test data into DataHero and visualize the entire funnel, with all the metrics that matter to your business.

Pull your Optimizely test data into DataHero easily through the Google Analytics integration. Then simply select the A/B test attributes you’d like to visualize and DataHero will import and categorize it for you, then create suggested charts based on the unique data you have imported.Note that if you do not have your Optimizely account integrated with your Google Analytics account, you can find a step-by-step guide here.

Also note that Google Analytics doesn’t provide statistical significance data so if you’re unsure, export that info to Excel, perform some calculations, and then load the Excel file into DataHero.

Visualize Your Whole Funnel

Visualize three (or more) attributes from your funnel broken down by test to see how each test affects spend, number of transactions or other custom attributes that you’re looking for in a customer. For example, you may know that users who sign up for your newsletter tend to be more profitable as customers. Thus, a signup or single purchase is not the only attribute you need to know about from your A/B test.
DataHero Registrations vs. Avg. Number of Transactions

In the example chart above, we see that Homepage Tagline 2 showed fewer sign ups but actually a higher average number of transactions per customer. This may mean that even though we get more sign ups from one tagline, it doesn’t necessarily produce higher quality sign ups. For the CTA tests, we see something different. CTA test 2 outperforms test 1 in sign ups and average number of transactions.

Visualize User Engagement

This is similar to visualizing your funnel, but instead we’re monitoring user engagement. The business goal may not be to convert a user to a customer right away, but to encourage them to learn more about your product through white papers, demos, etc. These metrics, again, may not be immediately apparent within your Optimizely dashboard if you’re testing only for user sign ups.

DataHero Landing Page AB Tests (1)

The chart above represents a few different tests you might run on any given landing page, as well as some user metrics associated with that. If you’re optimizing strictly for revenue, you’d create a landing page with “visual 2”, no product video, with navigation and a shorter form. If you’re optimizing for newsletter sign ups you’d use “visual 1” with a product video, navigation and a shorter form. Looking at these tests as a whole can inform more strategic-level decisions.

Visualize Testing Data as a Whole

If you run multiple tests consistently, it can be difficult to see the big picture in your testing. Sometimes you can get so caught up in questions like “should this button be blue or red?” on a particular landing page, that we forget to pull up and see the whole strategy. Visualize test performance side by side to keep track of which tests are performing best or worst overall. From this high-level view you may notice that copy that is more value-based than feature-based performs best. You can then apply this to future tests or update your site as a whole to be more value-based.

An Integrated Data Analytics Solution

Create a dashboard to visualize all your A/B testing data side by side to pick up trends more easily and will show your up-to-date data. Then share these dashboards with other members of your team to allow everyone within your marketing department to leverage the insight you gain from testing.

A/B Testing

Make sure you maximize your A/B testing data to dig a little deeper into your users’ behavior, and share that information with your team to help you make the right decisions for your business. Don’t leave your Optimizely data in a silo outside your other marketing metrics. Create a free DataHero account and import your Optimizely/Google Analytics today.


Connect Google Analytics To DataHero


By Kelli Simpson

Create my Free DataHero Account

Get the fastest, easiest way to understand your data today.