A/B and Multivariate Testing

Web design involves making choices between several options. In making the final choice, most marketers simply use their intuition. In other words, they guess.

Given the high stakes involved, this unscientific approach is no longer appropriate. You need to know which design option will work best. This is where A/B and multivariate testing come in.

A/B Testing: one against another

As its name implies, A/B testing puts one version of a page against another. We use this process to test simple alternatives, such as the color or position of a "buy now" button.

Two versions of a page are developed
The two pages are presented to two groups of users
Statistics are gathered for both versions (click-throughs, conversion rates, etc.)
Post-test analysis reveals which version performs better

For best results, the process is repeated: two top-performing versions are pitted against each other, the design painstakingly refined for optimal results.

Multivariate Testing: comparing combinations of variables

All elements of a web page are inter-related: images, colors, headlines, calls to action, body text, etc. The purpose of multivariate testing is to determine which combination of elements works best together.

Every additional variable multiplies the number of pages that need testing. For example, testing just four different elements of your website – each with four different alternatives – requires testing 256 different pages!

New technology makes the technical/analytical part of the process fairly straightforward. But even the most sophisticated technology can't help you with the trickiest questions:

Which variables do I test?
For each variable, which alternatives do I test?

What should you test?

In large part, what you test depends on the type of website (and your business goals). For lead-generating websites, you may want to test:

Registration forms: test versions with fewer/more questions
Service descriptions: test longer, shorter and different versions
Typefaces: test different fonts and point sizes
Privacy assurances: test whether they help or just add clutter

For Ecommerce Websites: you may want to test:

Product photos: test different sizes, angles, resolutions
Product descriptions: test style and length of copy
Calls to action: test different wording and graphical treatment
Promotions: test different discounts, incentives, suggested purchases

Running the Tests

After deciding which variables to test – and developing the alternatives for each – it's time to run the actual tests. We use sophisticated software to present different options to various groups of users, and it's soon revealed which combination of features performs best.

Computerized testing makes it possible to test potentially thousands of home pages, landing pages, check-out funnels… whatever you desire.

What you should expect

The exact process will vary depending upon your business goals, timelines, and other requirements. But generally, we will:

                  Examine your website's most important areas of customer interaction
Perform a website analytics review to understand how customers interact with your website
Identify areas that could be improved
Recommend an appropriate testing methodology
Develop alternate pages for testing (or help your design team do so)
Conduct the tests, ensuring accurate and statistically significant results
Present our findings in a clear, actionable format

		        