Future of Web Apps Miami - Conference 22-24 February 2010

News Flash

New article on Think Vitamin: "Finding your Flock in Twitter" by @briansuda http://bit.ly/9w4QPH

Blog:

17 February 2010

The Business Case for A/B Testing

By Paras Chopra

Does design of a sales page matter? Traditional reasoning says that the product always remains the same no matter how you dress it up on the sales page. So, one should focus on making the product more awesome rather than investing time to make it look awesome. Well, the reasoning sounds plausible in theory but the data says it is not well grounded.

Case Study

This post is about a recent A/B split testing case study where a redesign of a sales page resulted in 20% increase in sales. AquaSoft is a leading software company in the area of digital photo presentation. They have a complete portfolio of (desktop) software products related to photos – slideshow creation, desktop publishing, photo books, etc. Their software products can be downloaded for a free limited duration trial, following which a customer pays for the full version.

AquaSoft undertake more than 10 A/B tests per month using Visual Website Optimizer, an online A/B testing tool. They recently finished a test in which they tested their existing sales page against a new redesigned sales page. Note that this page wasn’t the free trial download page; rather it was a sales page where customers visit to actually buy the software.

Here is the control version of the page (click to view full version):

And here is the (final) redesigned version (click to view full version):

What’s different?

The pages are in German language, so let me point out a few key differences between the new and the control version:

  • The new design gave a modern, fresher vibe on the page and had better contrast.
  • The old design had two calls to action per product – one for instant download and other for purchasing software in a shrink wrapped box. The new design reduced the choice to primarily one option (instant download) and the call-to-action was also transformed into a snazzy button (as compared to a simple text link in old design).
  • The new design was optimized to build trust in the customer. The sidebar in new design contained different kinds of guarantees:
    • Free lifetime updates and bonus software
    • Lifetime guarantee on the software
    • 24h/365 days support
    • Free customer login id
  • The new design also incorporated a seal made out of their logo to provide authenticity to the guarantees

A key point to note is that AquaSoft did not change their product features neither their guarantee policy. The details were simply not present (or highlighted) on the old sales page.

Two phase testing

AquaSoft carried the A/B test in two phases. In the first phase, old design was pitted against a variation of the new design. Improvements started with about 50% but settled on about 35-40% after some time. They were very happy with the result but craved for more details because in this test they were measuring clicks into the shopping cart, not the actual sales.

So they measured the sales which showed an increase of 17.7% of actual sales. It also showed that the bounce rate within the shopping cart was somewhat higher in the winning combination but the overall increased click rate outtakes the bounce rate.

In the second phase, AquaSoft refined the winning variation by doing minor changes such as reducing whitespace and placing software names above their respective box shots. This resulted in a further increase of 10% in clicks on shopping cart button and the overall increase in sales settled to about 20% (up from 17.7% in previous phase). All results were statistically significant, so the 20% increase in sales is real.

What can we learn form this?

This case study has several lessons for all of us who are serious about optimizing online sales:

  1. Design matters! If your sales page has a boring design, expect boring sales.
  2. A/B split testing is totally worth it! AquaSoft invests time, money and effort into doing as many as 10 split tests per month. And that is because they understand what impact a successful A/B test can have on the bottom-line.
  3. Some customers do abandon the checkout process! The next logical step for AquaSoft would be to optimize their checkout process but their payment system is integrated into a third party so it won’t be easy to set up a split test there.
  4. A/B testing is best done in phases. Small improvements in multiple phases ultimately add up to give a significant boost to conversions.

AquaSoft summarized the key take-home lesson from this A/B test:

“Don’t think your website is good as it is. Always test, always improve. It’s a slow process but worth doing. Our concrete test results show that a clear modern design improves sales. Especially in the sales process, trust building is very important.”

For AquaSoft, A/B testing required one-time investment of design resources but the 20% increase in sales will hopefully continue for months to come. If AquaSoft can manage to get such a fantastic ROI, so can you. Think about it – is your sales page optimally designed? What elements can you test on it?

Beta Invite

To try out Visual Website Optimizer enter invite code ‘carsonified‘ when signing up.

3
CSS3 Online Conference March 22nd 2010 banner ad

3 Comments

Have your say:

Subscribe to our Newsletter

Sign up to the Think Vitamin Newsletter to get updates on web design, web development and web entrepreneurship as well as special offers and discounts from Carsonified. Rest assured we never share your email address.

Subscribe to the Think Vitamin articles RSS feed

CSS3 Online Conference March 22nd 2010

News

Twitter

Follow us on Twitter

Subscribe

Article Subscribers

Feedburner blog subscriber indicator

News Subscribers

Feedburner blog subscriber indicator

Subscribe by Email

You can receive Think Vitamin updates via email. Just pop your email address in the box below and click the arrows.

Subscribe by RSS

You can also receive new Think Vitamin posts via your RSS feed reader

Subscribe RSS Think Vitamin is a proud member of the Smashing Network

Ads Via The Deck