Site icon Treehouse Blog

The Business Case for A/B Testing

Does design of a sales page matter? Traditional reasoning says that the product always remains the same no matter how you dress it up on the sales page. So, one should focus on making the product more awesome rather than investing time to make it look awesome. Well, the reasoning sounds plausible in theory but the data says it is not well grounded.

Case Study

This post is about a recent A/B split testing case study where a redesign of a sales page resulted in 20% increase in sales. AquaSoft is a leading software company in the area of digital photo presentation. They have a complete portfolio of (desktop) software products related to photos – slideshow creation, desktop publishing, photo books, etc. Their software products can be downloaded for a free limited duration trial, following which a customer pays for the full version.

AquaSoft undertake more than 10 A/B tests per month using Visual Website Optimizer, an online A/B testing tool. They recently finished a test in which they tested their existing sales page against a new redesigned sales page. Note that this page wasn’t the free trial download page; rather it was a sales page where customers visit to actually buy the software.

Here is the control version of the page (click to view full version):

And here is the (final) redesigned version (click to view full version):

What’s different?

The pages are in German language, so let me point out a few key differences between the new and the control version:

A key point to note is that AquaSoft did not change their product features neither their guarantee policy. The details were simply not present (or highlighted) on the old sales page.

Two phase testing

AquaSoft carried the A/B test in two phases. In the first phase, old design was pitted against a variation of the new design. Improvements started with about 50% but settled on about 35-40% after some time. They were very happy with the result but craved for more details because in this test they were measuring clicks into the shopping cart, not the actual sales.

So they measured the sales which showed an increase of 17.7% of actual sales. It also showed that the bounce rate within the shopping cart was somewhat higher in the winning combination but the overall increased click rate outtakes the bounce rate.

In the second phase, AquaSoft refined the winning variation by doing minor changes such as reducing whitespace and placing software names above their respective box shots. This resulted in a further increase of 10% in clicks on shopping cart button and the overall increase in sales settled to about 20% (up from 17.7% in previous phase). All results were statistically significant, so the 20% increase in sales is real.

What can we learn form this?

This case study has several lessons for all of us who are serious about optimizing online sales:

  1. Design matters! If your sales page has a boring design, expect boring sales.
  2. A/B split testing is totally worth it! AquaSoft invests time, money and effort into doing as many as 10 split tests per month. And that is because they understand what impact a successful A/B test can have on the bottom-line.
  3. Some customers do abandon the checkout process! The next logical step for AquaSoft would be to optimize their checkout process but their payment system is integrated into a third party so it won’t be easy to set up a split test there.
  4. A/B testing is best done in phases. Small improvements in multiple phases ultimately add up to give a significant boost to conversions.

AquaSoft summarized the key take-home lesson from this A/B test:

“Don’t think your website is good as it is. Always test, always improve. It’s a slow process but worth doing. Our concrete test results show that a clear modern design improves sales. Especially in the sales process, trust building is very important.”

For AquaSoft, A/B testing required one-time investment of design resources but the 20% increase in sales will hopefully continue for months to come. If AquaSoft can manage to get such a fantastic ROI, so can you. Think about it – is your sales page optimally designed? What elements can you test on it?

Beta Invite

To try out Visual Website Optimizer enter invite code ‘carsonified‘ when signing up.

Exit mobile version