A Beginner’s Guide to Optimisation, A/B & MVT Testing.

At Building Blocks, we see site optimisation as a series of small, intrinsically important steps on the path to digital transformation. By investing in optimisation, you can de-risk projects, maximise campaign effectiveness, support commercial decisions, and provide the evidence you need to secure budget for larger projects.  But if you’re new to optimisation, you may be wondering where on earth to begin.

This article is the first in our series of optimisation blogs. At the end of the series, we’ll publish a free, downloadable ‘How To Guide and Tracker’ for you to use for your own optimisation initiatives. 

What do we mean by ‘optimisation’?

Before we get into the nitty gritty of A/B and MVT testing, let’s look at optimisation as a whole.

Adobe recently discovered that companies that embrace optimisation to make decisions increase conversion by 100%. Yes, you read right, 100%.  

Optimisation is the process of continual, small improvements to your digital touch points. It encompasses a number of things, including:

  • A/B and MVT testing
  • Analytics
  • CRO
  • User journeys
  • Usability
  • Information architecture
  • Traffic sources
  • Segmentation
  • User experience design
  • Personalisation

All of these activities help you to deliver a better customer experience, resulting in improved conversion rates. In this article, we focus specifically on testing. 

Why do we test

Testing enables you to make better-informed decisions when making website amendments, by measuring the impact of those changes on pre-defined metrics, such as purchases, downloads, registrations etc.

Testing also enables you to back up your ideas with scientific fact. Testing data is consistent, unbiased and trustworthy. It removes opinion and guess work when discussing potential changes, as all tests will be backed-up through comprehensive data. 

For example, imagine you conducted a usability test which discovered that users were having trouble locating the filters on one of your product results pages. Your internal design, UX and development teams may all have different ideas of what is the best solution to fix this problem. Rather than simply choosing one of these ideas, you can test the assumptions behind the ideas simultaneously, to uncover which solution works best for the end user.

Testing pays

According to research by eConsultancy, companies that are seeing a significant rise in sales are conducting 6.45 A/B and MVT tests per month. This is in comparison to the 2.42 test average among those companies whose sales are actively decreasing.

This tells us that dipping your toe into the testing waters is not enough to secure the uplift needed to surpass your business objectives. If you want to be among those witnessing a ‘significant rise’, you must embrace testing and use its valuable learnings to steer your digital development.

Testing is valuable at every stage. You should be doing continual small tests and using the known value to build business cases. Testing is about experimentation, and sometimes experiments go wrong – but it’s far more preferable to discover these bloopers in a test than to be blind to them once live. 

What is A/B and MVT Testing?

A/B and MVT testing are two types of testing methodologies you can use to understand whether your website or app is operationally efficient and delivering the right customer experience.

By delivering different web page variants to the users of your websites/apps, you can uncover the answer to that million-dollar question – ‘what works best for our users?’

The answer to this question is uncovered by monitoring the performance of the different variants over time against pre-defined metrics, such as registrations, sign ups, downloads, purchases etc.

A/B Tests

An A/B test is the most common and easiest type of test to conduct. It consists of creating alternative layouts of a specific page, and showing each of them to a certain percentage of visitors. A/B tests tell you which overall page layout has the most impact.


 Example of A/B test variants

MVT Tests

Multivariate tests experiment with elements inside one specific page. Elements are defined inside a page (e.g. a picture, a text or a button), and alternatives of each element are tested. The testing tool will show each element combined with all other elements to visitors. MVT tests help you to define which element change had the most impact. 


 Example of an MVT test element variants

In next week’s article, we’ll explain the underlying science behind testing and the meaning of ‘statistical significance’.

Find out more about Building Blocks’ site optimisation services.  

Building Blocks (part of Dept) is a digital agency based in Manchester (UK), Boston (USA) and Zaragoza (Spain) who help global companies with digital technologies


Want to know more? Get in touch.

Kyle Cassidy, Kyle specialises in user centred design,...

Ask your question!