A while back, Hayley posted a great article on improving the effectiveness of your website by looking at customer journeys throughout your website and making decisions on how to remove blockages in potential customers’ journeys to make sure your website goals are completed more often.
I’m going to introduce the idea of Multivariate Testing (or A/B split testing as it can be known), how to use it andwhat to expect from it which should aid in making your website even more effective.
I would say that a little knowledge of updating your website (or having a web designer handy!), empathising with your visitors and common sense is all that is needed to understand and utilise multivariate testing.
Multivariate Testing
Before we get a bit more technical, let’s define the term. Multivariate testing is quoted on Wikipedia as:
“In internet marketing, multivariate testing is a process by which more than one component of a website may be tested in a live environment. It can be thought of in simple terms as numerous A/B tests performed on one page at the same time. A/B tests are usually performed to determine the better of two content variations, multivariate testing can theoretically test the effectiveness of limitless combinations. The only limits on the number of combinations and the number of variables in a multivariate test are the amount of time it will take to get a statistically valid sample of visitors and computational power.”
This is a technical way of saying that we can offer slightly different versions of a webpage in your site to different visitors with the aim of finding out if the variations in each version of the page have made a positive impact. Doing this over and over, or with different areas of your website will evolve your website making it more and more effective over time.
When to use it
Multivariate testing can be time consuming and not the easiest thing to manage if used too often across your website. It is better to concentrate your time on running one or two tests at a time on key areas of your site. Try to use it in an area of the site that receives at least 100 visits per day otherwise the experiment may take a long time to complete.
Other occasions where it can be useful are:
- If a dispute over the best way to use a page arises within the web team
- If a dispute over the best way to use a page arises between the web team and client
- If a new design is proposed and needs to be tested for effectiveness
Where to use it
Key areas (in multivariate terms) were discussed in Hayley’s article. Basically, the most common customer journeys should be tested for effectiveness first. This could be the shopping catalogue, getting in touch or a search page depending on your site’s main goals.
You could also consider pages that you suspect to be ineffective or areas where you receive negative feedback.
What you will need to use it
To use multivariate testing successfully, you will need the following ingredients:
- A page to test
- At least one variant of that page
- A common goal of both pages
- Google website optimiser
A page to test
This has already been decided upon by now. It could be a contact form or part of your catalogue. It’s the page we’re going to improve.
At least one variant of that page
This is the alternative (and hopefully improved) version of this page. But what to vary? I would recommend keeping the changes in each variant rather small as if the changes are vast and the project is successful or unsuccessful, you haven’t learnt why. Subtle changes allow you to notice what has made a difference. If possible, change one or two things on the page. It could be the position of a button or the size of the product picture.
A common goal of both pages
If the goal of the original page was to make people get in touch, then the goal of the variant should be the same. Analysing two different goals in an A/B split test is simply not science.
Google Website Optimiser
www.google.co.uk/websiteoptimiser -This tool will be the platform for setting up your test and reading the results. There is no software to install as like pretty much all Google products it is web based. You may find other products you prefer (feel free to let us know and why in the comments) but Google Website Optimiser is free and widely accessible.
How to use it
Creating a new experiment
You should first of all create a new experiment and select either an A/B test or a multivariate one. This boils down to whether you have one variant or more of the page to be tested.
It is assumed at this stage that you have made your page to be tested and its variant live to the world. For instance you should have something like domain.com/page.htm and domain.com/page_alternative.htm accessible in a browser.
Identifying Pages
Then you must tell Google the web address of the test page, the variant page and the conversion page – the page that is the goal.
For a form this conversion page could be the thank you page (something like domain.com/thank-you.htm) or for a shopping cart it could be the basket page. For best results, the conversion page shouldn’t be too far away (in terms of actions like clicks) from the test pages.
Adding the tracking code
Now you (or your friendly web developer) will need to add some javascript (code) to the three pages – Google will guide you through what is needed where.
It is just a copy and paste job before making the changes live on your website. It’s a good idea to check these pages all look and work correctly at this stage.
Confirmation and Completion
Then Google will confirm that the code is added and start the experiment (asking you whether you want to start it yet).
Hurrah for science!
When anyone now visits the test page, they will either see the original page or the new version. Google will send traffic at random to one or the other, but once it has determined which you see, it will always be the case (as it may freak people out if they kept flicking between pages).
The Results are in!
Unfortunately the results aren’t in yet though. We now have to wait until sufficient data (normally 100 conversions for each variant in the test) stacks up to make a meaningful decision. You can look at your experiment results from about 24 hours after starting it but try to avoid jumping to conclusions so early on. Google will tell you when it has had sufficient data through an experiment. It could take a few days or a month depending on traffic to the page in question.
So what do you get when the experiment is done? Well, you get a really useful recommendation from Google on which variant has performed the best alongside the raw data in case you wish to interpret it yourself. Google have gone to great lengths to make an algorithm (complete with error margins) that decides which is the winner so most should be happy to be told which variant should win.
Conversions are graphed and a clear display of the ‘winner’ is shown as follows:
Conclusion
- Multivariate testing allows you to see what version of a page works best on your site
- Use multivariate testing to determine what tweaks work on your site
- Try to make tweaks small to help you understand what works and what doesn’t – a big variance results in you not knowing what has affected the result
- Make good decisions on what to test. Multivariate testing can be time consuming and it doesn’t want it be a wasted effort
Chime in below in the comments if you have any questions, suggestions or your own ideas. If you are interested in Senior optimising your website then let us know. We like to natter.