What is A/B testing?
A/B testing is a form of research that assesses the appeal of candidate options. This technique mimics the way things are judged in the real world. It also allows surveys to be kept short, which makes the approach well suited to research conducted via mobile phones. It is, however, more expensive than a single survey covering several test options as it requires a larger total sample size.
Where can A/B testing be used?
It’s suitable for testing a wide variety of phenomena – including new product ideas, advertising material, store displays, in-store messages and webpage layouts.
How is A/B testing undertaken?
Critically, each option is tested amongst a single band of people – rather than all options being shown to the same sample. It therefore avoids rational responses as respondents can’t compare options. In setting up the sample it’s important that the profile of each test cell perfectly matches the others. If this is not the case different results could merely be a function of different audiences. If A/B testing isn’t used it is important to rotate the order in which stimulus is shown – otherwise respondents are primed by the first option and subsequent scores are anchored to this.
What’s an example of A/B testing?
We used this technique to test the introduction of a strapline for a major retailer. 5 versions, a control option plus 4 candidates, were each put before a sample of 400 respondents. In each case, straplines were shown on various pieces of stimulus including a web homepage, an advert, an example of packaging and a piece of in-store messaging. Critically, respondents were only asked to give their gut response to the stimulus taken as a whole i.e. they were never asked to home in on the strapline alone. This ensured they consumed the material the same way as shoppers would in a real-life situation. Using image association alongside scores for instant appeal allowed us to identify a strapline which was both appealing and in harmony with the desired brand positioning.