A/B Testing

What is A/B Testing?

In simple words, an A/B test is a strategy where marketers compare two different versions of an asset, such as an advertisement or a campaign, and measure the performance of each version. It is mostly set on the premise of a hypothesis claiming to generate better KPI’s for the end goal of the asset. The two versions are typically exposed to equal halves of the target audience at the same time, and the ‘winning’ version is determined in regards to the goals set by the marketer.

However, it typically doesn’t stop there. Once the winner is determined, marketers usually go on to test other versions with the goal to convert more and more leads to paying customers.

What is an example of A/B testing?

If your goal is to optimize your campaign for clicks and conversions, you would first identify the current behavior of your audience. This is typically done through the use of heatmaps or other analytics tools. You would use these tools to answer questions like:

1. What are my users clicking?
2. What are they not clicking?
3. What do I want them to start clicking?

The next step would be to use this information on their click behavior to generate various hypotheses that could lead to more and more converting users. If your end goal is to get more people to sign up to your product or offering, then you can use the places where users are currently clicking the most to lead to the sign up page. Alternatively, you can hypothesize that users are not clicking on a particular image or CTA because of the weak imagery or unattractive choice of words and replace them.

Once you have listed your hypotheses, you would pick the strongest ones and launch an A/B test showcasing the changes for a set (and equal) amount of time. The A/B test can be run between a treatment and a control group where the treatment group experiences the change and the control group remains as it was. Alternatively, if you are running a new campaign altogether and don't have any past data on clicks and other user behavior, you would be testing out two new hypotheses at the same time.

After a set amount of time, you would compare the conversion rates for the new design (or the treatment group) to the control group. Did the new design get you more clicks, and therefore more conversions? If your answer is yes, then you would stick to the new design and move on to the next A/B test.