How to create an A/B Test

An A/B Test consists of modifying one element on one page (e.g. homepage, basket page, etc.) or a string of equivalent pages that share the same layout (e.g. all of your product pages, all of your results pages, etc.), and measuring if the new variation(s) will be more or less performing than the original.

To learn more about the differences between each type of test, refer to the Types of campaigns article. If you don’t know which type of campaign to choose, our virtual assistant Ally can help you.

To create an A/B test, go to Web Experimentation, click the Create button and select A/B Test.

👉 To learn about the different ways to create a campaign, please refer to How to create a campaign.

Configuration

Step 1: Main information

The Main Information page is the first step toward your new A/B Test campaign.

We recommend establishing a hypothesis of your test according to the following model: If I apply [this change on my webpage] to [this audience], then [it will impact] and enables to enhance [this goal]. And to add it in the hypothesis field.

👉 For more information on how to configure this step, refer to How to set-up Main Information step.

Step 2: Editor

You have two ways to create content for your campaign in AB Tasty: 

With the visual editor

With the Visual Editor, you can make simple modifications such as changing the color of a button, replacing an image, changing the wording of an element. If it is clickable, always put an action tracking on the element you modify.

👉 To learn how to use the Visual Editor, discover options, and dive into our widget libraries, please read the Visual Editor Guide

With the code editor

The Code Editor enables you to declare your code (JavaScript, CSS) without having to load the Visual Editor. The Code Editor is the best solution for developers who want to save time. To learn how to use the Code Editor, please read the Code Editor Guide.

👉 To learn how to use the Code Editor, please read the Code Editor Guide.

Step 3: Goals

As the primary goal enables you to determine which variation takes precedence over the others, you should choose the action tracking related to the element you have modified. Indeed, it is the user behavior that is more likely to be affected by the modification you have made in the editor.

👉 To learn how to configure Goals, please read Campaign flow: Goals step.

Step 4: Targeting

You have to declare WHERE your campaign should be visible on your website (e.g. on every page, on specific pages, on only one page, etc.), for WHOM (e.g. your whole traffic, mobile users only, etc.), and in WHICH conditions (e.g. after a certain number of pages, when it’s cold outside, etc.) it should appear. You can also decide WHEN visitors will see the campaign depending on the recurrence you choose.

Be careful when entering the targeted pages (in the Where section) of your test to ensure that it will display correctly on the pages you want to target (e.g. Product pages).

👉 To learn how to configure targeting, please read How to setup a campaign targeting.

Step 5: Traffic allocation

By default, traffic allocation is set equally for each variation (50/50 for an A/B test, 33/33/33 for an A/B/C, etc.). For statistical reasons, it is recommended to not make uneven traffic allocation. 

👉 To learn how to configure Traffic allocation, please read Traffic allocation.

Step 6: Advanced options

The advanced options are not mandatory. 

• Enable the third party tool(s) you have connected to AB Tasty to send your test data to your tool.
• Select a loading method for your campaign.
• Activate Sequential testing to detect if your experiment will not be successful at all based on the results of your primary goal (if it is based on a Conversion/Transaction Rate).

👉 To learn how to configure Advanced options, please read Campaign flow: Advanced Options.

Step 7: QA

The QA of the campaign is one of the most important steps before launching your campaign into production. The QA allows you to verify and test in real condition : 

- the targeting configuration
- the tracking
- the modifications you've made

👉 To learn how to use the QA Assistant, please read QA Mode & QA Assistant.

If you want to launch your A/B Test at a specific hour on a specific day, and/or if you want to pause it at a specific hour on a specific day, see our guide on scheduling.

Use cases

A/B tests can be used in the following cases: