Creating a Multivariate Test

Introduction to testing

A/B testing is an activity that consists of challenging, or testing, elements of a website to find out whether an alternative will increase engagement and conversions. A/B testing is also a great way to identify the next innovation worth investing in, or for determining what is minimally viable. 

AB Tasty offers several types of testing campaigns with regard to your departure hypothesis and what you want to challenge.

To learn more about the differences between each type of test, you can discover our definitive guide to find the best test and if you don’t know what type of test to choose, our virtual assistant Ally can help you. 

You’ve chosen to create a Multivariate Test

A Multivariate Test consists of modifying several elements on one page (e.g. homepage, basket page, etc.) or a string of equivalent pages that share the same layout (e.g. all of your product pages, all of your result pages etc.), and measuring if one of the new combinations of variation(s) will be more or less performant than the original - for example, changing the shape AND the color of a button, generating several combinations of shapes and colors to test.

Part 1: Create a new campaign

Just like the other types of campaigns, a Multivariate Test can be created directly from the campaign dashboard, audience manager dashboard, Chrome extension, and/or when duplicating and transforming tests. 

👉 Directly from the dashboard

On the test dashboard, hover over the CTA Create, and you’ll discover the following list. Click on Multivariate Test to enter the creation flow. 

👉 With the Chrome extension

AB Tasty has released a Chrome extension to create a campaign directly from your website page. 

To download the Chrome extension, please follow this link.

This option is useful for loading logged pages (anchor).

👉 Duplications and transformations

If you want to duplicate a previous Multivariate Test and create a new one (and potentially modify some elements of the original test setup), you can use the Duplicate option in the dashboard. 

• Click on the three dots on the right side of the campaign line you want to duplicate
  
  
• Click on the first option Duplicate
  
  

• The duplication modal opens
  
  
• Here, you can:
  
  
• Modify the account where you want to create your new campaign
• Modify the name of your new campaign (otherwise the name will be the same with (duplicate) added at the end)
  
  

For more information about duplication and transformation use cases, please refer to our specific article about this topic.

Part 2: The Main Information page

The Main Information page is the first step toward your new Multivariate Test campaign. 

General use case: pasting the URL you want to test

At the top of the page, the dropdown pre-selects “Multivariate Test”. At this stage, you can still change your mind and select a different type of test.

The fields Name and Sample URL are mandatory and must be filled in to be able to click Save.

The Sample URL is the exact page, or a sample of the pages that share the same layout, that you want to test.

In your campaign, the URL will be used: 

• To load the Visual Editor
  
  
• As a pre-filled targeted URL in the targeting step
• Note that this won’t be the final targeted page of your campaign. See the targeting article for more information. 

❗️Caution: Don’t use URL parameters in the sample URL field, otherwise you won't be able to save the step.

⭐️ Tips: Filling in a “Hypothesis” field is a best practice, especially for collaboration across teams. Making the objective of the Multivariate Test clear and easily understood in the hypothesis (e.g. “Testing which color and shape of the CTA on the product pages increases click rates”) will help you and your teams avoid confusion between versions in the Visual Editor.

📎 Note: Once you click Save, your Multivariate Test will be generated (with an unique ID) and will appear in the Tests dashboard. When you come back to the Main Information Page, everything will be editable except the type of test (the dropdown menu will no longer be available). At this stage, if you need to change the type of campaign, you’ll have to start from scratch or transform your current Multivariate Test to another type of test directly on the dashboard, using the duplication action.

Specific use case: logged pages

By default, when you load a page that requires an authentication (e.g. a client account page), or a conversion funnel page, the page displayed in the editor will be empty or will show an error because it often requires session information (e.g. products to be displayed in the cart page).

To work around this issue, you need to inject the source code of the page:

 
There are two ways to do this:

• Using the Load Editor with embedded source code option manually (as shown)
  
  
• Using the Apply HTML option within the AB Tasty Chrome extension (see this article)

The first method with source code consists of pasting the page’s source code directly into AB Tasty. To do this:

• Go to the URL that you want to load in the editor
  
  
• Right click on the page, and select Inspect
  
  
• In the Elements tab of the console, go to the first line of code: the <html> tag
  
  
• Right click on it, then select Copy → outerHTML. You will get the entire source code of the page with all the scripts and information needed
  
  
• Paste this code into the dedicated window within the Main Information step
  
  
• Caution: The URL field needs to remain filled with your URL - this is mandatory in order to save and go to the next step
  
  
• Click Save
  The page will load in the editor and you can apply your desired changes.
  
  

Part 3: The content of your campaign

Usually, there are two ways to create your content for your campaign in AB Tasty: with the Visual Editor and with the Code Editor. For Multivariate Tests, the Code Editor is not available. 

With the Visual Editor:

The Visual Editor enables you to visually create the variations you want to test on your website. For example, a blue CTA instead of the current red one, or a new welcome message on your homepage.

The Visual Editor enables you to manage your Multivariate Test by adding or removing new variations and creating visual modifications.

To learn how to use the Visual Editor, discover options, and dive into our widget libraries, please read the Visual Editor Guide. 

With the Code Editor:

This feature is not available for the Multivariate tests. 

To learn more about the Code Editor, and instances when you can use it, please read the Code Editor Guide. 

Part 4: Goals

In Multivariate Tests, you can’t add goals with our standard goals setup step - it is not available for this type of test.

You can only set up Click Trackings that you add though the Visual Editor and retrieve them in the report, which is specific to this kind of campaign. 

For more information about Multivariate Test reports, please refer to its specific section in our Reports guide.

Part 5: Targeting

Targeting is one of the most important moments of the campaign setup. 

You have to declare WHERE your campaign should be visible on your website (on every page, on specific pages, on only one page, etc.), for WHOM (all of your traffic, mobile users only, etc.) and in WHICH conditions (after a certain number of pages, when it’s cold outside, etc.).

Please refer to our complete guide about targeting and learn how to set up segments of visitors, saved pages, and saved triggers to save time on your future setups.

Part 6: Traffic allocation

The traffic allocation step enables you to declare if you want to test all your targeted traffic or just a portion of it. It also allows you to change the iso repartition between variations if you need to.

To learn more about traffic allocation, please read the specific section about Multivariate Tests in this complete guide. 

Part 7: Advanced options

The advanced options are not mandatory. 

Third-party Integrations (analytics)

We integrate with a number of analytics tools, and you can link your Multivariate Test to your preferred analytics tool(s). By doing this, you will be able to analyze your testing cohorts in your own dashboards. To learn how to integrate your analytics tool with AB Tasty, please refer to the following guide (especially if you want to set up the API connection at the account level) to avoid having to set it up in each specific campaign. This way, the connection will be fully automatic (for new campaigns moving forward).

If you want to set up an analytics integration in a specific A/B Test, you can find your analytics tool in the dropdown and select it.
Some analytics tools, like Google Analytics, require additional settings, which appear in a form panel.  

Once launched, your test will start sending data to all the analytics service(s) that you configured here.

Part 8: QA

The QA portion of the campaign is one of the most important steps before launching your campaign into production. 

Please refer to our QA & QA Assistant guide for detailed information on QA.

Part 9: Scheduler

If you want to launch your Multivariate Test at a specific hour on a specific day, and/or if you want to pause it at a specific hour on a specific day, please refer to our guide to scheduling.