đź’ˇ Good to know This feature is currently available in Early Adoption. Please contact your CSM to enroll into the Early Adopter program. |
A Minimum Detectable Effect (MDE) is the minimum uplift you should reach in a given timeline to consider the results of a campaign as statistically significant.
MDE is important in the implementation of an A/B test because it directly influences how long the test needs to run and how large a sample size is required.
The MDE can be calculated using open-source tools, but you can also base it on data calculated directly by AB Tasty.
MDE calculation methods
You can either choose to rely on data AB Tasty collected on your previous experiments (that have been live and collected data) [COMING SOON], or enter data you want to let AB Tasty calculate it manually.
Manual Calculation
The form
You can still manually calculate the MDE of your future experiments by providing data you can have in your third party tools.
The, you must indicate the following data:
- Daily/Weekly/Monthly visitors
- The reference Conversion rate on your primary goal
- The number of variations you want for your future A/B test
The results
The calculation results are displayed as a graph. Its structure is the same as the AB Tasty Based data calculation.
The value indicated by the curve is the growth you should reach if you want your test to be statistically significant in the given timeline. Considering that, you might ask yourself if you can reach 40% growth in 14 days. If it’s not the case, your experiment might not be worth launching.
The next step
After analyzing your MDE results, you can either create a campaign or try another configuration and start the process again.
Use cases
As a CRO manager, I want to build a test on a specific page of my website, but I’m afraid of the time the statistical significance can take to be reached. Therefore, using the MDE with the AB Tasty based data will help me understand if I should (or shouldn’t) launch my experiment on that perimeter.