📈 A/B test checker

Before change

10 %

After change

6 %

(p = )


If I read that there is no significant difference, should I rollback my change?

Not necessarily, what it means is that you don't have enough data to make a decision. If you observe a 1 point increase in conversion over your next 100 visitors you can't be sure of anything. On the contrary, if you observe a 2 points increase in conversion over your next 1400 visitors, you are now able to make a decision. This tool is here to help you make them.

I don't get it, my conversion went from 1% to 2%, it nearly doubled and your tool tells me that there is no significant difference?

Yes, I know it can be a little bit confusing. Let me put is this way, this tool only tells you that you can't be absolutely sure that your conversion actually increased.

It means that you will need a larger sample (more visitors), with the same conversion rate, to actually be sure that there is a change.

How much should I wait before results become significant ?

This is a tricky question and this is exactly why I made this tool. How much should you wait before being sure that you observe a significant difference depends on two things. First it depends on the difference between your two conversions, the greater the gap, the faster you can be sure that there is a gap. Secondly it depends on the number of visitors you used to compute your conversions.

Can I read more about this somewhere ?

Sure, this tool is actually a simpler version of this one. It leverages statistics and probabilty to ensure you that you make the right decisions. If you really like mathematics, we suggest you read this wiki article and this one.

There are other tool like that on the web, why shoud I use this?

To my knowledge, this one is the only one giving you an estimate of when will your be significant if it is not alreay. Then this one is more lightweight and not cluttered with dozens trackers waiting to retarget you on Facebook or Linkedin 😉.

Who made this ?

My name is Pierre de Wulf. I left my full-time job in 2018 and I try ever since to build profitable buisinesses. You can follow me on Twitter as I continue to document my journey.

© 2020 by daolf