A/B Buddy 4+

Sample Size & Power Calculator

Evan Miller

Designed for iPad

    • 4.0 • 4 Ratings
    • Free
    • Offers In-App Purchases



If you're planning to run an experiment – either on the web or in real-life – it's critical to collect the right amount data so that you know the experiment results are scientifically valid.

A/B Buddy is a sample-size calculator that helps you design A/B experiments with a quick slide of a finger. It uses a power-analysis formula to tell you exactly how many subjects, or users, that you'll need in an experiment in order to detect a statistically significant difference between two groups. Because if you have too few users in the experiment, you'll never see any difference between "A" and "B"; if you have too many enrolled, the experiment will never end. A/B Buddy tells you exactly the right number.

How does it work? Let's say your current conversion rate is 10%, and you want to see if a proposed change from the design team can boost that number up to 12% – that is, a 20% lift in the conversion rate. Simply pop open A/B Buddy, set the Baseline conversion rate to 10%, and the Minimum Detectable Effect to 20%, and your answer is waiting for you at the top of the screen.

A/B Buddy includes a Share button, which links to a web page displaying your sample size calculation. Your numbers, however, remain 100% private – all the information is stored in the URL fragment, and is never transmitted to any servers.

By default, A/B Buddy uses the most common statistical thresholds: 5% for the significance level (false-positive rate), and 80% for the power level (ability to detect the Minimum Detectable Effect). Power users can adjust these values after unlocking the Pro Features in-app purchase. But most users will find the defaults adequate for their needs.

Questions? Suggestions? Ideas? Feedback is always welcome at emmiller@gmail.com

What’s New

Version 1.1.0

• Full support for Dark Mode
• A/B Buddy now requires iOS 13 or later

Ratings and Reviews

4.0 out of 5
4 Ratings

4 Ratings

App Privacy

The developer, Evan Miller, has not provided details about its privacy practices and handling of data to Apple. For more information, see the developer’s privacy policy.

No Details Provided

The developer will be required to provide privacy details when they submit their next app update.

You Might Also Like