# A/B Test Calculator

> Check A/B significance and sample size.

A/B Test Calculator is a client-side statistics utility in MyBox. It analyzes two-arm conversion results and plans the sample size needed before a test.

## Analyzer

- Requires integer visitor and conversion counts for variants A and B.
- Shows conversion rate, absolute difference, relative lift, z-score, two-tailed p-value, and the 95% confidence interval for the absolute difference.
- Verdicts are withheld until each arm has at least 1,000 visitors and 100 conversions, and neither rate is 0% or 100%.
- The p-value is the probability of data this extreme under no real difference; it is not the probability that B is better.
- The relative-lift interval is an approximation derived from the absolute interval.

## Sample size

For a two-sided test:

```text
p2 = p1 * (1 + mdeRelative)
n = ceil((z_{1-alpha/2} + z_power)^2 * (p1(1-p1) + p2(1-p2)) / (p2-p1)^2)
```

Example: 5% baseline, 10% relative effect, 95% confidence, 80% power gives about 31,000 visitors per variant; at 2,000 daily visitors that is about 32 days.

## Peeking and limits

- Do not repeatedly check the test and stop at the first significant result; peeking inflates false positives. Fix the sample size and test window before starting.
- The calculator does not model sequential or Bayesian testing, segments, or multi-arm tests.
- It does not read analytics providers or store experiments.

## Notes

- This Markdown twin is static. Open the interactive page for calculations.

## Related

- [A/B Test Calculator](https://mybox.id/ab-test-calculator)
- [Campaign Link Builder](https://mybox.id/utm-builder): Build campaign links with UTM tags.
- [Apps](https://mybox.id/apps): Every app with a short description.
