Power: the chance you’ll notice a real effect.
- Northstar expects a small conversion gain, not a miracle.
- At a 10% baseline, a 2% relative lift is only 0.2 percentage points.
- Detecting that reliably can require hundreds of thousands of buyers.
| Concept | What it means | Northstar example |
|---|---|---|
| Specify an effect worth finding. |
| Northstar plans to detect a 2% relative conversion lift from a 10% baseline: only 0.2 percentage points in absolute terms. |
| Smaller effects cost more. | For this approximate two-sided 5% test at 80% power with equal groups, halving the absolute planning effect requires about four times as many independent buyers. | Halving Northstar's absolute planning effect approximately quadruples required independent buyers under the displayed normal-test assumptions. |
| Duration is not just traffic divided by n. |
| Divide by newly eligible distinct Northstar buyers, not repeat requests, then allow business cycles and seven days for included outcomes to mature. |
Power, MDE & sample size · Northstar
Approximate two-sided 5% normal test, 10% baseline, equal allocation, independent binary buyer outcomes. The traffic count is unique newly eligible buyers across both arms, not daily active users counted repeatedly. Add full business cycles, warmup, and operational constraints; this is not a universal sample-size calculator.