PART VI · In practice · CONCEPT 22 OF 24
A 1% test is not a 100% load test.
EXPLORE~7 min
Back to journey- A tiny canary checks basic safety, but a cache that works at 1% can fail at full traffic.
- Warmup, contention, and capacity can make treatment effects depend on saturation.
| Concept | What it means | Northstar example |
|---|---|---|
| Ramp to protect buyers. |
| Northstar starts with a monitored safety canary and a tested rollback path before exposing a large share of buyers. |
| Keep cohorts and phases interpretable. |
| Log Northstar's allocation, configuration, and capacity phase boundaries; pooling a 1% canary and a saturated full ramp blindly can change the question. |
| Combine different kinds of evidence. |
|
|
LAB 22
Safe ramps & load · Northstar
SATURATION SCHEMATIC
Below the supplied threshold
Constructed error rate: 0.10%. Small-scale success does not predict the whole curve.
This deliberately simple nonlinear load illustration is not a capacity forecast. Ramps require measured saturation, rollback capability, phase-aware analysis, and safety monitoring. Do not interpret the supplied threshold as an engineering recommendation.
Constructed teaching example · not Amazon data or a production experimentation engine
A canary succeeds at 1% traffic. What is still unknown?
Your learning note
Private · just for youFurther reading: Microsoft Experimentation Platform ↗NIST statistical handbook ↗