PART I · Foundations · CONCEPT 3 OF 24
A buyer should not switch teams on every click.
EXPLORE~6 min
Back to journey- Northstar buyers browse repeatedly and may use several devices.
- Randomizing each request creates mixed experiences and correlated observations.
- Our default is a stable buyer-ID assignment for logged-in eligible buyers.
| Concept | What it means | Northstar example |
|---|---|---|
| Keep assignment sticky. |
| A stable Northstar buyer ID stays on the same cache variant across repeat visits instead of switching treatment on every request. |
| Analyze at the unit you randomized. |
|
|
| Shared infrastructure can force another design. |
| If the new cache changes a database queue shared with control, Northstar may need isolated pools, clusters, or randomized time blocks rather than ordinary buyer buckets. |
LAB 03
Unit of randomization · Northstar
Control · existing stack
20% premium
Expected conversion: 8.0%
Treatment · new cache
80% premium
Expected conversion: 17.0%
True within-type treatment effect0.00 pp
Apparent lift from buyer mix+9.00 pp
One buyer’s next 4 requestsA → A → A → A
Premium buyers have 20% conversion; other buyers have 5%, on either stack. Randomization shows equal mixes in expectation here, not guaranteed finite-sample balance. Dot colors show buyer types. Sticky routing keeps one buyer on one experience; repeated requests remain correlated.
Constructed teaching example · not Amazon data or a production experimentation engine
A buyer makes 100 requests. How many independent randomized buyers is that?
Your learning note
Private · just for youFurther reading: Microsoft Experimentation Platform ↗NIST statistical handbook ↗