Layout
PART V · Hidden traps · CONCEPT 20 OF 24

Your control group may be using the same queue.

EXPLORE~8 min
Back to journey
  • The new cache reduces load on a database shared with control.
  • Control pages also get faster.
  • The buyer-level contrast may understate what would happen if the whole store moved to the new stack.
Shared infrastructure: concepts and Northstar examples. Illustrative Northstar examples, not measured business results.
ConceptWhat it meansNorthstar example
Interference breaks isolation.
  • A buyer’s outcome can depend on other buyers’ assignments through shared queues, cache warming, inventory, sellers, or capacity.
  • The standard no-interference assumption no longer holds.

Northstar's new cache may reduce a database queue shared with control, helping control buyers and shrinking the measured buyer-level contrast.

Pick a design for the mechanism.
  • Isolated comparable infrastructure pools can help.
  • Randomizing regions or clusters requires enough independent clusters and cluster-aware inference.
  • A single treated region versus one control region is weak evidence.

Consider isolated pools or enough randomized infrastructure clusters for Northstar's mechanism, with inference at the actual independent unit.

Switchbacks need washout.
  • Alternating treatment over randomized time blocks can target shared systems, but cache state, time-of-day demand, and carryover matter.
  • Randomize blocks, allow washout, and account for temporal dependence.

A Northstar switchback must account for cache warming, lingering state, demand cycles, and washout rather than simply alternate busy and quiet hours.

LAB 20

Shared infrastructure · Northstar

Buyer A
Existing stack
↘ Shared database / queue ↙
Less load helps both arms
Buyer B
New cache
Treatment improvement vs old world100 ms
Control improvement from spillover50 ms
Observed randomized contrast50 ms

A supplied linear spillover schematic, not a queueing model or an estimate of full-rollout impact. Increase spillover: a beneficial system change can yield a small buyer-level contrast. Real saturation may be nonlinear; isolated pools, cluster designs, or randomized switchbacks need their own assumptions and inference.

Constructed teaching example · not Amazon data or a production experimentation engine
CHECK YOURSELF · CHOOSE AN ANSWER

Treatment reduces a queue shared with control. What should you question?

Your learning note

Private · just for you