A/B Experiment Lifecycle

flowchart TD
H[Hypothesis +\nprimary metric] --> N[Power analysis →\nsample size]
N --> Guard["Define guardrail metrics<br/>(latency, errors, revenue)"]
Guard --> Bucket[Bucket users 50/50]
Bucket --> Run[Run until N met]
Run --> Stop{Guardrail breach?}
Stop -- yes --> Kill[Stop early — keep control]
Stop -- no --> Done[Run complete]
Done --> Stat{p < 0.05 &\neffect size practical?}
Stat -- yes --> Ship[Ship treatment]
Stat -- no --> Learn[Document learning]