Glossary · Design
Design
Plan tests that can answer a useful question without creating avoidable validity risks.
Glossary terms
Browse this topic
Adaptive Experimentation
Changing allocation or design as evidence arrives while preserving valid inference.
Adaptive Sequential Design
Planned interim analyses and adaptations that maintain statistical guarantees.
Always-Valid Inference
Methods designed for continuous monitoring without invalid optional stopping.
Before-and-After Analysis
What change-over-time comparisons can show and why they rarely establish causality alone.
Blocking
Grouping similar units before randomization to improve precision and balance.
Cluster Randomized Experiment
Design choices, assumptions, and practical guidance for experiments.
Cluster-Level Randomization
Design choices, assumptions, and practical guidance for experiments.
Conversion Lift
Design choices, assumptions, and practical guidance for experiments.
CUPED
Design choices, assumptions, and practical guidance for experiments.
Difference-in-Differences
Design choices, assumptions, and practical guidance for experiments.
Experiment Duration
Business cycles, seasonality, novelty, and stopping rules.
Factorial Design
Design choices, assumptions, and practical guidance for experiments.
Fixed-Horizon Test
Design choices, assumptions, and practical guidance for experiments.
Fractional Factorial Design
Design choices, assumptions, and practical guidance for experiments.
Full Factorial Design
Design choices, assumptions, and practical guidance for experiments.
Geo Experiment
Design choices, assumptions, and practical guidance for experiments.
Geo Lift Test
Design choices, assumptions, and practical guidance for experiments.
Group Sequential Design
Design choices, assumptions, and practical guidance for experiments.
Heterogeneous Treatment Effect
Design choices, assumptions, and practical guidance for experiments.
Hypothesis
How to write falsifiable, mechanism-based product hypotheses.
Multiple Testing
Bonferroni, Holm, FDR, variants, metrics, and segments.
Sample Size
Baseline, MDE, power, alpha, allocation, and duration.
Explore the glossary
Other topics
Fundamentals
Fundamentals
The conceptual foundation: what experiments are, how variants are compared, and how a testing program creates evidence.
CRO & funnels
CRO & funnels
Research-led optimization of the complete path from discovery to meaningful action.
Statistics
Statistics
Statistical terms will be published with notation, formulas, worked examples, assumptions, and interpretation warnings.
Metrics
Metrics
Metric pages will connect formulas to denominators, business outcomes, guardrails, and experiment use cases.
Data quality
Data quality
Checks that determine whether an experiment result can be trusted before it informs a decision.
Implementation
Implementation
The delivery and measurement foundations behind reliable product and web experiments.
Product analytics
Product analytics
Concepts for understanding user behavior, product value, and the outcomes experiments are meant to improve.
Privacy
Privacy
Responsible measurement concepts for experimentation, marketing, and product analytics.