Glossary · Statistics
Statistics
Statistical terms will be published with notation, formulas, worked examples, assumptions, and interpretation warnings.
Glossary terms
Browse this topic
Alpha (α)
The false-positive threshold, why it is chosen before launch, and its link to power.
Alpha Spending
How sequential tests allocate false-positive risk across planned interim looks.
ANOVA
Comparing group means, interpreting the F-test, and when it fits multi-variant experiments.
Average
The arithmetic mean, when it can mislead, and how to interpret average experiment metrics.
Bayesian Inference
How priors and observed data produce posterior estimates for experiment decisions.
Bayesian vs Frequentist
Different questions, outputs, priors, stopping, and practical choice.
Bayes’ Theorem
The relationship between prior beliefs, new evidence, and updated probabilities.
Bernoulli Distribution
The probability model for one binary outcome such as a conversion.
Beta (β)
Type II error, its relationship to statistical power, and planning trade-offs.
Beta Spending
Managing false-negative risk and futility boundaries in sequential designs.
Bias
Sources of systematic error that can distort an experiment estimate or decision.
Binary Metric
Defining and analyzing outcomes with two states, such as converted or not converted.
Binomial Distribution
Success counts, assumptions, and why the distribution matters for conversion metrics.
Binomial Metric
A metric based on successes and opportunities, with a clearly defined denominator.
Bonferroni Correction
A conservative way to control false positives across multiple comparisons.
Bootstrap
Resampling observed data to estimate uncertainty when analytic formulas are difficult.
Causal Inference
Definition, assumptions, interpretation, and analysis context.
Causation
Definition, assumptions, interpretation, and analysis context.
Chi-Square Test
Definition, assumptions, interpretation, and analysis context.
Confidence Bound
Definition, assumptions, interpretation, and analysis context.
Confidence Interval
Formula intuition, uncertainty, width, and experiment reporting.
Confidence Level
Definition, assumptions, interpretation, and analysis context.
Confidence Threshold
Definition, assumptions, interpretation, and analysis context.
Consistent Estimator
Definition, assumptions, interpretation, and analysis context.
Continuous Metric
Definition, assumptions, interpretation, and analysis context.
Correlation
Definition, assumptions, interpretation, and analysis context.
Credible Interval
Definition, assumptions, interpretation, and analysis context.
Descriptive Statistics
Definition, assumptions, interpretation, and analysis context.
Distribution
Definition, assumptions, interpretation, and analysis context.
Efficient Estimator
Definition, assumptions, interpretation, and analysis context.
Error-Spending Function
Definition, assumptions, interpretation, and analysis context.
Estimator
Definition, assumptions, interpretation, and analysis context.
Expected Loss
Definition, assumptions, interpretation, and analysis context.
Expected Uplift
Definition, assumptions, interpretation, and analysis context.
External Validity
Definition, assumptions, interpretation, and analysis context.
False Discovery Rate (FDR)
Definition, assumptions, interpretation, and analysis context.
False Negative
Definition, assumptions, interpretation, and analysis context.
False Negative Rate (FNR)
Definition, assumptions, interpretation, and analysis context.
False Positive
Definition, assumptions, interpretation, and analysis context.
False Positive Rate (FPR)
Definition, assumptions, interpretation, and analysis context.
Family-Wise Error Rate (FWER)
Definition, assumptions, interpretation, and analysis context.
Fisher’s Exact Test
Definition, assumptions, interpretation, and analysis context.
Frequentist Statistics
Definition, assumptions, interpretation, and analysis context.
Holm–Bonferroni Correction
Definition, assumptions, interpretation, and analysis context.
Independent Samples T-Test
Definition, assumptions, interpretation, and analysis context.
Inferential Statistics
Definition, assumptions, interpretation, and analysis context.
Internal Validity
Definition, assumptions, interpretation, and analysis context.
Linear Regression
Definition, assumptions, interpretation, and analysis context.
Logistic Regression
Definition, assumptions, interpretation, and analysis context.
Mann–Whitney U Test
Definition, assumptions, interpretation, and analysis context.
Mean
Definition, assumptions, interpretation, and analysis context.
Median
Definition, assumptions, interpretation, and analysis context.
Minimum Detectable Effect
Business value, sample-size trade-offs, and realistic MDE setting.
Mode
Definition, assumptions, interpretation, and analysis context.
Multiple Comparisons
Definition, assumptions, interpretation, and analysis context.
Normal Distribution
Definition, assumptions, interpretation, and analysis context.
P-value
Correct interpretation, null hypothesis, common misreadings, and examples.
Paired T-Test
Definition, assumptions, interpretation, and analysis context.
Posterior Distribution
Definition, assumptions, interpretation, and analysis context.
Power Analysis
Definition, assumptions, interpretation, and analysis context.
Prior Distribution
Definition, assumptions, interpretation, and analysis context.
Probability of Being Best
Definition, assumptions, interpretation, and analysis context.
Probability of Superiority
Definition, assumptions, interpretation, and analysis context.
Ratio Metric
Definition, assumptions, interpretation, and analysis context.
Regression Analysis
Definition, assumptions, interpretation, and analysis context.
Regression to the Mean
Definition, assumptions, interpretation, and analysis context.
Resampling
Definition, assumptions, interpretation, and analysis context.
Sample Size Calculator
Definition, assumptions, interpretation, and analysis context.
Selection Bias
Definition, assumptions, interpretation, and analysis context.
Significance Level
Definition, assumptions, interpretation, and analysis context.
Standard Deviation
Definition, assumptions, interpretation, and analysis context.
Standard Error
Definition, assumptions, interpretation, and analysis context.
Standardization
Definition, assumptions, interpretation, and analysis context.
Statistical Power
Power, beta, sample size, MDE, and underpowered decisions.
Statistical Precision
Definition, assumptions, interpretation, and analysis context.
Statistical Significance
Evidence, practical significance, confidence, and decision boundaries.
Statistical Test
Definition, assumptions, interpretation, and analysis context.
Survivorship Bias
Definition, assumptions, interpretation, and analysis context.
T-Test
Definition, assumptions, interpretation, and analysis context.
Two-Proportion Z-Test
Definition, assumptions, interpretation, and analysis context.
Type I Error
Definition, assumptions, interpretation, and analysis context.
Type II Error
Definition, assumptions, interpretation, and analysis context.
Variance
Definition, assumptions, interpretation, and analysis context.
Z-Test
Definition, assumptions, interpretation, and analysis context.
Explore the glossary
Other topics
Fundamentals
Fundamentals
The conceptual foundation: what experiments are, how variants are compared, and how a testing program creates evidence.
CRO & funnels
CRO & funnels
Research-led optimization of the complete path from discovery to meaningful action.
Metrics
Metrics
Metric pages will connect formulas to denominators, business outcomes, guardrails, and experiment use cases.
Design
Design
Plan tests that can answer a useful question without creating avoidable validity risks.
Data quality
Data quality
Checks that determine whether an experiment result can be trusted before it informs a decision.
Implementation
Implementation
The delivery and measurement foundations behind reliable product and web experiments.
Product analytics
Product analytics
Concepts for understanding user behavior, product value, and the outcomes experiments are meant to improve.
Privacy
Privacy
Responsible measurement concepts for experimentation, marketing, and product analytics.