Quick definition: The alternative hypothesis, written H1 or Ha, is the formal claim an experiment evaluates against its null hypothesis—for example, that a treatment changes a conversion rate relative to control.
What is an alternative hypothesis?
In frequentist hypothesis testing, the alternative hypothesis specifies the pattern that would contradict a chosen no-effect model. For an A/B experiment with treatment conversion rate pB and control rate pA, a two-sided alternative is Ha: pB ≠ pA. A directional alternative could be Ha: pB > pA. Its counterpart, the null hypothesis, commonly states pB = pA.
The alternative is not a prediction that must come true, a marketing objective, or a description of every possible business outcome. It is part of a statistical model used to determine what evidence counts against the null. A product team may hope a new checkout increases orders, yet responsibly test for either an increase or a decrease because both are plausible and both matter to customers. The formal choice should follow the decision context and be set before outcomes are examined.
Why it matters in experimentation
A precise alternative turns a vague idea into an analyzable question. It guides the metric, the test statistic, whether a one- or two-sided procedure is appropriate, the sample-size calculation, and the interpretation of the result. It also forces the team to distinguish statistical evidence from practical value. An alternative saying “the rates differ” has no built-in statement about whether a 0.02 percentage-point change is worth deploying.
For that reason, pair the formal alternative with a practical decision rule. A test brief can say: “Among new self-serve workspaces, the shorter checklist will increase 14-day activation; we will roll it out only if the estimated increase is at least 1 percentage point, the uncertainty is acceptable, and support contacts do not rise beyond the guardrail.” The hypothesis describes the statistical comparison; the rule describes the product action.
| Form | Example | Use with care because |
|---|---|---|
| Two-sided | Ha: pB ≠ pA | It detects credible harm or benefit and is the usual default when both directions matter. |
| One-sided increase | Ha: pB > pA | It is defensible only when the opposite direction would not change the inference or action framework. |
| Minimum-effect claim | Ha: pB − pA > δ | It needs a justified practical threshold δ and an appropriate analysis method. |
One-sided versus two-sided alternatives
A one-sided test concentrates its rejection region in one direction. With the same nominal threshold, it can require less evidence for an increase than a two-sided test, but it gives up the ability to claim significance for a decrease under that procedure. It is not a formatting choice to make a favorable result easier to obtain. Select it only before looking at data and only when the decision genuinely treats the excluded direction differently.
In product work, decreases normally matter: they can harm users, revenue, or trust. Therefore, two-sided inference plus explicit guardrails is frequently more transparent. Even if a team would only ship a positive change, a negative estimate remains vital evidence. Never choose the direction after a result is visible; that uses the data twice and overstates evidence.
Failure to reject the null is not proof that the alternative is false. An imprecise test may simply be unable to distinguish a valuable benefit from no effect or harm. Read the interval and planned power, not only the threshold label.
Practical scenario: a shorter checkout form
A retailer proposes removing an optional address field. The mechanism is reduced friction. Before launch, it defines the primary metric as completed orders per eligible visitor within a seven-day window. The null is equal order rates. Because an unexpected decrease would be important, it uses a two-sided alternative. It also chooses a minimum worthwhile lift, plans sample size accordingly, and sets payment failures, refund rate, and revenue per visitor as guardrails.
At readout, the estimated order-rate difference is positive, but the interval ranges from a small loss to a small gain and does not exclude a meaningful negative effect. The team cannot honestly say it has established the alternative. Nor should it say the versions are equivalent. It should collect the planned additional information, improve precision, or decide that the potential gain is not worth further exposure. If the test instead provides strong evidence of a tiny increase well below the practical threshold, the statistical alternative may be supported while the rollout case remains weak.
Decision and use guidance
- Write the alternative alongside the population, treatment, metric, and time window before assignment begins.
- Choose a direction from the real decision and plausible harms, not from a desired p-value.
- Define the minimum practically important effect separately from the null-versus-alternative test.
- Use the same predeclared hypothesis for sample-size planning and final analysis.
- Report effect size and uncertainty with any threshold-based conclusion.
In multi-metric or multi-variant work, be explicit about which alternatives are confirmatory. Each extra claim is another opportunity for chance findings. A favored result among many post-hoc metrics or segments should be treated as exploratory and tested again. The alternative hypothesis is a commitment that helps prevent that kind of result-shopping.
Limitations and common mistakes
The alternative hypothesis does not guarantee that the metric is valid, that randomization succeeded, or that the effect generalizes beyond the test. It also does not express a probability that the treatment works; a p-value is computed under the null model, not the probability that Ha is true. Bayesian analyses use different model statements and interpretations, though they still require preplanned decisions and sound measurement.
- Writing Ha after inspecting the winning direction.
- Confusing “we want B to win” with a falsifiable, population-specific claim.
- Using a one-sided test only to lower the apparent bar for significance.
- Claiming equality when a test merely fails to reject the null.
- Ignoring multiplicity after testing many outcomes, variants, or segments.
Frequently asked questions
Is the alternative hypothesis the same as the product hypothesis?
No. The product hypothesis explains a mechanism and decision; Ha is the formal statistical claim about a parameter or comparison.
Should every A/B test use a two-sided alternative?
Not always, but it is often appropriate because both benefit and harm matter. A one-sided alternative needs a defensible pre-data rationale.
Can I change Ha after results arrive?
You may formulate a new exploratory hypothesis, but do not present it as the original confirmatory test. Validate it in a new experiment.
Does rejecting H0 prove Ha?
No. It provides evidence against the specified null under the model and procedure. Assess effect size, uncertainty, data quality, and practical consequences too.
How does practical significance fit in?
It asks whether the estimated effect is large enough to matter. It is a separate decision criterion from whether a statistical threshold was crossed.
Summary
The alternative hypothesis states the effect pattern a test is designed to detect relative to a null model. Define it before data is read, choose direction from the real decision, and pair it with a practical threshold and guardrails. It disciplines interpretation, but it cannot replace good design, reliable data, or business judgment.
Sources
- NIST/SEMATECH e-Handbook: hypothesis testing concepts.
- American Statistical Association statement on p-values.
- Kohavi, Tang, and Xu, Trustworthy Online Controlled Experiments.