Baseline

What is a baseline?

A baseline is the reference state against which progress or change is measured. It can be a current conversion rate, an average order value, a page-load metric, a retention curve, a process time, or the existing product experience.

Eurostat describes a baseline study as an analysis of the current situation that establishes a starting point for assessing future progress [1]. In CRO, the baseline conversion rate is the rate observed before optimization work and becomes the internal reference for measuring the impact of later changes [2].

In an A/B test, “baseline” can refer to the experience, the control group, or the control metric. These ideas are connected but not identical:

Using these terms interchangeably creates confusion. A target is not a baseline, and an industry benchmark is not necessarily a suitable control estimate.

Why the baseline matters in experiment planning

The baseline affects how sensitive and feasible an experiment will be. Sample-size calculations for a conversion metric use the current rate, the smallest meaningful effect, the significance threshold, and statistical power.

Suppose a page has a baseline conversion rate of 5% and the team wants to detect a 10% relative improvement. The target rate is:

That is very different from a 10 percentage-point increase. Always state whether an effect is relative or absolute.

Baseline, MDE, and sample size

For a two-proportion comparison, a simplified planning relationship is:

Here, p is the baseline rate, δ is the absolute difference the test is designed to detect, α is the significance level, and β is the Type II error rate. The exact calculation may use pooled or separate rates and should match the statistical method.

The important planning relationships are more useful than memorizing the formula:

Optimizely’s sample-size guidance makes the same connection between baseline, MDE, significance, and required visitors [3]. MDE is not a property of the variant alone; it is a property of the experiment design.

How to calculate a useful baseline

1. Match the metric: use the same numerator, denominator, attribution window, and event definition that the experiment will use.
2. Match the audience: use the actual page, funnel step, device, source, geography, or user segment being tested.
3. Use a representative window: cover ordinary weekday and weekend behavior and avoid a one-day anomaly.
4. Check instrumentation: confirm events, identity, deduplication, and denominators before relying on the number.
5. Account for seasonality: promotions, holidays, campaign changes, and product releases can make old data irrelevant.
6. Use a range when necessary: if the baseline is uncertain, calculate sample sizes across plausible values.

A broad sitewide conversion rate may be a poor baseline for a mobile checkout test. A recent, mobile-specific rate for the exact checkout population is usually more useful.

Example: choosing between two baselines

A team plans to test a signup form for paid-search visitors. It has two possible inputs:

The 2.9% baseline is a better starting point. If the team used 4.8% in its sample-size calculation, it might underestimate the traffic required and assume more conversions than the experiment can realistically produce.

Baseline vs. control group

The baseline and control group are related, but they are not the same thing.

Historical baseline data helps plan the test. The concurrent control is the comparison that supports the final causal estimate. A before/after result against a historical baseline is not equivalent to a randomized treatment-control comparison.

Baseline vs. benchmark vs. target

These three reference points answer different management questions:

• Baseline: Where are we now?
• Benchmark: How do we compare with another reference population or standard?
• Target: Where do we want to go?

An ecommerce site may have a 2.4% purchase baseline, a 3.0% internal benchmark for returning users, and a 2.8% target for the next quarter. None of those numbers replaces a concurrent control in an A/B test.

Baseline quality checks

Use a baseline as a distribution, not just one number

A single average hides volatility. Plot the metric over time and inspect:

• daily and weekly variation;
• traffic-source mix;
• device and browser composition;
• campaign and product releases;
• holiday or promotional periods;
• outliers and tracking gaps;
• cohort and returning-user differences.

If the rate moved from 2% to 5% across the last month because of a campaign mix shift, choosing 5% as “the baseline” may be too optimistic. Use a representative range and explain the uncertainty in the experiment plan.

Example: baseline changes after a product release

A team plans to test a new checkout design. The historical checkout completion rate is 70%, but a payment-provider release reduced it to 62% last week. The team has three options:

The right answer depends on the experiment start state. A baseline should describe the population and system that the treatment will actually face, not a more convenient historical version.

Common baseline mistakes

1. Using a sitewide average: the test audience is narrower and behaves differently.
2. Using a benchmark as a baseline: external averages are not your current control.
3. Using a target: planning starts from hope instead of observed performance.
4. Ignoring metric definition: numerator or denominator changes between periods.
5. Using too short a window: weekday or campaign noise dominates.
6. Ignoring seasonality: holiday behavior is treated as normal.
7. Forgetting variance: the average is stable but daily values are volatile.
8. Changing the baseline mid-test: sample-size planning and interpretation drift.
9. Using baseline to declare a winner: the concurrent control is the causal comparator.

Baseline checklist

• Metric definition and denominator are documented.
• Baseline matches the exact test audience and funnel step.
• Data comes from a recent, representative window.
• Weekday, weekend, seasonality, and campaign effects are considered.
• Tracking, identity, deduplication, and event completeness are checked.
• Daily or weekly variability is plotted and reviewed.
• Baseline is expressed as relative and absolute context where useful.
• MDE is stated as a business-relevant effect.
• Sample size is calculated using baseline, MDE, alpha, and power.
• A range is used when baseline uncertainty is material.
• Concurrent control—not historical baseline alone—is used for final causal inference.

FAQ

What is a baseline conversion rate?

It is the current conversion rate of the experience or audience being tested before the experiment. It is calculated as conversions divided by eligible opportunities.

Is baseline the same as control?

Not exactly. Baseline describes a reference experience or metric; control is the randomized comparison group used during the experiment.

How recent should baseline data be?

Recent enough to reflect the audience, product, tracking, and traffic mix that will exist during the test. The window should also cover normal cycles and avoid one-off anomalies.

Can I use an industry benchmark as my baseline?

Use it for context, not as the experiment baseline. Sample-size planning should use your own current metric for the exact audience and denominator.

What if the baseline is unstable?

Investigate tracking, segment shifts, releases, seasonality, and outliers. If uncertainty remains, calculate a range of sample sizes and document the assumptions.

Why does baseline affect sample size?

The baseline affects the variance of a conversion metric and the absolute rate difference represented by a relative MDE. A lower baseline or smaller detectable effect usually requires more observations.

Sources

1. Eurostat: Baseline Study
2. Omniconvert: Baseline Conversion
3. Optimizely: How long to run an experiment
4. Baseline Conversion Rate in A/B Test Planning
5. MDE Calculator and Baseline
6. SplitMetrics: Baseline and Sequential Testing
7. Convert: Baseline

Related terms

• A/B Testing
• A/B Test
• A/B/n Testing
• A/A Testing
• Split Testing