Quick definition: An average usually means the arithmetic mean: the total of observed values divided by the number of observations. It is a useful summary, but its meaning depends on the unit, denominator, time window, and treatment of unusual values.
What is an average?
In everyday language, “average” can mean several summaries. In statistics, it most often means the arithmetic mean. Given values x1, x2, …, xn, their sample mean is x̄ = (Σxi)/n. Each observation contributes equally to the total, so the mean is the balance point of the data.
The word can also refer to the median (the middle ordered observation), mode (the most frequent value), weighted mean, or geometric mean. Those are not interchangeable. A report should name the statistic rather than relying on “average,” especially when the measure informs an experiment or business decision.
The denominator defines the metric
An average has two parts: a numerator and a denominator. “Average revenue” could mean revenue per order, per visitor, per active user, per purchaser, or per day. Each answers a different question. Revenue per purchaser may rise while revenue per eligible visitor falls if a treatment reduces the number of people who purchase.
State the analysis unit and inclusion rules precisely. For example, average revenue per randomized user should assign zero revenue to eligible users who did not purchase. Average order value includes only orders, so it conditions on ordering. That condition can become problematic when a variant changes who orders; it can create selection bias rather than describe full business impact.
Important: Never compare averages whose populations or denominators differ between variants. A mean is not self-explanatory: define eligibility, observation window, currency handling, refunds, and whether zeros are included before analyzing results.
How arithmetic and weighted averages work
For five session values of 2, 3, 3, 4, and 8 minutes, the mean is (2 + 3 + 3 + 4 + 8) / 5 = 4 minutes. The total divided by the count is easy to calculate and has useful algebraic properties: group means can be combined when their group sizes are known.
A weighted average gives observations different influence. If group A has mean 10 from 100 users and group B has mean 20 from 10 users, the combined mean is not (10 + 20) / 2 = 15. It is (100 × 10 + 10 × 20) / 110 = 10.91. The weights must represent the relevant exposure or population, not merely whichever summary is available.
The mean is sensitive to extreme values because every value enters the sum. That sensitivity is desirable when the total itself is the decision target: a $10,000 order genuinely contributes more revenue than a $10 order. It is less desirable when an extreme value is a bot, logging fault, duplicate event, or irrelevant exceptional case. Data-quality rules should be defined consistently and checked in both arms.
Assumptions and boundaries
A mean can always be computed for numeric observations, but inference about a difference in means needs more. The experimental units should be independently randomized or analyzed with a method that accounts for clustering. Measurements need a stable definition, and the sample must represent the population to which the result will be applied.
Normal raw data are not required for every mean comparison. With enough independent units, the sampling distribution of a mean is often approximately normal by the central limit theorem. However, very skewed, heavy-tailed, sparse, or dependent outcomes can make standard errors unstable, particularly at small sample sizes. Bootstrap intervals, robust estimators, transformations, or models suited to the outcome may be better choices.
Always inspect distributional context: sample size, zeros, quantiles, variance, and the contribution of the largest observations. The median and percentiles are useful companion views, but replacing the mean with the median changes the estimand. Choose the summary that matches the product question rather than the one that looks most favorable.
Averages in A/B testing
Mean metrics are central to experimentation: average revenue per user, average order value, average time to complete a task, average number of support contacts, and average page latency. The usual estimand compares treatment and control means at the randomization unit: Δ = x̄T − x̄C. A relative lift is Δ / x̄C, but report the absolute difference too.
For a continuous metric, a two-sample t-test or regression estimates the difference relative to its standard error. A confidence interval shows both the likely magnitude and uncertainty; the confidence interval guide explains why it is more informative than a pass/fail label. For highly skewed revenue, teams may report mean revenue per user as the business estimand while using a bootstrap or robust analysis as a sensitivity check.
Do not calculate a daily average and treat days as independent observations if users are randomized individually. Nor should a user who visits repeatedly be counted as several independent users without a deliberate event-level estimand. The primary metric and guardrail metric framework helps specify which average drives the decision and which averages protect customer experience.
Worked example: revenue per visitor
A checkout experiment randomizes 10,000 eligible visitors to each arm for fourteen days. Control generates $50,000 in net revenue, so its average revenue per visitor is $5.00. Treatment generates $54,000, so its average is $5.40. The estimated lift is $0.40 per visitor, or 8%.
Suppose the standard error of the user-level mean difference is $0.16. The test statistic is $0.40 / $0.16 = 2.50; a two-sided p-value is about 0.012. A 95% interval is approximately $0.09 to $0.71 per visitor. The team has evidence of a positive average revenue effect and can translate the interval into forecast revenue at expected traffic.
It should still inspect the distribution. If one treatment user produced an erroneous $30,000 order, the estimate is not decision-ready. If the order is legitimate, removing it would change the real revenue question and requires a prespecified, defensible policy. The team also checks conversion, refund rate, and support contacts before rollout rather than using one average in isolation.
Interpreting an average
A mean is an expected value over the defined population; it is not a typical individual experience. In a long-tailed distribution, most users can be below the average. Pair it with median, percentiles, counts, and uncertainty where those views clarify the story. Report the unit: “mean net revenue per eligible randomized visitor,” not just “average revenue.”
When an experiment reports a mean difference, distinguish statistical from practical significance. A narrow interval around a tiny lift can be highly significant but not worth engineering cost. A wide interval around a promising lift may justify more data or a cautious rollout. See how to interpret A/B-test p-values for the distinction.
Common average mistakes
- Leaving the denominator implicit. Per purchaser, per order, and per visitor are different metrics.
- Averaging averages without weights. Group size determines a combined mean.
- Discarding outliers because they are inconvenient. Investigate them and apply a predeclared policy consistently.
- Using the mean as “typical.” In skewed data, report a median or quantiles as context.
- Ignoring dependency. Clustered or repeated observations need an analysis matched to the randomization unit.
Frequently asked questions about averages
Is average the same as mean?
Often, but not always. “Average” is informal; mean normally means the arithmetic mean. Name the exact statistic in an analysis.
When should I use a median instead?
Use a median when the middle experience is the question or when a robust typical-value summary is needed. Do not substitute it for mean revenue if total expected revenue is the decision target.
Should non-purchasers be included in average order value?
No; average order value is conditional on an order. For overall experiment impact, pair it with conversion or use revenue per eligible visitor, which includes zeros.
Can a few outliers invalidate an A/B test?
They can make estimates unstable or reveal a defect. Validate them, use prespecified handling, and add sensitivity analyses rather than silently deleting observations.
Summary
The arithmetic average is the sum divided by the count, but its value depends on a clearly defined numerator, denominator, unit, and population. In A/B testing, compare user-level means with appropriate uncertainty, inspect skew and outliers, and choose a metric that represents the product decision.
Sources
- NIST/SEMATECH e-Handbook: Measures of Location
- NIST/SEMATECH e-Handbook: The Central Limit Theorem
- OpenIntro Statistics