Quick definition: A denominator is the explicitly defined set of eligible units against which a numerator is expressed. In an experiment, it determines who the metric represents and often determines whether a rate is causally interpretable.
What is a denominator?
A denominator is more than the number below a fraction bar. It is a membership rule: which users, sessions, orders, accounts, or other units have an opportunity to contribute to a metric. In “purchasers per eligible assigned user,” the denominator is every user who met the pre-defined eligibility rule and was randomized, including people who never visited, clicked, or purchased. In “purchases per search session,” the denominator is qualifying search sessions. Both rates can be useful, but they make different claims.
Every rate needs compatible units, timing, and eligibility. The numerator for a user conversion rate is usually the number of eligible users with at least one qualifying conversion in a fixed window; the denominator is the count of eligible users. Orders divided by users is orders per user, not conversion rate. A user numerator divided by sessions combines different units and is difficult to interpret. Write the formula in words before writing a query.
Denominators deserve attention because a treatment can change downstream behavior. If a new landing page changes who reaches checkout, checkout conversion among checkout starters may move even when purchases per assigned user do not. The rate is not necessarily wrong; it answers a conditional question. It is risky when presented as the overall effect of the experiment.
How denominators are constructed
Construct a denominator from a durable population table rather than from outcome events. Specify the randomization unit, eligibility time, inclusion and exclusion rules, stable identity key, attribution window, and zero-value treatment. For a user-randomized signup test, create one row per eligible assigned user. Join each user’s qualifying outcomes within the defined window and set the outcome to zero when none exists. This preserves the comparison that randomization created.
Use pre-treatment rules for primary causal denominators whenever possible. Geographic availability, account status before assignment, consent state at assignment, and a documented technical exclusion may be appropriate. Clicking, reaching a page, redeeming an offer, or generating an exposure event can be affected by the variant and should be treated as post-treatment conditions unless the estimand deliberately targets them.
| Metric | Numerator | Denominator | Interpretation |
|---|---|---|---|
| User conversion | Eligible users who convert | Eligible assigned users | Likelihood of conversion per assigned user |
| Orders per user | Qualifying orders | Eligible assigned users | Purchase frequency per user |
| Session conversion | Sessions with a conversion | Qualifying sessions | Efficiency per visit |
| Checkout completion | Completed checkouts | Checkout starts | Completion conditional on starting checkout |
Instrumentation and data mechanics
Denominator integrity depends on records that precede the outcome. Log eligibility evaluation, assignment, exclusions, assignment time, and the unit key. Keep the reason for exclusion rather than retaining only a final boolean. Log exposures separately so delivery can be diagnosed without redefining the randomized population. Event data alone commonly lacks records for people who did nothing, yet those zeros often belong in the metric.
Identity resolution is a denominator decision. A person may use several devices; an account may contain multiple users; anonymous visitors may later sign in. Document whether the unit is a device, person, account, or session and how aliases are resolved at the time of analysis. Apply the same identity logic to both arms. An identity match that succeeds only after purchase can overrepresent converters and turn an apparent technical convenience into selection bias.
Monitor denominator volume by arm, platform, country, release, and time. A sample-ratio mismatch can expose broken allocation or eligibility logic; see the published sample ratio mismatch guide. Also inspect eligibility rate, exposure rate, unmatched outcomes, and the raw numerator. A stable conversion rate can conceal offsetting movements in both parts of the fraction.
Experimental impact
In a conventional randomized test, the primary estimate often answers an intention-to-treat question: what happened to all eligible people assigned to each option? That makes the assigned eligible population a strong default denominator. It includes users who were assigned but did not load a page, did not see a message, or left immediately, because those possibilities are part of the experience being evaluated. An exposure-based denominator answers a narrower effect among delivered experiences and requires a clearly stated exposure rule.
Changing denominators after results appear is particularly dangerous. A team may remove “inactive” users because the resulting lift looks cleaner, but if treatment affected activity, the filter changes comparison groups. Pre-specify the primary population and label alternative denominators as diagnostics. The guide to primary and guardrail metrics helps keep one decision metric distinct from useful operational rates.
Practical QA example: trial conversion
A SaaS company tests a revised trial invitation. Its primary metric is paid subscription within 21 days per eligible invited account. Before launch it exports the eligibility snapshot: account ID, plan, region, assignment, assignment time, and exclusion reason. Each account receives one row even when no trial starts. A separate event stream records invite rendering and clicks, while billing records provide the subscription outcome.
- Reconcile population counts. Compare eligible assignments to the invitation service and investigate every exclusion category.
- Verify uniqueness. Ensure accounts are not duplicated by multiple users or invitation retries.
- Attach outcomes consistently. Count a paid account once in the 21-day window; retain zero for non-subscribers.
- Report a funnel separately. Invite view and trial start rates explain mechanics but do not replace subscriptions per assigned account.
- Check maturity. Compare only accounts with a full 21-day opportunity, and report late billing events separately.
The revised invitation raises trial starts among accounts that viewed it but has no clear effect on paid subscriptions per assigned account. That is not a contradiction. The downstream conditional denominator reveals engagement among starters; the assigned-account denominator answers whether the intervention created paid value at rollout scale.
Assumptions and limitations
No denominator is universally best. A ranking team may correctly prioritize conversion per query; a service team may need incidents per active account; a geo experiment may use store-week or region-day. The relevant unit follows from the intervention, decision, and randomization. What matters is that the denominator is defined before analysis, compatible with the numerator, and accompanied by uncertainty methods that respect repeated observations or clustering.
Primary denominators can dilute a narrow mechanism. If only 20% of users reach a feature by design, a feature-level conditional rate may be essential for diagnosis. Report it alongside—not instead of—the broader causal outcome, and say clearly that it is conditional. Confidence intervals quantify random sampling error but do not validate an ambiguous population; see confidence intervals in A/B testing.
Common denominator failures
- Starting from outcomes. This drops zero outcomes and changes the population.
- Using a treatment-affected funnel step. It can select different types of users in each arm.
- Mixing units. Use user/user, session/session, or another compatible pair.
- Hiding exclusions. Preserve reason codes and counts by arm.
- Averaging daily rates. Recalculate from summed compatible counts unless equal daily weighting is intentional.
- Ignoring maturity. Give both arms equivalent time to generate the outcome.
Frequently asked questions
Should the denominator be assigned users or exposed users?
Assigned eligible users are often the safer primary denominator for a rollout effect. Exposed users may support a narrower, explicitly defined estimand when exposure is reliably measured.
Why include users with no events?
They may be valid eligible units with zero outcomes. Excluding them changes the metric and can introduce treatment-driven selection.
Can a denominator change during a test?
It can only with clear documentation, but the resulting estimate may answer a different question. Avoid changing primary rules after seeing results.
Is a percentage always a conversion rate?
No. Percentages can represent errors, retention, exposure, or many other ratios. State numerator, denominator, unit, and window.
Summary
The denominator defines the population behind a rate. Build it from a pre-specified, durable assignment or eligibility population, align it with the numerator and analysis unit, retain zero outcomes, and expose changes in both parts of the ratio. A familiar percentage is trustworthy only when its membership rule is clear.
Sources
- Kohavi, Tang, and Xu: Trustworthy Online Controlled Experiments
- AB-Labz: Primary vs. Guardrail Metrics
- AB-Labz: How to Calculate A/B-Test Sample Size