Variant

What counts as a variant?

A variant is the condition an experiment exposes to one group instead of the baseline. It can be small—a different CTA label—or substantial—a redesigned checkout, a new ranking model, or an alternative onboarding flow.

Convert defines a variant as a distinct version of an element, page, or feature tested against the control [1]. Nielsen Norman Group uses “variant” for a design version of a variable, while “variation” can refer to the resulting complete design combination in a multivariate test [2].

In everyday product and CRO work, teams often use variant, variation, treatment, challenger, and test version as near-synonyms. The important thing is not the label; it is a clear record of what users saw and how the version differs from control.

Variant, control, and treatment

A variant is not automatically “better,” and it is not a winner before the test. It is a proposed explanation made visible in the product.

Build the variant from a hypothesis

A variant should answer a specific question. Start with the observed friction, the change, the expected outcome, and the mechanism:

Example:

The resulting variant should change the delivery-information component—not quietly rewrite the headline, price, layout, and payment flow at the same time. If the team wants to test a complete new concept, name it as a package and be honest that the test estimates the effect of the package.

What makes a strong variant?

• Relevant: it addresses a real user or business problem.
• Specific: the intended difference can be described and reproduced.
• Measurable: an outcome should plausibly respond during the test window.
• Safe: performance, accessibility, privacy, and business guardrails are considered.
• Comparable: control and variant differ in the intended way, not because of tracking or eligibility.
• Documented: screenshots, code version, configuration, copy, targeting, and exposure are archived.

Example: three signup variants

A SaaS team wants to reduce friction in trial signup. It creates three treatment options alongside the current control.

Variant C may produce more accounts but weaker activation. Variant D may create fewer accounts but more retained users. The primary metric determines what “better” means; otherwise the team will choose whichever number looks most flattering.

How to measure a variant

For a binary metric:

Use exposure as the denominator when exposure defines the population that received the experience. If the variant is assigned but never shown, distinguish assignment from exposure rather than silently discarding users.

One change or a bundle?

There is no universal rule that every variant must change exactly one pixel. The right scope depends on the question.

If a bundled variant wins, do not claim that every component caused the lift. Use the result to decide whether the package should ship, then run follow-up tests to isolate the valuable parts.

Naming and versioning variants

Names such as “Variant B” are easy to type and hard to remember. A useful name tells the team what changed without pretending to describe the result.

Archive the variant definition, screenshot, targeting, code commit, configuration, experiment ID, start and end dates, exposure event, and metric version. When the variant becomes the new control, preserve its original history rather than overwriting the record.

Assignment and delivery

Users should be assigned to variants randomly and persistently. A deterministic hash of an experiment salt and stable identifier is one common implementation. A feature flag can deliver the version, but the flag must emit or connect to an exposure event and outcome data.

• Assign before the treatment can influence eligibility.
• Keep user, account, or cluster assignment stable.
• Log assignment and actual exposure separately.
• Check that every variant renders correctly across key browsers and devices.
• Monitor errors, latency, accessibility, and fallback behavior.
• Do not change variant weights mid-test without documenting the new design.

Variants in A/B/n and multivariate tests

In an A/B/n test, each selected alternative is a complete variant. If A is control and B, C, and D are treatments, traffic is split across four arms and the analysis needs a multiple-comparison plan.

In a multivariate test, a variant may mean a level of one factor, while a full variation is a combination of levels. For example, two images and two CTA labels create four complete combinations. Nielsen Norman Group makes this distinction because the number of combinations grows quickly [2].

When is a variant a winner?

“Highest observed rate” is not enough. A variant is ready to ship when:

1. the primary metric meets the prespecified decision rule;
2. the effect size is meaningful enough to justify implementation;
3. the uncertainty is acceptable;
4. control and treatment passed SRM and instrumentation checks;
5. guardrails did not cross unacceptable thresholds;
6. the result is understood well enough for rollout and monitoring.

A result can also be useful when the variant loses. It may show that the mechanism was wrong, the problem was not important, the change was too weak, or the metric was too distant. Record that learning instead of deleting the experiment.

Common variant mistakes

1. Variant without a hypothesis: the test becomes a random design contest.
2. Changing too much without naming the package: the result cannot be attributed cleanly.
3. Variant name hides the change: future teams cannot reuse the learning.
4. Variant is assigned but not exposed: the analysis mixes delivery failure with treatment effect.
5. Different tracking by arm: the measurement path becomes part of the variant.
6. Adding variants mid-test: sample size and multiplicity rules change.
7. Choosing a metric after the fact: the result is vulnerable to cherry-picking.
8. Calling an observed leader a winner: ranking does not establish a meaningful effect.
9. Ignoring downstream quality: more signups can hide lower activation or retention.

Variant checklist

• Variant is tied to a clear, falsifiable hypothesis.
• Difference from control is written in plain language.
• Scope is explicit: one element, coherent cluster, or complete concept.
• Variant has a descriptive, versioned name.
• Primary, driver, guardrail, and diagnostic metrics are defined.
• Assignment unit and exposure rule are appropriate.
• Variant renders consistently across key environments.
• Tracking and denominators match the control.
• Sample size, MDE, power, duration, and multiplicity are planned.
• Rollout, rollback, and follow-up isolation tests are considered.
• Screenshot, code, configuration, and result are archived.

FAQ

What is a variant in an A/B test?

A variant is the alternative version being compared with the control. In a basic A/B test, B is usually the variant.

Is a variant the same as a treatment?

In everyday digital experimentation, almost always. Variant describes the experience; treatment emphasizes the intervention assigned to a group.

How many variants should an A/B test have?

As many as the decision requires and the traffic can support. Fewer, distinct variants are often more useful than many minor alternatives because every arm needs data and increases analysis complexity.

Can a variant contain several changes?

Yes, if the hypothesis is about the combined experience. Be precise about what the test estimates: it measures the package, not the isolated contribution of every element.

What happens when a variant wins?

Validate the analysis and guardrails, roll it out in a controlled way, monitor downstream outcomes, and use it as the new control only after preserving the experiment record.

Sources

1. Convert: Variant
2. Nielsen Norman Group: Multivariate Testing
3. Omniconvert: Variation in A/B Testing
4. Kissmetrics: Variant
5. Inspectlet: A/B Testing Guide
6. Conversion: Experiment Design
7. Wikimedia: Variant Experiments
8. VWO: A/B Testing

Related terms

• A/B Testing
• A/B Test
• A/B/n Testing
• A/A Testing
• Split Testing