Quick definition: A feature flag is a runtime-controlled rule that enables, disables, or changes a product capability for a defined audience without requiring a new deployment. It supports safer delivery and experiments, but it needs ownership, exposure logging, testing, and removal discipline to avoid operational and measurement debt.
What is a feature flag?
A feature flag, also called a feature toggle, evaluates a rule at runtime and returns a product configuration. The rule may be as simple as “off for everyone” or as specific as “show the new importer to 10% of eligible Android workspaces in Germany.” Engineers can separate code deployment from customer release, turning a capability on only after the code is available and reverting it quickly if a problem appears.
Flags are used for several different purposes. A release flag exposes an incomplete or risky capability gradually. An experiment flag assigns stable variants to compare outcomes. An operational flag protects a dependency or disables an expensive path under incident conditions. A permission or entitlement rule grants a paid capability to a customer group. These uses have different lifetime, audit, and measurement needs; calling them all “a flag” should not erase those differences.
A flag is not itself an experiment. It can deliver an experiment variant, but causal interpretation requires a stable randomization unit, an eligible population, correctly logged assignment and exposure, a pre-specified outcome, and a valid analysis. Similarly, an “on” value does not prove that a user saw or could use the feature. Product analytics needs separate evidence for evaluation, delivery, and meaningful use.
Feature-flag design and governance
Give each flag a clear owner, purpose, audience, default, expected lifetime, rollback behavior, and removal date. Use a readable name that describes the capability rather than a temporary implementation detail. Store configuration centrally where possible, protect changes with access controls and audit history, and ensure the default is safe if the flag service is unavailable. A critical safety switch may require a fail-closed default; a noncritical user-interface enhancement may reasonably fail open.
| Design item | Question | Example |
|---|---|---|
| Purpose | Why does this flag exist? | Gradual release of a new import path. |
| Unit | What receives a stable decision? | Workspace ID, not a changing session ID. |
| Audience | Who is eligible for evaluation? | Supported-plan workspaces with import access. |
| Fallback | What happens on evaluation failure? | Existing import flow remains available. |
| Lifetime | When will it be removed? | Within two releases after full rollout. |
Stable assignment matters for customer experience and experiments. If a user receives “on” during one session and “off” in the next because the identifier changed, the experience can flicker and the estimate can be contaminated. Choose the unit that owns the experience: user for an individual interface, workspace for a shared workflow, account for a billing rule. Document how anonymous identities are handled and what happens when users join or leave an account.
Evaluate flags early enough to avoid visible layout shifts, but avoid making them invisible to observability. Client-side flags can create flicker or be affected by network timing; server-side flags can make rendered content stable but require propagation across services. Cache and timeout behavior, SDK versions, evaluation latency, and fallback state are part of the product contract. Test the off path as carefully as the on path.
Measurement for flags and experiments
Log at least three distinct facts: assignment, flag evaluation or delivery, and meaningful exposure. Assignment records the intended variant for the analysis unit. Evaluation records that the rule returned a value and its context. Exposure records that the person or account had a real opportunity to experience the treatment, such as a rendered module or server response. Downstream behavior records whether the feature was used and whether it delivered value.
These events answer different operational questions. A lower-than-expected exposure rate can reveal a rendering bug, ineligibility mismatch, or a user path that bypasses the surface. A low use rate after high exposure may be a discoverability or value issue. Do not define an experiment’s primary population as only those who exposed after assignment unless the design explicitly supports that estimand; in the usual intent-to-treat analysis, every randomized eligible unit remains in its assigned group.
Measure flag health with evaluation volume, variant distribution, error rate, fallback frequency, latency, and the share of evaluations missing an expected identifier. For a percentage rollout, inspect whether the allocation matches the planned split and whether major segments are balanced. A sample-ratio mismatch can signal bucketing, logging, or eligibility problems that make result interpretation unsafe.
Product and experiment scenario
A product team has built a new report-import experience with earlier permission guidance and an automatic retry option. It first deploys the code behind a release flag that is off by default. Internal workspaces and a small support cohort use the capability while the team monitors server errors, client crashes, import latency, and data integrity. The flag provides a fast rollback route if the importer creates duplicate records.
For a causal product question, the team creates a separate experiment configuration. Eligible new workspaces are randomly assigned at the workspace level to the old or new flow, and assignment is sticky for 14 days. The primary metric is the percentage of all assigned eligible workspaces that save a first report within that window. Import completion, retry use, and permission errors diagnose the pathway. Duplicate import rate, support contacts, and repeat use are guardrails.
The team does not compare only people who received an exposure event, because a treatment may change exposure itself. It checks exposure rates to validate delivery, but estimates the primary effect among assigned eligible workspaces. If the release flag is expanded during the experiment, its target audience and configuration changes are recorded; uncontrolled changes can contaminate the comparison. A documented sample-size plan and duration rule prevent a rollout decision from being driven by an early favorable fluctuation.
Limitations and operational risks
Flags add branches to the codebase. Multiple interacting flags can produce combinations never tested by engineering, especially when configuration differs across environments or regions. Old flags obscure which path is canonical, increase incident risk, and make security reviews harder. Treat removal as part of completion: assign an expiration date, alert owners, delete dead code after a decision, and test the simplified path.
Targeting can introduce fairness, privacy, and legal considerations. A rule based on geography, plan, behavior, or account attributes needs a legitimate product purpose and clear access controls. Avoid targeting on sensitive data unless it is necessary, authorized, and governed. Flags should not become an unreviewed mechanism for permanently serving materially different experiences to groups without documentation.
Flag data may still be incomplete. Ad blockers, offline clients, SDK failures, and server/client disagreements can make logged exposures diverge from actual delivery. Audit records against application logs and authoritative state. For outcomes that matter, establish an independent server-confirmed measure rather than trusting a single flag-evaluation event.
Common mistakes
- Using one flag for every purpose: distinguish release, experiment, entitlement, and emergency controls.
- Randomizing on an unstable identifier: this creates inconsistent experiences and contaminated groups.
- Logging assignment but not exposure: delivery failures then remain invisible.
- Analyzing exposed users only: treatment-selected filtering can bias an experiment.
- Leaving flags indefinitely: stale branches accumulate technical and operational debt.
- Skipping fallback tests: the disabled or service-failure path must be safe.
FAQ
Is a feature flag the same as an A/B test?
No. A flag is a delivery mechanism. An A/B test needs random assignment, a defined eligible population, measurement, and an analysis plan; a flag can implement those requirements.
When should a feature flag be removed?
Remove it as soon as the decision is complete and the stable path is verified. Record the owner and target removal date when the flag is created.
Should a flag be evaluated on the client or server?
Choose based on the delivery need, security model, latency, and observability. Server evaluation often gives stable rendering; client evaluation can support dynamic interfaces but needs flicker and failure safeguards.
What is feature-flag exposure logging?
It records that a unit had a real chance to experience the configured treatment. It is a delivery diagnostic and may support planned analyses, but it is not automatically the experiment population.
Summary
A feature flag controls runtime product behavior for a defined audience and helps separate deployment from release. Use stable units, safe fallbacks, auditable targeting, assignment and exposure logging, health monitoring, and an explicit removal plan. When a flag delivers an experiment, preserve randomization and analyze the planned eligible population rather than confusing operational delivery data with causal evidence.
Sources
- Feature toggle
- Progressive rollout
- Random assignment
- Sample ratio mismatch