Quick definition: Identity resolution is the controlled process of linking records that represent the same person, account, device, household, or other analysis unit. It makes cross-session measurement possible, but an incorrect link can change both an experiment’s denominator and its outcome.
What is identity resolution?
Identity resolution connects identifiers collected at different times or in different systems into an explicitly defined entity. A browser cookie can be linked to an authenticated account, a mobile installation to a user ID, or several user IDs to a billing account. The correct entity depends on the question. “Unique device,” “logged-in customer,” and “paid workspace” are not interchangeable units, even when records can be joined between them.
The boundary is therefore analytical as well as technical. Identity resolution does not prove that two devices belong to the same individual, that an anonymous user was exposed before login, or that all events from an account should count once. A deterministic link based on a verified login is different from a probabilistic link inferred from shared attributes. Teams should document confidence, scope, effective time, and whether a link may be revised. Use the smallest reliable identity scope that answers the metric question.
Data mechanics and identity graphs
Most systems begin with source identifiers: browser ID, app installation ID, session ID, account ID, subscription ID, or organization ID. An identity service stores edges such as “anonymous ID became authenticated as account A at this time.” A reporting model then selects a canonical key and applies a rule for merging or separating records. That rule must be reproducible. Save source system, link type, observed time, processing time, and version; otherwise a historical result can change when the identity graph is rebuilt.
Deterministic links are created from explicit evidence, such as authentication or a verified account action. They generally offer clearer auditability but remain vulnerable to shared devices, account switching, logout mistakes, and implementation bugs. Probabilistic links may increase coverage using signals such as device characteristics or matching behavior, but they introduce model error and privacy concerns. They should not silently alter experiment analysis. If used, evaluate their false-match and missed-match rates, preserve the confidence rule, and report the population to which the conclusion applies.
Late linkage is a common source of confusion. An anonymous visitor may see a treatment, convert after logging in, and later appear under an account key. Decide in advance whether the outcome belongs to the original anonymous assignment, the later account, or neither. A many-to-one merge can reduce denominators and aggregate several exposures; a one-to-many split can duplicate outcomes. Reconcile distinct counts before and after every join, and never allow a generic `COALESCE` rule to hide a changed unit of analysis.
Experiment impact
Stable randomization depends on a stable assignment key. If an experiment buckets anonymous visitors but analysis switches to account IDs after login, one customer may receive multiple variants across devices or several visitors may collapse into one row. This can dilute effects, contaminate treatment arms, or make variance estimates too optimistic. The allocation rule, exposure key, metric key, and aggregation rule should be compatible or the conversion path between them must be explicitly modeled.
Identity resolution affects attribution windows too. Suppose a user receives a mobile notification under an app ID and purchases on a desktop after authenticating. Joining the purchase may be appropriate for a customer-level hypothesis, but it should not cause unrelated account activity to enter the numerator. Define the exposure-to-outcome link before launch, including maximum time, accepted identity transitions, and how account sharing is handled. A confident-looking metric is not valid merely because its join rate is high.
Review identity diagnostics by variant: anonymous-to-known conversion, link rate, merge rate, unmatched outcomes, duplicate assignments, and cross-device exposure rate. Differences can be product effects, especially when treatment encourages login, but they also expose measurement bias. Preserve a pre-treatment assigned analysis as a primary check where appropriate. For broader guidance on defining the experimental population, see how to write an A/B-test hypothesis.
Practical QA scenario
A streaming service tests an onboarding prompt in its mobile app. Assignment occurs at installation ID because visitors can browse before creating an account. The business metric is first paid subscription per assigned installation within fourteen days. During QA, analysts join subscriptions through account ID and find the treatment appears to lift conversion sharply. Investigation shows family members install the app on several devices but complete a single subscription under one shared account. A naive join attributes the same subscription to multiple treatment installations.
The team keeps installation as the primary denominator, assigns each subscription to the earliest eligible installation under a documented fourteen-day rule, and reports a separate account-level sensitivity analysis. It tests account switching, reinstalls, app upgrades, simultaneous device use, and a login occurring after exposure. It also checks whether the attribution rule produces different duplicate rates by variant. The decision is based on a versioned query whose identity rule can be reproduced after the graph changes.
- State the randomization unit, reporting unit, and allowed identity transitions.
- Test known identifiers through anonymous, login, logout, account-switch, and multi-device paths.
- Measure unmatched, many-to-one, and one-to-many joins by variant.
- Reconcile raw assignments and outcomes before and after identity mapping.
- Version the mapping rule and retain it with the experiment analysis.
Limitations and failure modes
No identity graph is complete. Users refuse consent, clear storage, use private browsing, share devices, create several accounts, or transact through channels with no stable join key. A low match rate can limit generalizability; a high match rate can conceal systematic exclusion of less engaged or privacy-conscious users. Do not “fix” uncertainty by attributing every unmatched outcome to the closest known profile.
- Key switching: randomization and analysis use different units without a predeclared bridge.
- Retroactive contamination: later login information rewrites historical exposure inconsistently.
- Duplicate outcomes: one purchase is credited to several linked devices.
- Overconfident inference: probabilistic matches are treated as ground truth.
- Privacy overreach: identity data is retained or joined beyond its documented purpose.
Important: Identity resolution can improve coverage but cannot create valid evidence where the identity relationship is unknown. Quantify uncertainty and restrict the claim to the population that the rule actually represents.
FAQ
Is identity resolution the same as a customer data platform?
No. A customer data platform may store and activate identity-linked data; identity resolution is the linking process and its governing rules.
Which ID should an A/B test use?
Use the most stable identifier available before treatment that matches the intervention and minimizes a person or unit receiving more than one variant.
Can anonymous and logged-in users be analyzed together?
Yes, if the join and attribution rules are defined before analysis. Report coverage and sensitivity checks because the observed populations can differ.
Why retain event-time identity information?
It allows a reviewer to reconstruct what was known when an event occurred rather than applying a later identity graph to past behavior without notice.
Summary
Identity resolution links fragmented evidence into an analysis unit, but the link itself is a measurement decision. Define the unit, preserve link history, validate merges and unmatched records by variant, and avoid extending experiment conclusions beyond the identity coverage you can defend.
Sources
- NIST Privacy Framework
- W3C Decentralized Identifiers
- Microsoft Research: Online Experimentation at Microsoft