Quick definition: Event tracking is the deliberate collection of records describing meaningful actions or system occurrences, together with the identifiers, timestamps, and properties needed to interpret them. In experimentation, it supplies the evidence used to determine whether assigned users saw a treatment and what outcomes followed.
Event tracking: definition and boundary
An event is a dated observation such as product_viewed, checkout_started, payment_failed, or subscription_renewed. Event tracking specifies when that observation is emitted, which producer emits it, its schema, and where it is stored. A useful event is more than a label sent from a browser: it has an unambiguous trigger, an event time, an identity or join key, a versioned set of properties, and an agreed treatment of retries and late delivery.
Tracking is not the same as page analytics, application logs, or an experimentation report. Page views can be events, but many product outcomes occur without a new page. Technical logs may describe a failure without identifying the customer journey. A report aggregates events into a metric. Event tracking is the measurement layer that makes those products possible. It also differs from a tracking plan: the plan is the contract; tracking is the implemented and monitored behavior.
Collect only events that have a decision, operational, or legal purpose. More instrumentation is not automatically better. Excess properties increase privacy risk, schema drift, cost, and opportunities for analysts to select a flattering proxy after results are known. For each event, document ownership, retention, consent requirements, and whether it is suitable for product analytics, billing reconciliation, or experimental inference.
Implementation mechanics and data contract
A robust event contract names the event in a stable verb-object form, defines its trigger, and lists required and optional fields. Common required fields are an immutable event ID, event timestamp in a declared time standard, producer timestamp, anonymous and authenticated identifiers when permitted, session or request ID, schema version, and source platform. Domain properties should be typed and bounded: order_id, currency, amount in minor units, experiment key, route, app version, and error code are usually safer than a free-form payload.
Instrument at the authoritative point for the decision. A client event can capture that an interface was displayed; a server event may be authoritative for payment acceptance. When both matter, use separate events and a documented join. Do not call a button click a purchase merely because it often precedes one. Likewise, avoid emitting “screen viewed” before the screen is actually rendered. The trigger should represent the event’s stated meaning, not a convenient callback.
Delivery is normally at-least-once, so duplicates are expected. Include an event ID generated before transmission, retain it through the pipeline, and deduplicate by that ID or a documented business key. Record both client and received timestamps to measure delay. Events can arrive out of order because of offline clients, retries, queues, and clock errors; downstream queries should not assume ingestion order equals user-action order.
Example metric contract: completed-order rate = distinct eligible assigned users with at least one valid order_completed event in the attribution window ÷ distinct eligible assigned users. The event definition, validity checks, window, and identity rule are part of the formula.
Schema validation should happen before production data is trusted. Reject or quarantine malformed records, alert on missing required fields, and preserve the rejected-record reason. Version a schema when a property changes meaning, not only when a field is added. A field named price cannot silently change from tax-exclusive display price to paid total and remain comparable.
Event tracking in A/B testing
An experiment typically needs at least assignment, exposure, and outcome evidence. Assignment says a unit was selected for a variant. Exposure says it had a meaningful opportunity to receive the variant. Outcomes describe subsequent behavior under a prespecified window. Join these records using the randomization unit, often user or account, rather than whichever identifier happens to be available in the dashboard.
Instrument assignment before treatment-dependent navigation where possible, and keep it stable for returning users. An exposure event should include experiment key, variant, treatment version, and the same identity basis used for analysis. Outcome events should not contain a guessed variant as their only link; join them to the assignment history. This prevents a UI race, cached payload, or client bug from rewriting analytical history.
Define eligibility before observing treatment effects. If a checkout experiment is assigned at checkout entry, the intention-to-treat denominator should not become only people who clicked a treatment-specific control. That post-treatment filter can create bias. For broader guidance on denominator choice, see primary and guardrail metrics.
Track diagnostic events as well as business metrics: flag evaluated, component rendered, request failed, payment authorization attempted, and consent state applied. They are not necessarily endpoints, but they locate a discrepancy between delivery and measurement. A clean top-line conversion comparison cannot compensate for a variant that loses exposure records only on one browser.
Practical QA workflow: a new checkout event
A team adds shipping_option_selected before testing a delivery-message redesign. The contract says the client emits it after a user selects an available option; required properties include option ID, currency, displayed price in minor units, checkout ID, page version, and event ID. The server separately emits order_completed after payment and includes the checkout ID.
- Trace a small set of known test orders from rendered checkout through selection and completed order. Verify one event per meaningful action and a valid checkout join.
- Compare browser payloads against the contract, including a blocked request, offline retry, refreshed page, and returned user. Confirm that retries preserve the event ID.
- Reconcile daily distinct checkout IDs, event counts, null-property rates, and client-to-server timestamp delays. Segment by platform and app version.
- Run an identical-experience A/A check before interpreting the redesign. Compare assignment-to-exposure and exposure-to-outcome paths by arm, not just final conversion.
- Set release monitors for schema errors, duplicate rate, unexpected event-volume change, and lag. Archive the query and sample trace with the experiment record.
If iOS has a lower selection-event rate but server orders reconcile, the issue may be a client instrumentation gap rather than a behavioral change. The experiment should not use the missing client event as a primary metric until the gap is fixed or the estimand is explicitly restricted and justified.
Failure modes and safeguards
- Ambiguous triggers: “purchase” fires on click in one client and on payment success in another. Define a single meaning and separate intent from completion.
- Duplicate events: retries inflate counts or revenue. Use immutable IDs and test idempotency end to end.
- Schema drift: a reused property changes units or categories. Version the contract and retain a mapping.
- Identity fragmentation: anonymous and logged-in activity do not join, producing biased denominators. Monitor join rates and state the identity rule.
- Clock and lag errors: late events fall outside a window. Use event time, record receipt time, and define a maturity delay.
- Variant-specific instrumentation: only the new UI emits an event. Keep common outcome tracking independent of treatment whenever feasible.
Important: An event’s presence is not proof that the action occurred correctly. Validate against an authoritative system when the metric informs revenue, entitlement, compliance, or a high-stakes rollout.
FAQ
What makes an event tracking plan testable?
It specifies a trigger, owner, required fields, identity rule, expected volume, duplicate behavior, and acceptance checks. Another person should be able to reproduce the validation without interpreting the name informally.
Should every event include the experiment variant?
Exposure events should include it. For outcomes, joining to a durable assignment record is usually safer than relying on a client-provided variant property, though redundant diagnostic fields can help detect mismatches.
How do we handle late events?
Store event and receipt times, choose a data-maturity cutoff, and report it. Do not silently compare a mature control window with a treatment window still receiving delayed events.
Is client-side tracking enough for conversion?
Usually not for a transactional conversion. Client signals can measure visible progress, while a server or payment system should establish authoritative completion.
Can an event schema change during an experiment?
Only with a documented version, a compatibility plan, and analysis treatment agreed before interpreting results. An undocumented semantic change can split a metric series.
Summary
Event tracking turns product behavior into auditable evidence. Define event meaning before implementation, preserve identifiers and timing through the pipeline, validate duplicates and schemas, and connect assignment, exposure, and outcomes without treatment-dependent shortcuts. These controls make experiment results interpretable rather than merely measurable.
Sources
- UK Government Service Manual: collecting and analysing data
- OpenTelemetry: signals and event data
- Microsoft Research: Online Experimentation at Microsoft