Interrupted Time Series

What is interrupted time series?

An interrupted time series analyzes a sequence of measurements collected at regular intervals before and after a policy, launch, outage, campaign, or other intervention. Instead of comparing one “before” average with one “after” average, it models the existing level and trajectory, then estimates whether the intervention is associated with an immediate step change, a change in slope, or both. For example, weekly support contacts may drop immediately after a self-service feature, while retention may change only gradually.

ITS is often used when individual randomization is impossible or inappropriate: a company changes a site-wide policy, a government introduces a regulation, or an operations team deploys an infrastructure improvement for all users. It is stronger than a simple before-and-after comparison because it uses the pre-intervention pattern, but it is not a randomized A/B test. Its causal claim depends on whether alternative explanations for the timing can be ruled out or made implausible.

The “interruption” must be a meaningful, well-documented change. A gradual or uneven rollout can still be studied, but the intervention variable should reflect actual treatment intensity, not an arbitrary launch announcement.

Methodology and model

A basic segmented regression represents time, an indicator for the post-intervention period, and a post-intervention time trend. The pre-period intercept and slope describe the baseline path. The post indicator estimates an immediate level change; the interaction between post status and time estimates a slope change. For an outcome Y at time t, the model might include β₀ + β₁time + β₂post + β₃time-after-intervention.

Interpret estimates in the outcome’s natural decision scale. For weekly conversion, report percentage-point level and trend changes; for count outcomes, consider population exposure or an offset; for revenue, specify whether values are nominal, inflation-adjusted, or per active user. Plot raw data, fitted pre-trend, fitted post-trend, uncertainty, intervention date, and known shocks. A model without a transparent plot is hard to audit.

Time-series residuals are often autocorrelated: this week resembles last week. Ignoring that dependence makes intervals too narrow. Use an appropriate autocorrelation structure, robust methods, or a model selected from diagnostics. Also account for seasonality, recurring calendar patterns, changing denominators, and enough pre- and post-intervention observations. There is no universal number of time points; a few monthly values rarely establish a credible trend.

Assumptions and threats to validity

The central assumption is that, absent the intervention, the modeled pre-intervention trajectory would have continued in a sufficiently similar way. This is not directly observable. A competing policy, pricing change, product release, holiday, acquisition campaign, tracking migration, economic shock, or regression to the mean occurring at the same time can produce the same apparent interruption.

Strengthen the design with a comparison series that was not exposed to the intervention but shares broad trends. A controlled interrupted time series or difference-in-differences design can reduce concern about common shocks, though it introduces the need for a credible comparison and parallel-trend reasoning. Negative-control outcomes and placebo interruption dates can also reveal whether the chosen model finds “effects” where no intervention occurred.

Define the analysis window and intervention date before reviewing post data. Do not shift the date or exclude inconvenient weeks until the curve looks compelling. Document data revisions, outcome maturity, denominator changes, and concurrent events. ITS is a design plus a transparent argument, not a regression command that creates causality by itself.

A/B testing application

Use an ordinary randomized A/B test when it can answer the decision: it has a concurrent control and fewer assumptions. ITS is appropriate when the change must occur for everyone, such as a payment-provider migration, legal consent flow, global reliability fix, or organization-wide pricing policy. It can quantify the pattern around the launch and identify whether any effect appears immediate or gradual.

For product work, combine ITS with the evidence that is available. A phased rollout can create controlled geographic or account-level variation. Stable holdout groups may enable a stronger comparison. When none is possible, describe the result as an ITS estimate with its assumptions, not as an A/B-test lift. Definitions for before-and-after analysis are useful context, but ITS offers a richer trend-based baseline.

Worked scenario: a site-wide performance release

A marketplace deploys a new image-delivery system to all visitors after an outage risk prevents a prolonged split test. The team collects daily checkout completion per eligible session for 26 weeks before and 12 weeks after the release. The primary question is whether completion improves after accounting for day-of-week and a gradual pre-existing trend. They also collect a comparison outcome, account-profile completion, which should not depend materially on image delivery.

The segmented model shows no immediate jump in checkout completion but a positive post-release slope change. After eight weeks, observed completion is 0.6 percentage points above the continuation of the pre-release trend. Profile completion shows no analogous break. The plot also identifies a holiday spike; the model includes a predeclared holiday indicator. The team reports the effect as consistent with a performance-related improvement, but acknowledges that a simultaneous search-engine algorithm change cannot be fully excluded.

The conclusion is useful for monitoring and investment prioritization, yet it is weaker than a randomized estimate. The team preserves a small feature-flag holdout for future performance changes where feasible.

Practical workflow

1. Define the interruption, actual implementation date, outcome, denominator, and causal question.
2. Gather enough regular pre- and post-period observations with stable measurement.
3. List anticipated seasonality, known shocks, concurrent changes, and plausible comparison or negative-control series.
4. Predefine segmented model terms, autocorrelation handling, analysis window, and sensitivity checks.
5. Plot raw and modeled data before interpreting coefficients; inspect residuals and data quality.
6. Estimate level and slope changes, compare with controls or placebo dates, and disclose competing explanations.
7. Use the result for the stated decision while preserving randomized controls for future changes where possible.

Interpreting an ITS estimate

An estimated level change describes an association at the interruption conditional on the model; a slope change describes a change in trajectory. Neither says every user’s outcome changed nor that the intervention was the only event occurring. A narrow interval can reflect many serial observations but still understate uncertainty from an incorrect counterfactual or unmeasured concurrent shock.

Focus on a decision-relevant horizon. A positive slope difference can accumulate into a useful long-run impact, but extrapolating far beyond observed post data is speculative. Report absolute effects at selected dates, counterfactual assumptions, data coverage, and the role of any comparison series.

Limitations and common mistakes

• Simple pre/post comparison: it confuses a baseline trend with an intervention effect.
• Too few points: a handful of observations cannot establish trends or autocorrelation reliably.
• Ignored seasonality: calendar patterns can look like a launch effect.
• Concurrent changes: another release or campaign at the interruption destroys clean attribution.
• Autocorrelation neglect: independent-observation standard errors are too optimistic.
• Date shopping: choosing the intervention date after inspecting outcomes inflates false findings.

Frequently asked questions

Is ITS a randomized experiment?

No. It is a quasi-experimental design. Its credibility relies on a defensible counterfactual trend and evidence against coincident explanations.

Do we need a control series?

Not always, but a credible unexposed comparison substantially strengthens the design against common time shocks.

Can ITS analyze daily data?

Yes, if daily measurement is stable and the analysis models weekly patterns, autocorrelation, and outcome maturity appropriately.

What if rollout was gradual?

Model actual treatment intensity or use a design suited to staggered adoption. A single binary interruption date may misrepresent exposure.

Summary

Interrupted time series estimates whether an outcome’s level or trend changes around a defined intervention. It improves on a simple before-and-after comparison by using a pre-intervention trajectory, but causal interpretation still depends on strong assumptions about concurrent events and time patterns. Transparent plots, autocorrelation-aware models, controls, and prespecified sensitivity checks make ITS more credible.

Sources

• Bernal, Cummins and Gasparrini, “Interrupted Time Series Regression for the Evaluation of Public Health Interventions”
• Ramsay et al., “Interrupted Time Series Designs in Health Technology Assessment”
• Turner et al., “The Use of Segmented Regression in Interrupted Time Series Studies”

Related terms

• Multivariate Testing
• Multivariate A/B Test
• Factorial Design
• Full Factorial Design
• Fractional Factorial Design