Quick definition: An application programming interface (API) is a documented contract that lets one software component request data or an action from another. The contract defines what may be requested, how it is authenticated, what response is returned, and how failures are represented.
What is an API?
An API is an interface for programs, not necessarily a public web service. A browser can call a backend endpoint; a mobile app can call a feature-flag service; two internal services can exchange events; and a data platform can expose a query interface. HTTP APIs commonly describe routes, methods, request fields, response schemas, status codes, authentication, rate limits, pagination, and versioning. Other interfaces may use RPC, GraphQL, streaming protocols, or message queues.
The important boundary is the contract. A database table is not automatically an API, and an API is not automatically an SDK, user interface, or analytics event. A stable interface lets independently deployed systems cooperate without exposing their internal implementation. It also creates obligations: callers must handle documented errors and providers should preserve compatibility or announce a migration path.
API mechanics
A request normally includes a destination, method or operation, structured input, identity credentials, and optional idempotency or correlation identifiers. The provider validates authorization and input, performs work, and returns a response with data or an error. Reliable clients impose bounded timeouts, retry only operations that are safe to retry, and distinguish temporary transport failure from a definitive rejection.
Idempotency is especially useful for write operations. A unique idempotency key lets the server recognize a repeated request caused by a network retry and avoid creating a second conversion, exposure event, or order. Versioning can be explicit in a URL or header, but a version label alone is not enough: additive schema changes, default behavior, and deprecations should be documented and observed.
API design also includes operational constraints. Rate limits protect a service from overload; caches can reduce latency but can make a response stale; and authorization should follow least privilege. Avoid sending sensitive attributes merely because an endpoint accepts arbitrary metadata. Validate and minimize data at the boundary, encrypt in transit, and establish retention and access rules appropriate to the data.
APIs in experimentation
Experimentation platforms often expose APIs to assign a variant, retrieve a remote configuration, log an event, or export results. APIs can also be the subject of an experiment: a ranking endpoint might serve a new model, or a pricing service might test a response rule. In both cases, the API contract is part of treatment delivery.
Define the unit and timing before launch. A user-level product decision generally needs a stable user or account assignment; request-level randomization is different and can produce inconsistent experiences across calls. Log an assignment identifier, configuration version, request context, and delivery outcome. Do not label an assignment as exposure unless the requested decision reached the downstream component or user in the way the analysis defines exposure.
Failures may be correlated with treatment. One arm could have a larger payload, a slower model, a cache miss pattern, or a fallback path. If it has higher timeout or error rates, comparing only successful responses can select a non-comparable subset. Monitor availability and latency as guardrails, retain an explicit default behavior, and decide whether the estimand is assignment effect, delivered effect, or another clearly defined quantity.
Concrete engineering and product scenario
A SaaS application calls an internal decision API when an account opens a workflow. The API uses a persistent account assignment to choose between the current workflow and a redesigned one, returning a configuration version and an experiment ID. The frontend later logs that the workflow mounted; it does not infer viewing simply from the API response. If the call times out, the backend returns the established workflow with a fallback flag.
Engineers add a correlation ID that joins request logs, response logs, and client delivery events. They use a short timeout, avoid retrying the decision after the page has chosen its fallback, and test malformed responses. Product analysts track eligible accounts, assigned accounts, successful API decisions, mounted workflows, completions, p95 latency, errors, and fallbacks by arm. A rising fallback rate in one arm pauses interpretation of its completion rate until the delivery issue is understood.
Monitoring and diagnostics
Useful API telemetry includes request volume, status-code distribution, timeouts, retries, p50/p95/p99 latency, payload size, cache-hit rate, rate-limit responses, and schema validation failures. Distributed tracing or correlation IDs can show where time was spent across services. Protect privacy in logs: redact secrets and sensitive fields, and avoid emitting full request bodies when a small diagnostic code is enough.
For experiments, segment these signals by experiment, configuration version, arm, route, client version, and relevant geography. Check that traffic allocation resembles the planned split and investigate an unexpected sample ratio mismatch. Alerts should favor sustained deviations, since a single transient network issue need not be a product regression. Release dashboards should make it possible to compare baseline behavior before and after a configuration change.
Trade-offs and limitations
APIs support separation of concerns, reuse, and consistent enforcement of business logic. Their costs include network latency, availability dependencies, security exposure, schema coordination, and operational complexity. A remote call on a critical render path may be unsuitable even when the interface itself is well designed. Local evaluation may reduce latency, while central evaluation can simplify control; neither is inherently correct.
An API contract cannot guarantee that a downstream UI displayed the returned data, that a client was online, or that a human understood a treatment. These are separate delivery and measurement questions. Likewise, an API experiment may reveal an effect for calls that reached a service without establishing a durable effect on customers or accounts.
Common failures
- Assignment treated as exposure: a returned decision can be abandoned before it is rendered or acted upon.
- Unsafe retries: repeating writes without idempotency can duplicate events or business actions.
- Breaking schema changes: changing a field’s meaning or default can silently affect older clients.
- No timeout or fallback: an optional decision becomes a dependency that stalls the primary journey.
- Arm-specific performance: a slower treatment can change who receives it and distort outcome analysis.
- Excessive data collection: convenient request payloads can create unnecessary privacy and security risk.
FAQ
Is an API always an HTTP endpoint?
No. HTTP is common, but APIs also include library interfaces, RPC services, event streams, and database-access abstractions.
What is the difference between an API and an SDK?
An API is the contract. An SDK is a packaged set of client tools that may make calling that API easier.
Should an experimentation decision API be called on every request?
Only when the decision is genuinely request-level. Persist a user or account assignment when consistency across interactions matters.
Which API metrics are experiment guardrails?
Latency, error rate, timeouts, fallback rate, rate-limit responses, and delivery success are common guardrails, chosen according to the product’s risk.
Summary
An API is a software contract with behavior, reliability, security, and compatibility implications. In experimentation, model assignment, response delivery, retries, fallbacks, and exposure separately. Monitor operational outcomes by arm so a statistical conclusion is not mistaken for a result produced by unequal technical delivery.