Experiment Terminal States

When an experiment reaches a conclusion, it takes one of a few paths: the winner is deployed to all traffic, the experiment ends without a rollout, or a single weak variant is disabled while the rest keeps running. This page explains each outcome and what it means for your site.

Deploying a winner

Deploying rolls the winning variant out to 100% of your traffic. The change is served through the Shuttlebase snippet already installed on your site, so no release or code change is needed on your end. Every visitor sees the winning version from that point on, including visitors who were in other variants during the experiment.

The experiment’s status changes to Deployed. Deployed changes keep running through the snippet until you either remove them or build them into your own site (see marking as implemented below).

A deploy can start in two ways:

• Shuttlebase sends you a deploy request in the dashboard, and someone on your team with approval permission accepts it.
• Auto-deploy handles it without a request, if you have enabled that setting.

Note

Deployed changes still render for visitors who decline tracking consent. Consent controls data collection, not the serving of a change that has already won. See Privacy and consent.

Ending an experiment

Ending stops an experiment without rolling anything out. All traffic returns to your unmodified site. The status changes to Ended, and the results stay available in the dashboard.

Experiments end when the variants lose to the baseline or when the data shows the test cannot produce a useful result. An ended experiment is not wasted work. Knowing that a change does not help is information, and it feeds the ideas behind future experiments.

Disabling a single variant

In an experiment with several variants, one may fall clearly behind while the others still look promising. In that case the weak variant can be disabled on its own. The experiment keeps running, and the disabled variant’s traffic is reallocated to the remaining variants.

Shuttlebase’s recommendations engine flags variants that warrant this. Team members with experiment control permission can also disable a variant manually. See Statistics for how these recommendations are made.

Marking as implemented

A deployed change is served by the snippet on every pageview. Many teams eventually build winning changes directly into their site’s own code, which makes the change permanent and independent of the snippet.

When your team hard-codes a winning change, mark the experiment as Implemented in the dashboard. This tells Shuttlebase to stop serving the change through the snippet, so it does not apply twice, and records that the win now lives in your codebase.

Exporting variant code

You can export the winning variant’s code from the experiment’s results page. The export gives your developers the exact change to build into your site, which makes hard-coding straightforward. The usual flow is export, implement in your codebase, verify, then mark the experiment as implemented.

Automation settings

Two account settings control how much of this happens automatically:

Setting	Default	What it does
Auto-end losing experiments	On	Ends experiments automatically when the data shows the variants are losing.
Auto-deploy winning experiments	Off	Deploys winners automatically instead of sending a deploy request for approval.

With the defaults, losing experiments clean themselves up while every deploy waits for your explicit approval. Teams that trust the process often turn auto-deploy on so wins reach all traffic sooner. You can change both on the Automation settings page.