Variants and QA

A variant is a real change to your site: new design, new copy, or new behavior, implemented as JavaScript that runs on your live pages. Variants are not mockups or suggestions. Shuttlebase writes working code, and that code goes through several layers of checking before and after it reaches your visitors.

Staying on brand

At kickoff, Shuttlebase’s agents crawl your site and learn your brand: tone of voice, design language, and policies. You can upload brand guidebooks, such as color, font, and voice documents, to ground the output further. If a variant ever looks off-brand, flag it during review and the engine learns from your feedback. Combined with the approval workflow, nothing reaches your site without a person on your team seeing it first.

The QA gate

Before any experiment reaches you for review, each variant passes an automated QA gate.

1. AI QA. An automated QA agent checks the variant.
2. Internal sanity check. Shuttlebase’s pipeline verifies the experiment holds together as a whole.

A failure at either stage sends the variant back for fixes, and the fixed variant starts the gate again from the top. You only ever see candidates that passed the gate. When a candidate does reach you, you review and approve it as described in Reviewing and Approving.

Live error monitoring

QA happens before launch, but sites change. A theme update or a new page layout can break a variant that worked yesterday. Shuttlebase watches every variant running in production and catches issues such as:

• Broken selectors. The page element a variant modifies no longer exists, so the change cannot apply.
• Failed insertions. Content a variant adds to the page does not appear where it should.

Monitoring classifies the root cause of each error, which tells Shuttlebase whether the fault lies in the variant code or in a change on your site. The snippet also reports when each variant executes successfully, so Shuttlebase can confirm changes are applying, not just watch for failures. See Events for the diagnostic events involved.

When a variant needs a fix

Fixed variants always go back through QA. It does not matter where the problem surfaced:

• A QA stage failed during the initial build.
• You requested changes during review.
• Live monitoring caught an error in production.

In every case, Shuttlebase fixes the variant and it passes the full QA gate again before it returns to your site or to your review queue. Changes you requested come back as a fresh approval request, shown as Pending Modifications while the work is in progress. See Experiment Lifecycle.