Design a creative experiment that can tell you what changed

Do not test a tool-shaped bundle

The popular test has two oversized arms: one ad made with an AI workflow and another made conventionally. They often differ in proposition, art direction, length, offer, targeting, budget and landing page. If one wins, the team has learned that one bundle outperformed another on that flight. It has not learned whether synthetic production was the reason.

Break the question into two passes. First test the message: for the same audience and offer, does ‘save time’ beat ‘make room for better work’? Then test the production method only where it matters: can a generated visual treatment express the winning message as clearly, accurately and accessibly as a commissioned route? This makes the method a production decision with its own criteria, rather than a vague ideology contest.

Write a one-variable worksheet

A usable worksheet fits on one screen and can be read by media, production and the client before anything ships. Google’s experiment guidance similarly emphasizes a clear hypothesis, a chosen success metric and comparable campaign arms. Its help materials caution against multiple simultaneous experiments because they can interfere with one another. That is not a demand for laboratory purity; it is a reason to reduce preventable ambiguity.

Decide before launch what result would change a decision. ‘We will make more variants if engagement rises’ is too loose. ‘If Version B produces enough qualified demo starts at no worse complaint rate, we will commission two adaptations’ is an operating decision. It grants a result a consequence.

• Decision: the single choice this test will inform.
• Hypothesis: Version B’s specific change and the directional outcome expected.
• Held steady: audience, offer, spend split, placement, destination, flight and measurement window.
• Primary measure: one business or communication outcome; secondary measures include completion quality, complaints and accessibility failures.
• Stop and scale rule: the threshold, owner and next action before data is viewed.

Source context: Google Ads Help — About the Experiments page · Google Ads Help — Experiments FAQs

Give production method its own scorecard

Some questions do not belong in media performance alone. Compare the routes for briefing time, number of review loops, factual corrections, rights or consent checks, version control, accessibility preparation, talent involvement and usable final assets. A quick first frame can hide a slow approval path. A more expensive shoot can supply durable stills, cutdowns and a sound library. These are project facts, not proxies for audience preference.

Log them separately from the audience experiment. If the same message performs comparably but one route produces ten approved local versions with fewer late corrections, that is an operational finding. It does not prove the technology is more persuasive. Keeping the scorecards apart lets both conclusions be true or false on their own terms.

Run a small honest test before a grand comparison

For a low-volume launch, a test may not have enough traffic to decide a hard conversion question. Do not manufacture certainty with tiny samples. Use the pilot to find broken tracking, confusing interaction steps, unanticipated production effort and the range of audience reactions. Mark it as directional, preserve the learning, and decide whether the next flight can support a controlled test.

Our recommended release packet has the creative files, a change log, the worksheet, a preflight screenshot of settings and a one-page readout. The readout must include what the test did not isolate. That sentence is how a future team avoids turning a narrow win into folklore.