Design the test before you spend the budget

Geo tests are the most honest evidence most marketing teams can get. You switch spend off (or on) in some regions, compare them with regions that carried on as normal, and measure the difference. No attribution rules, no platform self-reporting.

They're also easy to waste. Almost every disappointing test I've seen was decided before it launched: too short, too small, or set up in a way that couldn't tell an effect from noise. The good news is that all of those can be checked in advance.

Start from the decision

Before choosing a region or a date, write down two things: what you'd do differently if the test showed an effect, and how big that effect would need to be to matter.

If a 2% lift wouldn't change your budget but a 10% one would, you're designing a test to detect 10%. That one sentence decides everything that follows.

Pick regions you can match

A geo test needs a credible answer to "what would this region have done without the change?" The usual way is a synthetic control: a weighted blend of other regions that tracked your test region closely before the test.

Some regions can be matched closely. Some can't, usually because they behave unlike anywhere else. Rank candidate regions by how well they can be matched before the test, and choose from the top of the list, not the region that's convenient.

Work out how long it needs to run

Power analysis tells you the smallest effect a test can reliably detect at each length. It comes from your own data: how noisy the gap between a region and its comparison is, week to week.

If the smallest detectable effect at eight weeks is 9% and you expect 4%, an eight-week test can't tell you anything useful. Run longer, choose a bigger or quieter region, or accept that this question needs a different method.

Check it where nothing happened

This is the step most teams skip, and the one that matters most. Run exactly the same analysis on regions you didn't touch, over the same period, as if each of them had been the test region.

If those placebo regions show "effects" as large as the one you're hoping to find, your design will mistake noise for a result. If your real result stands clearly apart from the placebos, you can trust it.

Agree the rules before launch

Write down the test regions, the comparison regions, the start date, the minimum duration and the analysis method before the test starts. Then don't change them.

Then don't peek

Early results swing. A test that looks like a clear win in week three can look like nothing in week ten, and the reverse. Decisions made on early peeks are decisions made on noise.

Agree the minimum duration up front and don't read the result until it's reached. Watch delivery and data quality while it runs; leave the answer alone.

The checklist

Before launch: the decision and the effect that matters, regions ranked by fit, a power estimate for the planned length, placebo results, written rules. If any of those are missing, the test isn't ready, and the budget is safer waiting a week than running a test that can't answer the question.