Platforms grade their own homework

Add up the conversions every ad platform reports, and you'll often get more than the business actually made. Each platform counts the customers it touched, and many customers touched several. That overlap is a known problem.

The bigger problem is quieter. Even without overlap, a platform reporting the conversions it was present for is not the same as a platform causing them.

What "reported" means

When a platform says a campaign drove 1,000 conversions, it usually means 1,000 people saw or clicked an ad within some window before converting. It doesn't mean those people wouldn't have converted without the ad.

For some campaigns the two numbers are close. For others they're far apart. The gap is largest wherever people were already on their way to buying:

• Brand search. Someone searching your name has usually decided. The paid ad at the top intercepts them; it rarely creates them.
• Retargeting. Showing ads to people who already visited your site targets the most likely buyers by design.
• Existing customers. Campaigns aimed at people who buy regularly will record many purchases that would have happened anyway.

None of this means those campaigns are worthless. It means their reported numbers can't tell you what they're worth.

Why platforms can't fix it themselves

To know what a campaign caused, you need to know what would have happened without it. A platform only sees the world where its ads ran. It has no view of the customers who'd have bought anyway, and it has every commercial reason to count generously.

That's not a criticism of any one platform. It's the structure. Every platform is marking its own exam, and nobody would accept that anywhere else in the business.

The test that settles it

The way to see the real effect is to create the world where the ads didn't run, in part of your market, for a while. A geo holdout does exactly that: pause or reduce the campaign in some regions, keep it running elsewhere, and compare.

The difference between the regions is the campaign's incremental effect: the conversions it caused. Compare that with what the platform reported for the same period and you have a ratio you can use when reading its reports from then on.

Some practical points:

• Choose regions you can match well, and check the test can detect the effect you expect before you start.
• Run it long enough to cover the time customers take to decide.
• Expect the answer to vary by campaign type. Prospecting often holds up better than retargeting or brand search.

What to do with the result

Use the incremental number, not the reported one, when deciding budgets. Keep the platform's reporting for day-to-day optimisation inside the platform, where it's useful. And feed the test result into your mix model, so the model's view of that channel is anchored to evidence rather than to self-reporting.

The point isn't to distrust platforms. It's to stop letting anyone, including us, grade their own homework when real money is at stake.