If your spend never changes, no model can measure it

Here's a problem that no amount of modelling skill can fix. If a channel's spend was the same every week for two years, the data contains no information about what that channel does.

How models learn

A mix model works out what each channel contributes by watching how sales move when that channel's spend moves, after accounting for everything else. Weeks with more spend and weeks with less are the evidence.

If spend never moved, there's nothing to compare. The model can't tell whether the channel drove a large share of sales, a small share, or none, because every week looks the same.

What models do anyway

A model will still produce a number for that channel. It has to. But the number comes almost entirely from its starting assumptions (its priors), not from your data. A good model will say so, and give the channel a very wide range. A careless one will report it as confidently as any other.

This is common with always-on channels: a fixed monthly retainer, a sponsorship, display running at a steady level, a search budget capped at the same daily amount.

The fix is planned variation

You don't need to switch channels off. You need spend to move, on purpose, in a way the model can learn from.

• Vary weekly levels. Alternate higher and lower periods rather than spreading budget evenly. The total can stay the same.
• Vary by region, if you have regional data. Different levels in different places create variation at the same time.
• Run a test. A geo holdout is the most direct version: off in some regions, on in others, for a set period.
• Write it down. Record when and why spend changed, so the variation isn't confused with something else that happened that week.

Check before you model

Before commissioning a mix model, look at each channel's weekly spend for the last two years. If a line is flat, that channel's result will be an assumption, however good the model. Better to know that now, and plan the variation that fixes it, than to discover it in the readout.

A model can't learn what happens when spend changes if spend never changes.