When two models disagree, that's the finding

Every mix model is built from choices: how carryover is shaped, how saturation is modelled, how seasonality and trend are handled, what the model assumes before it sees the data. Reasonable experts make different choices.

Usually only one model gets built, so those choices stay invisible. The output arrives as a single answer, and nobody knows how much of it came from the data and how much from the choices.

Run two, on the same data

Open-source tools now make it practical to run two well-regarded modelling approaches, for example PyMC-Marketing and Google's Meridian, on exactly the same dataset. Same weeks, same channels, same controls. The only difference is the modelling approach.

Then compare what they say about each channel.

What agreement tells you

Where both engines give similar returns with overlapping ranges, the result doesn't depend much on modelling choices. The data is speaking clearly. That's a channel you can act on with more confidence.

What disagreement tells you

Where the engines disagree, the data isn't strong enough to settle the question on its own, so the answer depends on the choices each engine makes. That usually happens for a reason you can find:

• The channel barely varied, so both models are leaning on assumptions, and the assumptions differ.
• The channel moves with another one, so the models split the credit differently.
• The channel's effect is slow, and the engines model carryover differently.

Disagreement isn't a failure of either model. It's a map of where your evidence is thin.

What to do with it

1. Act on agreement. Channels where both engines agree are your firmest ground.
2. Test the disagreement. A geo test on the channel the engines disagree about settles what the models can't.
3. Feed the test back in. Use the result to calibrate both models, then see whether they converge.

Why not just average them?

Averaging two disagreeing answers produces a third number that neither model supports, and hides the disagreement that told you something. Show both, say plainly where they differ, and let that guide the next test. A single model hides its own uncertainty about itself. Two models put it on the table.