At 30,000 a month, skip the incrementality vendor

A brand spending 30,000 dollars a month on Meta sent me a vendor deck last week. The pitch was clean and the charts were good. Geo holdout tests, a fresh lift read every six weeks, a fee around four thousand a month. The buyer was ready to sign. Then he got to the part about test design. To read a clean holdout he would have to turn off ads across a third of the country.

That is the real cost, and the fee is the small half of it. The large half is the spend you agree to distort while the test runs. At 30,000 a month you do not have the volume to pay that price and get a clear answer back. You get a number. You just cannot trust it.

A noisy test is worse than no test

Haus simulated 36 million experiment driven budget decisions. In those runs, noisy measurement finished below a do nothing baseline 38 percent of the time. Precise measurement finished below the same baseline 18 percent of the time. Read that again. More than a third of the time, the underpowered test left the brand worse off than sitting still and changing nothing.

The obvious fix is to test more often. It does not work. Haus found that tripling the tests per year moved the win rate half a point, from 62.3 percent to 61.8 percent. That is the wrong direction. Noise does not average out when each reading is already wide enough to justify any decision you like.

A noisy test does not hand you a partial answer. It hands you a confident wrong one.

Haus sells incrementality testing. Publishing a simulation that undercuts cheap testing makes the finding more credible, not less. The result is also not new. Lewis and Rao analysed 25 large digital advertising field experiments covering 2.8 million dollars of ad spend. The median confidence interval on ROI spanned more than 100 percentage points. That paper ran in the Quarterly Journal of Economics in 2015, and the arithmetic has not softened since.

Those intervals rarely reach the report you are shown. Recast documented a vendor reporting an incremental CPA of 75 dollars. The underlying range ran from 25 dollars to 750 dollars. Seventy five went in the deck. The range did not. So ask any vendor for the interval on incremental CPA, never the lift percentage.

A range from 25 to 750 dollars is not a measurement. It covers scaling up, holding steady, and shutting the channel off. Whatever you believed on Monday survives it intact. That is the trap. The point estimate feels like evidence, so it wins the meeting, and real budget moves behind a number that was never there.

Meta's own setting just crossed over

Meta has a delivery option called Incremental Attribution. It tells the bidding system to chase conversions it believes the ad caused, rather than conversions it can claim credit for. Haus compared it against standard attribution across two years of geo tests. From July 2024 to June 2025 it lost, at 0.80x. From July 2025 to June 2026 it won, at 1.26x. That is 26 percent more incremental value than standard attribution across those tests.

The split matters more than the headline. Haus put DTC only brands, meaning ones that sell through their own site, at 1.38x, and omnichannel brands at 1.02x. That second number is a wash. If most of your revenue lands on your own checkout, the setting is working in your favour right now. It costs nothing. It holds out no spend. It is a toggle.

You are letting Meta grade its own homework

That is the strongest objection to all of this, and it is fair. The setting optimises toward Meta's own model of what its ads caused. Meta gets paid when that model is generous. No amount of good intent removes the conflict, and anyone who tells you otherwise is selling something too.

So keep a check on it, and run that check rarely. Meta GeoLift and Google Meridian with GeoX are open source and free. GeoX feeds experiment results into the model as priors, so one good test keeps paying off for months. The counterweight is real. IP geolocation is only 55 to 80 percent accurate at region level, and a proper geo test costs 7 to 13 weeks. That buys one or two honest reads a year, not six. Longer windows and fewer decisions are how you narrow an interval without paying anyone. Then put the fee where the variance actually sits, which is the creative. At 30,000 a month your results move more from a new hook than from a better attribution model, and creative tests hold out no spend at all.

What to do on Monday

• Write down your blended cost per acquisition for the last 28 days, before you touch anything.
• Turn on Incremental Attribution for one purchase campaign and leave every other campaign alone.
• Let it run four weeks without changing budgets, then compare blended cost per acquisition against what you wrote down.
• If a vendor already sends you lift reads, email them today and ask for the confidence interval on your last incremental CPA.
• Move the fee you were about to spend into more creative concepts this quarter.

Four weeks from now you will have one blended number in front of you and one decision to make. That is a smaller answer than a lift deck promises. It is also one you produced yourself, in an account you control, without paying to switch off a third of your ads to get it.