Platform reporting is wrong in both directions

It is Monday morning. Ads Manager is open on one screen and Shopify on the other. Meta claims more purchases than the store recorded. Someone on the call says the platform is lying again. Everyone nods, the number gets discounted by feel, and the meeting moves on. That reflex runs most of the accounts I look at. It came from real research, and the research said something much narrower than the folklore it turned into.

The studies did not find that every platform number is inflated. They found that one method of measurement is inflated. That method is now the one platforms sell you inside the interface, labelled as lift.

What the 663 experiment study actually found

Gordon, Moakler and Zettelmeyer ran 663 large randomised experiments on Facebook. Randomised here means a share of users was held back from the ads at random. That holdout gives a clean read on lift, meaning the sales the ads actually caused. True lower funnel lift averaged 5 percent. The authors then asked what the common modelled methods would have reported on the same people. Double machine learning returned 24 percent. Propensity score matching returned 64 percent. Those are roughly 5 times and 13 times the truth. The paper is Close Enough? A Large-Scale Exploration of Non-Experimental Approaches to Advertising Measurement, arXiv 2022, later published in Marketing Science.

Read what broke. It was the act of matching exposed users to similar looking unexposed users and calling the gap lift. Nothing in those 663 experiments says a 7 day click report is padded. The study measured what happens when you model your way to an answer instead of holding people out.

Modelled matching was what lied. The click window was never the thing on trial.

On a strict click window Meta undercounts

Haus analysed 640 Meta incrementality experiments run since the start of 2024. Incrementality means the sales that would not have happened without the ads. For every 100 dollars of platform attributed DTC revenue on a 7 day click window, Meta generated 115 dollars in incremental revenue. About 32 percent of Meta's impact landed in retail, marketplace and store sales. Source: Haus, The Meta Report, 28 July 2025.

Sit with that for a second. If you buy on 7 day click, then shave the number again because platforms lie, you are cutting below the floor. You turn off campaigns that were paying. The retail and marketplace share explains most of it. A buyer staring at a DTC dashboard cannot see roughly a third of the outcome, so the ads look worse than they are.

Video shows the same error running backwards

Haus ran 190 YouTube incrementality tests across 74 brands. YouTube drove 3.4 times more incremental lift against DTC sales than Google Ads reported. Video Reach campaigns beat platform attribution by 10 times on average. For some advertisers the gap was over 30 times. Source: Haus, 6 March 2025.

So a video buy can be worth many times its reported number. Meanwhile a view based window on another platform can hand you credit for buyers who were already coming. Same kind of channel, opposite errors, and both sit in the same account. The variable is not platform honesty. It is the counting rule you happen to be looking through.

The method is the problem, not the platform

Recast catalogues the three methods platforms use to produce always on incrementality without running an experiment. One of them matches exposed users to similar looking unexposed users. That is the same family of method the 663 experiment study found overstated lower funnel lift by about 13 times. Platforms now print that output in the interface and call it lift. Source: Recast, The Problem with Modeled Conversions, 15 February 2023. So the number most likely to be badly wrong is not the plain click report. It is the one the platform offers to reassure you about the click report.

The strongest objection to all this

Haus sells incrementality testing. A firm that sells holdout studies benefits when platform reporting looks broken. That is a fair reason to hold those two results loosely, and I do. The weight of the argument sits on the academic study, which was not selling software, and which cuts at the modelled shortcut rather than at any platform. You also do not have to take a side on faith. A holdout is the one piece of measurement you can run yourself, in your own account, with no vendor involved.

What to do on Monday

• Set your attribution setting to 7 day click with no view window, and write that number down as your floor rather than your answer.
• List every campaign still reporting on a view window, and note what share of your reported sales sits inside it.
• Pull the same date range from Shopify and compare order counts, not revenue, so currency and discounts stop muddying the read.
• Pick one campaign and one region, turn it off there for two weeks, and read total store orders against a matched region.

The gap you find is information, not an insult. Write down which direction it runs before you move a single budget.