How to Reduce Customer Acquisition Cost with Incrementality Testing

Your reported customer acquisition cost is almost certainly wrong. Not because your tracking is broken (though it might be), but because attribution models take credit for conversions that would have happened anyway.

When Google Ads reports a €50 CPA and Meta reports a €45 CPA for the same customer, someone is lying. Or more accurately, both platforms are claiming credit for a conversion they didn’t fully cause.

Incrementality testing is how you find out what’s actually true.

Why CAC Lies

Attribution models have a fundamental problem: they can only measure correlation, not causation.

When someone sees your Meta ad, clicks your Google ad, and then buys, every platform in that journey claims credit. But would they have bought anyway? Did any of those touches actually change the outcome?

Common attribution lies:

• Brand search cannibalization. Your branded Google Ads campaign shows amazing ROAS because it’s capturing people who were already going to buy. You’re paying for traffic you’d get organically.

• Retargeting overclaim. That retargeting campaign looks efficient because it targets people already deep in your funnel. Many would have converted without the ad.

• Multi-platform double-counting. Meta and Google both claim the same conversion. Your aggregate numbers show twice the actual result.

• Last-click bias. The final touchpoint gets all credit, regardless of what actually influenced the decision.

None of this means your marketing doesn’t work. It means you don’t actually know which parts work, by how much.

What Is Incrementality Testing?

Incrementality testing measures the true causal impact of your marketing. Instead of asking “who converted after seeing an ad,” it asks “how many additional conversions did this ad create?”

The methodology:

1. Split your potential audience into two groups
2. Show ads to one group (test), hide ads from the other (holdout)
3. Measure conversion rates in both groups
4. The difference is your incremental lift

If the test group converts at 4% and the holdout converts at 3%, your ads drove a 1% incremental conversion rate. The other 3%? They would have converted anyway.

This changes everything about how you evaluate and optimize marketing spend.

Attribution vs Incrementality

You need both. Attribution helps you optimize within a channel. Incrementality tells you whether to invest in that channel at all.

Geo Holdout Tests for Paid Media

The most practical incrementality test for most advertisers is the geo holdout test:

How it works:

1. Identify comparable geographic regions (similar demographics, similar historical performance)
2. Pause advertising in some regions (holdout) while continuing in others (test)
3. Run for 2-4 weeks minimum
4. Compare conversion rates between test and holdout regions
5. Calculate incremental lift

Example:

You run Google Ads across Germany. You pause ads in Bavaria and Saxony (holdout), continue in other states (test).

After 4 weeks:

• Test regions: 1,200 conversions on €30,000 spend
• Holdout regions: 800 conversions (scaled for population)
• Expected holdout conversions if ads worked: ~900

Incremental conversions: 1,200 - 800 = 400 (not the full 1,200 you attributed) True incremental CPA: €30,000 / 400 = €75

Your attributed CPA said €25. Your true incremental CPA is €75. That’s a 3x difference.

When to Run Incrementality Tests

Not every campaign needs incrementality testing. Prioritize:

High spend campaigns. If you’re spending €10,000+/month on a channel, it’s worth knowing if it actually works.

Retargeting campaigns. These are particularly prone to overclaiming. Test them.

Brand search. Are you paying for traffic you’d get anyway? Find out.

Declining efficiency. If ROAS is dropping, is it the market or was your baseline inflated?

Budget allocation decisions. Before shifting major spend between channels, validate your assumptions.

Reducing Wasted Spend with Incrementality Data

Once you know true incremental impact, you can make better decisions:

Kill the overclaimers. Campaigns that look great in attribution but show low incrementality are burning money. Scale them back.

Double down on real winners. Campaigns with high incremental lift deserve more budget, even if their attributed metrics look worse.

Rebalance brand vs. non-brand. If branded search is mostly cannibalistic, reduce investment and let organic capture those conversions.

Set realistic expectations. Your true CAC is probably higher than reported. Price and plan accordingly.

Practical Limitations

Incrementality testing isn’t perfect:

Requires scale. You need enough conversions in both test and holdout groups to reach statistical significance. Small accounts struggle.

Takes time. Tests need 2-4 weeks minimum. Seasonality and external factors can confuse results.

Geo tests have spillover. Someone in a holdout region might see an ad elsewhere, or someone in a test region might buy in a holdout region.

Not continuous. You can’t run holdout tests forever. They’re periodic checks, not ongoing measurement.

Complex for multi-channel. Testing one channel at a time is straightforward. Testing channel interactions is harder.

When Incrementality Testing Makes Sense

Incrementality testing is worth the effort when:

• You’re spending €15,000+/month on paid media
• You suspect retargeting or brand campaigns are overclaiming
• You’re making major budget allocation decisions
• You have enough conversion volume for statistical significance
• You can afford to pause spend in test regions temporarily

It’s probably overkill when:

• Your total spend is under €5,000/month
• You’re still optimizing basic campaign structure
• You only have one channel to test
• Conversion volume is too low for meaningful results

For accounts between these extremes, start with a single test on your highest-spend campaign.

Beyond Testing: Building an Incrementality Mindset

Even if you can’t run formal tests, you can think incrementally:

• Be skeptical of campaigns that only target people already in your funnel
• Question retargeting efficiency—would these users convert anyway?
• Watch for brand search eating into organic traffic
• Don’t trust platform ROAS at face value

Attribution tells you what happened. Incrementality tells you what you caused.

Need Help Evaluating Your Media Efficiency?

If you’re spending significant budget on Google Ads or Meta Ads and suspect your true efficiency is lower than reported, we can help design and execute incrementality tests. Get in touch to discuss whether testing makes sense for your situation.

For more on setting up the tracking foundation needed for accurate measurement, see our guide to GA4 reporting.