What Is Incrementality Testing?
Incrementality testing is an experiment that measures how many sales your ads actually caused. It splits people (or regions) into a test group that can see your ads and a holdout group that can't, then compares how many people in each group buy.
The difference between the two groups is the lift. Everything else, the people who would have bought anyway, is excluded from the result.
That's the part normal reporting can't do. Ads Manager counts conversions that happened after someone saw or clicked an ad. Some of those people were going to buy no matter what. An incrementality test is the only clean way to separate the two.
Why Attribution Isn't Enough
Attribution answers "which ad was near the sale?" Incrementality answers "would the sale have happened without the ad?" They're different questions, and the gap between them is where budgets get wasted.
A few places it shows up:
- Retargeting looks great. It shows ads to people who were already close to buying, so it collects a lot of credit. Some of those sales would have happened anyway. We cover this in our guide to Facebook retargeting ads.
- Branded search and bottom-of-funnel ads look great for the same reason.
- Top-of-funnel creative looks weak. UGC, motion explainers and hook-led videos start demand that later ads get credit for. More on that in Meta attribution and our post on saves, shares and top-of-funnel attribution.
Meta itself is clear that the two shouldn't be mixed. Its help center says Conversion Lift results aren't meant to be compared with campaign results in Ads Manager, even for the same campaigns, because they count conversions differently.
Incrementality vs. Attribution vs. MER
| What it tells you | What it can't tell you | How often | |
|---|---|---|---|
| Platform attribution | Which ads were near each conversion | Whether the ad caused it | Daily |
| MER (blended) | Whether total revenue moves with total spend | Which channel or ad did it | Weekly |
| Incrementality test | How many conversions the ads caused | What happens outside the test window | A few times a year |
You need all three. Attribution runs the day-to-day, MER keeps you honest week to week, and incrementality tests check whether the first two are telling the truth.
How Incrementality Testing Works
Every incrementality test has the same parts:
- A randomly assigned test group. These people (or regions) are eligible to see your ads.
- A holdout group. These are similar people (or regions) who don't see the ads during the test.
- A conversion you can match to both groups, like purchases sent through the Conversions API.
- A comparison. Conversions in the test group minus the conversions you'd expect from the holdout group equals the incremental conversions.
Randomization is what makes it work. If the groups are random, the only real difference between them is the ads, so the difference in sales is caused by the ads.
Meta's Conversion Lift uses what it calls an intent-to-treat approach. The test group is everyone eligible to see your ads, including people who never scrolled to one. That sounds less precise, but it keeps the two groups comparable, which is what matters for a fair result.
The Two Main Ways to Test on Meta
Conversion Lift (people-based)
This is Meta's built-in lift test. Meta randomly holds back a group of people from your ads and compares their conversions with the people who could see them.
From Meta's help center:
- There's no extra cost for the test itself, but the campaigns in it may need to meet budget requirements.
- As a guide, the ad account needs a campaign that started in the past year with $5,000 or more in spend and at least 500 conversions (1-day click, 7-day click or 1-day view). Longer campaigns are prorated: a 180-day campaign needs $10,000 and 1,000 conversions.
- Self-serve tests have signal quality requirements, such as using the Conversions API with an Event Match Quality score above 5.
- You can create one in the Experiments tool or through a Meta account representative if you have one.
Geo testing (region-based)
Instead of holding out people, you hold out places. Ads run in some cities or regions and not in others, and you compare sales between them.
Meta's Marketing Science team built an open-source tool for this called GeoLift. It's an R package that picks test markets and measures the lift using synthetic control methods. Meta says it's for cases where people-based Conversion Lift isn't feasible, and recommends people-based tests where possible because they have more statistical power.
Geo tests are also how you test channels Meta can't hold out for you, like CTV, or a channel-wide question like "what happens if we pause Meta in these regions?"
How to Run an Incrementality Test
- Write one question. "Does our prospecting campaign drive purchases we wouldn't get anyway?" is a good question. "Are our ads working?" is not.
- Pick the method. Use Conversion Lift if you meet the requirements. Use a geo test for cross-channel questions or when you can't run a people-based test.
- Fix your conversion data first. Meta recommends the Conversions API as the data source for Conversion Lift, and says higher Event Match Quality may give better results.
- Check the power before you start. Meta suggests an estimated power of 80% or higher to give the test a good chance of a clear result. If power is low, add budget or time instead of hoping.
- Don't touch it mid-test. Big edits to budget, targeting or creative make the result hard to read.
- Read it on the test window. Meta doesn't recommend using the post-test conversion window, since the holdout isn't maintained after the test ends.
- Decide what you'll do with each outcome before you see it. Otherwise every result gets explained away.
How to Read the Results
Meta's lift reports use a handful of metrics. These are the ones that matter, with Meta's definitions:
| Metric | What it means |
|---|---|
| Incremental conversions | Conversions that wouldn't have happened without the ads, from people reached by them |
| Conversion lift % | Incremental conversions divided by the conversions you'd have had without the ads |
| Incrementality share | The share of conversions from people reached by the ads that were incremental |
| Cost per incremental conversion | Spend divided by incremental conversions |
| Incremental ROAS | Incremental sales divided by spend |
| Confidence | The probability the ads drove more than zero incremental conversions |
A simple, made-up example: the test group has 1,000 purchases, and the holdout group, scaled to the same size, has 800. That's 200 incremental purchases and 25% lift. If the test cost $20,000, the cost per incremental purchase is $100, even if Ads Manager showed a much lower cost per purchase.
Two things to watch:
- Confidence. Meta says 90% or higher is a statistically reliable lift result. Below that, treat the number as a hint, not a fact.
- The interval. Meta reports a 90% credible interval around incremental conversions and sales. A wide range means the test didn't have enough data to pin it down.
Then compare cost per incremental conversion with what a customer is worth to you. That's the real version of the question in our post on what a good ROAS is.
Using Incrementality Tests to Judge Creative
Most incrementality talk is about channels and budgets. It's also one of the best ways to settle creative arguments.
Meta's lift reports support multi-cell tests, with metrics that rank each cell by its chance of a better cost per incremental conversion or a higher incremental ROAS. That lets you compare creative strategies on what they cause, not what they get credit for. Questions worth testing:
- Does our top-of-funnel UGC drive sales, or just views? Test a campaign with it against one without it.
- Does a broader creative mix beat a tight set of winners? This is the creative diversity question, answered with real lift.
- Is retargeting adding sales, or just collecting credit?
For everyday creative decisions, use a regular A/B test and a sensible creative testing setup instead. Meta's A/B testing can compare versions on cost per result, and Meta says a 65% or higher confidence counts as a winner there. It also warns against testing informally by switching ad sets on and off, because the audiences overlap and the results aren't reliable. Save lift tests for questions big enough to change a budget.
Common Mistakes
- Testing too small. An underpowered test comes back "inconclusive," which people then read as "the ads don't work."
- Comparing lift results to Ads Manager. They count differently. Meta says not to compare them.
- Running one test and calling it final. Results hold for the conditions you tested. New creative, seasons and offers change them. Re-test the big calls a few times a year.
- Judging creative only on attributed results. The ads that look worst in the dashboard are sometimes the ones creating the demand. Look at reporting and lift together.
How We Think About Incrementality at Biddyco
We make the ads, not the measurement. But the ads are what a lift test is really testing, and a test is only as useful as the creative in it. If every ad in the account is a variation of the same retargeting offer, there isn't much to learn.
We build accounts with real creative range: UGC, motion, statics and hybrids made for each stage of the funnel, so a test can show which ones actually move sales. Our US-based senior team has done this across 100+ accounts spending $20M+ a month on Meta. Files land every 2 to 4 weeks, and we aim for one round of feedback.
See our Meta creative agency work, browse the Meta ads statistics hub, or book an intro call.
FAQ
What is incrementality testing in marketing?
Incrementality testing is a controlled experiment that measures how many conversions your marketing caused. It compares a test group that can see your ads with a holdout group that can't. The difference in conversions is the lift.
What is the difference between incrementality and attribution?
Attribution assigns credit for a conversion to ads that were near it. Incrementality measures whether the conversion would have happened without the ads. Attribution can credit ads for sales that would have happened anyway; an incrementality test can't.
How do I run an incrementality test on Meta?
Use Meta's Conversion Lift test in the Experiments tool, or through a Meta account representative. Meta's guide calls for a campaign from the past year with at least $5,000 in spend and 500 conversions, plus good signal quality such as the Conversions API. For region-based tests, Meta's open-source GeoLift package is an option.
What is a good lift percentage?
There isn't one number. What matters is whether the cost per incremental conversion is below what a customer is worth to you, and whether the result is reliable. Meta treats 90% or higher confidence as a statistically reliable lift result.
How long should an incrementality test run?
Long enough to reach the power the test needs, which depends on your budget and conversion volume. Meta suggests an estimated power of 80% or higher before you start. Don't stop early or edit the campaign mid-test.
Is Meta incremental attribution the same as an incrementality test?
No. Incremental attribution is an attribution model that uses machine learning to predict whether a conversion was caused by an ad, and optimizes delivery toward those. A Conversion Lift test measures the cause directly with a randomized holdout group.