Incrementality: Proving the Real Value of Your UA and Retargeting Channels

Clément Favier, COO at Adikteev, explores incrementality and why it is essential for measuring true advertising impact. He outlines the key principles and best practices required to design incrementality tests that deliver clear, reliable results.

Watch his full talk from UA Society London on Unlocking Incrementality here:

As a COO, my role isn’t just to oversee operations. More broadly, it’s to ensure every investment we make drives measurable business impact. That means answering a question every executive asks: “Are our advertising dollars actually making a difference?”

This is where incrementality testing becomes indispensable. It’s a decision-making framework that aligns campaign execution with company-level priorities. This means from revenue growth through to retention. Ultimately, the answers incrementality testing improves shareholder confidence.

The Executive Question: Are Your Campaigns Working?

Incrementality testing, sometimes called uplift testing, answers a deceptively simple but crucial question:

What impact did my company’s ads actually have on user behavior, compared to what would have happened without them?

By comparing a test group (shown ads) with a control group (not shown ads), we can isolate the true incremental value of campaigns.

This is particularly important in app marketing, where truly strategic decisions can be nuanced. Marketers often ask:

• Should we retarget active users, or risk cannibalizing conversions that would have happened organically?
• Are dormant users worth re-engaging, or is the cost outweighing the benefit?

Incrementality testing allows marketers to present executives with the real answer to these questions using data, not guesswork. It’s the difference between spending to look good on dashboards and spending to truly grow your business. 

The Real Impact of Incrementality on Business Decisions

Incrementality delivers insights that are directly relevant to executive priorities. For example, one gaming app saw a 40% increase in purchases by retargeting active users, countering fears of cannibalization. Or, we can look at the mobile space more widely. Across industries, retargeted inactive users had up to 54% higher retention after 30 days compared to users not shown ads.

These outcomes demonstrate exactly what stakeholders care about: revenue lift, retention, and ROI. Incrementality helps answer the executive question: “Should we be spending this money?”

How to Generate Trustworthy Incrementality Insights

To provide meaningful guidance to leadership, incrementality tests require careful planning. To successfully set up incrementality tests, you must:

1. Define audience segments: Focus on homogenous groups like “active buyers” or “lapsed payers” to ensure results aren’t skewed.
2. Split test and control groups: The test group sees ads; the control group does not.
3. Filter out noise: Remove users who weren’t realistically eligible to see ads.
4. Run the campaign and measure uplift: Compare key metrics (purchases, engagement, retention) between test and control groups.
5. Validate results with statistical significance: Only report results that are reliable and unbiased.

This structured approach ensures you're basing decisions on data they can trust, rather than noisy or misleading metrics.

Avoid the “Big Number Trap”

It’s easy to be impressed by headline numbers. But without statistical rigor, they can be misleading. Executives want the additional context to understand:

• Was the sample size large enough?
• Were segments truly comparable?
• Did inactive users skew results?

True incrementality isn’t about vanity metrics. It’s about precision and actionable insight.

Bridging Rigor and Speed with Adjusted ROAS

While incrementality testing provides the most rigorous answers, it isn’t always practical to run for every campaign or decision. This is where Adjusted ROAS plays a complementary role. It applies the same incrementality mindset to regular measurement by modeling expected organic behavior and estimating which results were likely driven by advertising. Rather than replacing controlled tests, Adjusted ROAS helps teams avoid being misled by headline numbers between experiments, offering a faster, more scalable way to sanity-check performance and guide optimization with impact in mind.

Why You Should Prioritize Incrementality

Incrementality testing is key for proving that your marketing efforts are worth the company’s investment. Mobile marketers can use it to:

• Demonstrate real ROI: By comparing test and control groups, you can show exactly which campaigns, creatives, or user segments generate revenue that wouldn’t have happened organically.
• Make data-driven budget decisions: Use incrementality insights to scale campaigns that deliver true impact and stop spending on activities that don’t move the needle.
• Protect against wasted spend: Avoid targeting users who would have converted anyway, ensuring every dollar is contributing to measurable growth.
• Build credibility with leadership: Showing incremental lift in engagement, retention, or purchases gives executives confidence that marketing is not just active, but strategically effective.

In short, incrementality testing allows mobile marketers to prove the value of their work, even to the c-suite. This invaluable data turns every campaign into evidence that marketing is a smart, measurable investment for the company.