Today’s user acquisition and retargeting landscape is evolving fast—but the principles remain the same: better signals lead to better bidding. That’s exactly why churn-based bidding is becoming a go-to strategy for performance marketers.
At Adikteev, we asked a simple but critical question:
If churn scores are going to power our bidding—how do we prove they’re actually reliable?
This case study walks through how we tested that, what we learned, and what it means for clients looking to drive more accurate prediction and smarter spend.
What is a Churn Score?
Churn scores estimate how likely a user is to stop engaging with your app. They’re calculated using real in-app behavior like drop-offs in session frequency, reduced engagement, or time since last open.
Adikteev uses churn scores as part of our bidding logic to prioritize users who are at risk of leaving, but still valuable to win back and convertible. These scores help allocate spend more strategically.
But strategy isn’t enough—we wanted to test whether churn scores actually improve model performance.
Do they make predictions more accurate? Do they help us target better? Do they drive real results?
That’s what this study set out to prove. We focused on three critical metrics:
- Calibration Error: Do actual outcomes match the original predictions?
- Discrimination: Can the model better separate converters from non-converters?
- Budget Efficiency: Does it spend more of the budget on users who are likely to convert?
How is Conversion Predicted?
Before diving into the results of our case study, it’s worth understanding how conversion models work.
These models typically rely on:
- Bid context (time, app, placement)
- Demographics (age, gender, location)
- Behavioural signals (this is where churn scores come in)
Churn scores are calculated based on how users behave inside the app. Think: engagement drop-offs, frequency declines, or behavioral falloff.
Adikteev’s Case Study: Can churn scores reliably power bidding?
Our hypothesis: if churn scores are behavior-based, they should help models perform better.
Calibration Error–Does It Distort Accuracy?
Calibration measures how close predicted conversion rates are to actual results. We set out to find out if a model says a user has a 10% chance to convert, are we seeing something close in reality?
The answer? Yes, the model’s conversion prediction is close to reality.
What we found:
- Mean variation: –0.4%
- No statistical significance
- Most models didn’t change meaningfully
The implication: Churn scores do not distort model accuracy. They’re safe to include because they won’t over- or underestimate conversion risk.
Discrimination–Can We Rank Users Better?
Discrimination is about ranking users. We set out to know: can the model tell who’s likely to convert versus who’s not?
The answer: Yes, the model can meaningfully distinguish between users who are likely to convert and those who aren’t.
What we found:
-
Mean improvement: +1.15%
-
Some models show strong gains (up to +16%)
-
Statistically significant improvement across models
The implication: Churn scores lead to consistent and measurable gains in how the model separates likely vs. unlikely converters, making targeting more precise.
Budget Efficiency–Is the Result Smarter Spend?
Next, we looked at the business side. Does the model actually spend more efficiently when churn scores are used?
The answer: Yes, the model spends more efficiently when churn scores are used.
What we found:
-
On average, churn scores improve budget efficiency by +1.24%. In the best cases, gains can reach up to +23%.
-
Reflects better ad spend targeting to actual converters
The implication: This is where churn scores shine. They lead to better budget allocation, which means real impact for campaigns with better returns.
The Verdict on Churn Scores: Reliable, Targeted, and Efficient
Churn-based bidding works because churn scores are predictive. They’re not guesswork—they’re grounded in actual user behavior and improve models where it counts. In a nutshell:
- They don’t harm accuracy
- They improve model intelligence
- And they deliver better budget outcomes
Want to spend smarter? Get in touch to start using the power of Adikteev’s churn-based bidding model.