What I look for when retention drops but nobody’s complaining.
03
✦ AdTech · Marketplace Design · Personal Anecdote
What I look for
a healthy yield curve, a quietly leaving mid-spend tier →
What I look for
when retention drops but nobody's complaining.
a healthy yield curve, a quietly leaving mid-spend tier →
Win-rate and revenue-per-auction looked healthy on every dashboard. Advertisers who couldn't tell "you got outbid" from "you were never eligible" were leaving quietly, without a single metric noticing until the churn had already compounded. I found it by accident, in a support-ticket cluster nobody on the launch review was watching.
one signal, two causes 🔍legibility > yield 💡quietly on fire 🔥
Personal Anecdote
This is a Personal Anecdote, not an employer engagement. Written to show how I diagnose and structure a marketplace-health problem in AdTech, the numbers below are original.
0
People on the team
0
% lift, mid-spend retention
0
Loss categories exposed
0
Losing bids manually audited
What I Check
One loss label was hiding two different failures, only one of which advertisers could fix
I wasn't looking for this. I was reviewing a routine support-ticket cluster when I noticed the same complaint from a segment whose dashboard numbers looked completely fine. That mismatch is what made me dig further.
When a marketplace gives two very different failure modes the exact same label, the people experiencing the fixable one have no way to help themselves, so they leave quietly instead of complaining. So now, whenever a loud metric looks healthy, I go check the support-ticket and churn data for the segment that metric can't see. That's the whole practice. Everything else here is just the one time it caught something real.
Here's the auction that taught me to check, and the four things I built to act on it. 👀
The Case
Business problem
The self-serve auction was hitting every yield target while mid-spend advertisers, the segment without a dedicated account manager, churned quietly because the UI couldn't tell them why they lost.
Goals
Confirm the churn was tied to loss-reason ambiguity, ship a fix that made the real reason legible without touching the pricing model, and do it without regressing yield or win-rate.
The Tension
Win-rate and yield looked healthy, while the auction gave two very different losses the exact same label.
How do you fix a churn problem the auction's own dashboards are structurally blind to?
plot twist 🌊wait, what? 😱
The auction was, on paper, working: win-rate and yield curves looked healthy, and short-horizon revenue dashboards were green. But support tickets and account-manager anecdotes told a different story, mid-spend advertisers, the segment without a dedicated rep, were leaving quietly, without complaint, without a clear signal in the top-line metrics until the churn had already compounded.
I realized doing nothing meant optimizing a metric that was structurally blind to the thing actually killing the marketplace: advertisers who couldn't tell the difference between "you lost this auction because you were outbid" and "you lost this auction because your creative or targeting mismatch made you fundamentally ineligible." Both looked identical in the UI. Only one was fixable by the advertiser.
Where the auction quietly loses advertisers
Two very different reasons, outbid vs. structurally ineligible, both drain out through the exact same door.
From Unlabeled Stack to Itemized Stack · how the problem statement first looked
Unlabeled bid stack · week 1
?yield is fine, but mid-spend logos keep vanishing
?same "you lost" screen shown to everyone
Itemized bid stack · the formal problem statement, week 3
Blind spot: Mid-spend, no-rep advertisers
Root cause: One label, two very different causes
Confirmed, not a data artifact
A hunch isn't a case yet. Time to go find out if it holds up. 🔎
Research
What the data actually showed
[1] YIELD METRICS
"Win-rate and revenue-per-auction were flat-to-up; advertiser logo churn in the mid-spend tier was up double digits quarter over quarter."
[2] SUPPORT DATA
"A rising cluster of 'why did I lose' tickets, specifically from advertisers without a named account manager."
[3] SESSION REPLAYS
"Advertisers repeatedly re-submitted near-identical bids after a loss, a strong signal they believed the loss was about price, not eligibility."
[4] MANUAL AUDIT
"Roughly a third of 50 losing bids reviewed were lost to eligibility or quality issues that no bid increase could fix."
From Unlabeled Stack to Itemized Stack · how I thought through the data science angle
Unlabeled bid stack · week 2
?pure price competition
?eligibility/quality mismatch
?creative fatigue
Itemized bid stack · the analysis I actually ran
✕ Ruled out: Pure price competition
Confirmed: Eligibility/quality mismatch, unlabeled
✕ Ruled out: Creative fatigue
Manual audit of 50 losing bids + loss-reason taxonomy
Illustrative, indexed for shape not scale
Every dot is one advertiser 👀
Retaining, self-serve or rep-covered
Mid-spend, sliding, no one watching
The only tier with no dedicated coverage is the only one sliding, and it never shows up in the yield curve.
Audit of 50 losing bids
A third of losses no bid increase could fix
Ineligible advertisers resubmitted almost as often as outbid ones, nearly a third of losses no amount of rebidding could touch.
Problem statement
Advertisers can't act on a loss because the auction gives them one signal (lost) for two different causes (priced out vs. ineligible).
North-star metric
Advertiser-quarter retention in the mid-spend tier.
Guardrail metrics
Auction yield, publisher fill rate, page-load latency for the explanation UI.
Non-goals
Rebuilding the ranking/pricing model itself, this was a transparency and feedback-loop problem, not a yield problem.
connect the dots 🔍
North Star, up close
- North star: advertiser-quarter retention, mid-spend tier
- Ring 2 , primary guardrail: auction yield (must hold flat)
- Ring 3 , secondary guardrail: publisher fill rate, explanation UI latency
KPIs I actually tracked week to week
Mid-spend advertiser retention
↑19%improving
Auction yield
→flat (target)
"Why did I lose" ticket volume
↓improving
Publisher fill rate
→flat (guardrail)
Average bid level
→flat (watched for conservative drift)
Explanation-driven corrective actions
↑improving
Four ways to fix it. Only one of them scaled to every advertiser without a new headcount line. 🧠
Ideation & Decision
Weighing the alternatives
1
Do nothing, keep optimizing yield. Cheapest, fastest, but the evidence showed it was actively corrosive to the marketplace's health.
2
Give every advertiser a dedicated account manager. Solves the information gap, but doesn't scale economically to the mid-spend tier.
3
Build a self-serve "why you lost" explanation surface.Chosen Scales to every advertiser, reuses existing auction signals, turns a black box into an actionable feedback loop.
4
Simplify the auction to reduce eligibility complexity. Valuable long-term, but too slow to address near-term churn; parked as a follow-on.
Accepted riskSurfacing more auction internals could help sophisticated advertisers game the system. Mitigated by exposing categories of loss reason (budget, eligibility, quality band) rather than exact competitor bid values.
Deliberately not builtA real-time bid-shading recommendation engine, tempting, but it would have shifted us from "explain the system" to "play the system for the advertiser," undermining marketplace fairness.
From Unlabeled Stack to Itemized Stack · how I actually weighed this
Unlabeled bid stack · week 5
?trust vs. scale, which option wins
Itemized bid stack · the matrix I brought to the room, week 6
Explanation surface (chosen): High trust, scales to everyone
✕ Account manager (rejected): High trust, doesn't scale
Same four options, now impossible to misjudge
bullseye decision 🎯mic drop 🎤
My Mental Model
Trust vs. Scale vs. Speed
The account manager option wins on trust alone, but a radar makes the real gap obvious across three axes at once, not just two. It barely moves on scale, because it depends on headcount that doesn't grow with the mid-spend tier.
The explanation surface trades a sliver of raw trust for near-total coverage and a faster ship time, the shape of the tradeoff, not just its size.
Explanation surface (chosen)
Account manager (considered)
Deciding was the easy part. Convincing monetization it wouldn't tank bidding aggressiveness was the real work. 🛠
Solution
What if losing an auction came with an explanation?
Actors & Inputs
- Advertisers
- Auction engine
- Eligibility/quality pipeline
- Account managers (escalations)
→
System Change
- Internal loss reason surfaced, advertiser-safe
- Bucketed and rate-limited categories
- One recommended action per category
→
Outcome
- Mid-spend churn recovers
- Advertisers self-correct instead of blind rebidding
- Pattern reused on publisher dashboards
From Unlabeled Stack to Itemized Stack · how I thought through the categorization logic
Unlabeled bid stack · week 7
?if loss=price, recommend "raise budget"
?if loss=eligibility, recommend "fix targeting"
Itemized bid stack · the categories that actually shipped
Priced out: Recommend raise budget or bid
Ineligible: Recommend fix creative/targeting
Low quality score: Recommend improve quality score
Categories shown, not exact competitor bids, fair by design
Toggle the tab to see what the advertiser actually got told.
📈 Auction result, mid-spend advertiser
YOU LOST
ReasonNo reason given
📈 Auction result, mid-spend advertiser
Item 1Ineligible · fix targeting
Item 2Outbid · raise budget
Balance2 of 2 losses now actionable
I underestimated one thing going in: the monetization team's worry wasn't about the mechanism, it was that any advertiser transparency at all would make bidding more conservative and quietly erode the yield curve they were measured on. That wasn't a values disagreement I could out-argue; it needed a number.
Monetization wanted
Minimal visibility, worried transparency would depress bidding aggressiveness
Growth & Support wanted
Full transparency, worried opacity was quietly churning the mid-spend tier
Resolved by: a bounded experiment tied to monetization's own named metric
Instead of asking for buy-in on the full redesign, I asked the skeptical stakeholders to name the specific downside metric they were worried about, then built the experiment guardrail around exactly that metric, turning a values disagreement into a measurable bet everyone could accept.How I led
How the two quarters actually went
MONTH 1
Churn surfaces in tickets
Rising "why did I lose" cluster, concentrated in advertisers without a rep.
NoticedMONTH 2
Audit confirms it's not just price
A third of 50 losing bids were eligibility/quality, unfixable by rebidding.
DiagnosedMONTH 3
Explanation surface proposed
Monetization pushes back; bounded experiment agreed on their named metric.
ProposedMONTH 4
First copy rewritten
Raw quality-score language tested poorly; rebuilt in plain language.
AdjustedMONTH 6
Shipped, then extended
Live for mid-spend advertisers; pattern extended to publisher dashboards.
Shipped 🎉Proof
| Hypothesis | Advertisers who understand why they lost will take corrective action and stay active longer than those who don't. |
|---|---|
| Design | Randomized rollout of the explanation surface to a subset of mid-spend advertisers; holdback group kept the old binary win/loss UI. |
| Primary metric | 90-day advertiser retention. |
| Result (illustrative) | Retention improved meaningfully in the treatment group, with no statistically significant yield regression. |
| Follow-up | Extended the explanation categories to publishers' fill-rate dashboards, the same signal-vs-noise problem existed on the supply side. |
Advertiser understands the real loss reason
→
Corrective action taken (budget/creative/quality)
→
Reduced churn in the mid-spend tier
→
Healthier long-term auction liquidity and yield
Shipped isn't the same as done. Here's what I'd actually keep, and what I'd do differently. 🕑
Learnings
Key takeaways
What worked
Turning an opaque system into a legible one didn't require touching the pricing model at all, the fix was in the feedback loop, not the algorithm.
What didn't
My first version of the explanation copy was too technical, all raw quality-score language, I had to redo it in plain language before advertisers actually acted on it.
Next time
I'd start with the plain-language copy test before building the full categorization pipeline, I over-invested in backend precision before validating the front-end comprehension problem.
real talk 💬lesson locked in 🔒
"Metrics are dashboard lights, not the steering wheel."
Principle carried forward"Start with the decision, not the feature."
Principle in practiceFrameworks & skills applied
Marketplace & auction design
Trust & legibility design
Cross-functional alignment
A/B experimentation
Incentive-aware product framing
Trust vs. scale prioritization
Marketplace & auction design
Trust & legibility design
Cross-functional alignment
A/B experimentation
Incentive-aware product framing
Trust vs. scale prioritization
Quick Answers
FAQ
Is this a real project from a specific employer?
I wrote this as a personal anecdote to show how I think through a marketplace-health problem in AdTech, not to claim a specific past engagement.
Why an explanation UI instead of touching the pricing model?
The audit showed the auction was already reasonable, I decided the real gap was legibility, not yield, so that's what I fixed.
What would you do differently?
I'd test plain-language copy before building the full categorization pipeline, I'd have saved a cycle of re-testing after the mechanism was already validated.
Doesn't showing advertisers why they lost help them game the system? 👀
Only if you show them raw internals. Showing categories, not exact competitor bids, keeps the system fair while still being useful enough to act on.