Welcome to Product Unfiltered, where we talk with Product Leaders about real challenges, how they handled them, and give you the process tools, and frameworks they used to overcome them.
P.S. Did someone forward this to you? You can get this directly in your inbox by signing up for the newsletter here.
Every product leader has a dependency that keeps them up at night. The vendor that could raise prices. The API that could get deprecated. The platform that could change its rules tomorrow.
You tell yourself you'll build a contingency plan. Someday. When things slow down.
Dan Corbin's someday arrived with a six-month deadline and $100 million on the line.
He was running product for the data side of Return Path, an email deliverability company. Their edge: access to millions of real inboxes through a Google platform program. They could tell clients which emails got opened, starred, deleted—not by guessing, by watching actual behavior.
Then Google pulled the plug. Bad actors in the industry had done shady things with inbox access. Google's response: shut everyone off.
"We got a heads up because we had a relationship with them," Dan told me. "They were like, 'Hey, we're sorry you're getting swept up in this. We know this isn't directed at you. But for the safety of our users, this is the step we're taking.'"
He went through the five stages of grief. Denial. Bargaining. Anger. “We tried everything. Can't we do this? What about this? There was just no convincing them."
His team had an idea they'd kicked around before—using machine learning to replicate user behavior patterns. They had years of data showing what different personas looked like. They just hadn't built it. Always something they'd get to.
Now they had six months to make it work or watch the business drain away.
"You know how like a top spins," Dan said, "and then as soon as it starts to wobble, it just goes haywire? Can we keep that tight spiral where it maintains that consistent—so it isn't just all haywire three months down the road."
That was the question underneath every decision they made. Not whether they could build the machine learning—they knew they could build it. Whether they could keep the top spinning while they did.
So they did things they wouldn't normally do.
They needed 10,000 synthetic accounts signed up for hundreds of thousands of email subscriptions. Gap. Banana Republic. German grocery stores. This was 2017—no AI to help. So they hired humans through Mechanical Turk to click through signup forms one at a time. Whatever it takes.
Dan pulled in teams across the company. Legal. Privacy. Sales. Marketing. Product marketing. Data engineering and data science working in lockstep. "It really was an all-hands effort, all coming together for a very specific strategic goal."
He went to sales with a question nobody wanted to answer: if we start losing data, which customers do we let go first?
He thought about it like TV ratings. "You know how TV viewership works—adults 20 to 54, that's what matters. Once you get past that, they're not as important." Same logic. They couldn't rebuild everything at once. So they focused on the core personas and geographies that covered most of their revenue.
"Those smaller clients are like, hey, we care more about what's happening in Austria. Can you go and give me that? And we're like, well, sorry. We haven't really built out that part of the new platform yet. That wasn't as important. But if we can hit the data in these four places—Europe, North America, parts of Asia—we've got most of our revenue covered."
That answered what to prioritize. The next question was how good it needed to be.
The data scientists wanted 99.9% accuracy before shipping anything. In normal times, maybe that's the right call. This wasn't normal times.
"I'm like, nope, you need 98. We'll tweak that stuff later. I just need this up and running. If you can get to 98, holy crap." He paused. "I'm like, you know what? I'd probably even take 95."
They got to 98. It held.
But even with the accuracy locked in, they weren't going to bet everything on a single moment.
"You never want to flip the switch all at once. You want to do one thin slice at a time and kind of fade it over." Google actually delayed the shutdown because other companies couldn't move as fast, which gave them room to transition gradually.
Gradual meant clients would see their dashboards shift over time. Dan had to get ahead of that.
"We told them: we're going to have to do a little bit of recalibration as we move to the new platform. You're still going to be able to take the signals and react to them. We just need you to give us a window while we find the new normal."
The clients who'd been with them for years rode it out.
But keeping clients steady didn't mean the transition was painless inside the company.
The API engineers—the ones who'd spent years building a platform that doubled users while cutting costs in half—saw the capabilities that they had worked so hard to build be shut down. The old architecture couldn't survive the transition. Tough times, tough choices.
"I didn't deliver the message in a really thoughtful way," Dan admitted. "I was thinking about it from a business sense. Like, alright, we're shutting it down. I should have explained why. I should have given more context to people who had put in years."
A lot of them left after the acquisition. The company they'd joined didn't exist anymore.
But the data held. The clients stayed. Return Path got acquired shortly after—an exit that wouldn't have happened if they'd let the top wobble.
When I asked what he'd tell himself six months before everything hit, he didn't hesitate.
"I wish we had done this a year earlier. We didn't have any time to experiment. To play around. To A/B test. It was just—get from point A to point B as fast as possible. There's something good about that, like we're all united behind one goal. But if we'd started sooner, I think there's other things we could have built. "
The contingency you don't build is the one that arrives with a deadline.
The Secret Sauce
“We effectively used OKRs and KPIs to guide our product decisions.”
On a book that helped Dan navigate the crisis: “Radical Focus: Achieving Your Most Important Goals with Objectives and Key Results by Christina Wodtke was a critical book for me to structure our goals the right way and make sure everyone knew exactly what we needed to achieve.”
Here's what else I'm reading
Until next month,
- Matt