Welcome to Product Unfiltered, where we talk with Product Leaders about real challenges, how they handled them, and give you the process tools and frameworks they used to overcome them. 

P.S. Did someone forward this to you? You can get this directly in your inbox by signing up for the newsletter here.

Every product leader has the line they love to say.

"We have 200 integrations." "We support every major framework." "10,000 SKUs in the catalog."

It's the line sales opens with. The line that fills the trade show banner. The line that's supposed to end the conversation: we've got you covered.

And it works. Until the customer opens their dashboard on QBR day and starts doing math you didn't want them to do.

Josh Gilliam ran into this at an online training content company. The organization was built largely through acquisition. Plumbers, electricians, cops, EMTs, teachers — anyone whose license requires continuing education. By the time Josh moved into a product role for the commercial business unit, the catalog had passed 10,000 courses.

Sales loved it. "I'd go to a conference and hear them say, 'Oh yeah, we got over 10,000 courses,'" Josh told me. "They used that as a soothing mechanism to say, we got you covered."

Then the QBRs started getting weird.

A customer would pay their annual fee, log in to the reporting dashboard, see how many course completions their team had actually completed, and quietly do the division. The number they got back was not flattering.

"They started doing math and saying, 'Well, I'm basically paying this dollar amount divided into that number of courses. I'm not getting my money's worth.'"

The line sales opened with was the line customers were closing with. Same sentence. Different rooms. One was a pitch. The other was a churn signal.

You probably know where this is going, because you've been on one side of this conversation or the other. The strength becomes the weakness the moment someone runs the numbers. And someone always runs the numbers.

Josh wanted to know how bad it actually was. The problem wasn't messy data. It was a SKU mismatch. Vector sold at the library level — collections of courses bundled together — but the question he needed to answer was at the course level. And courses showed up in multiple libraries. The same first aid module might live in three different bundles for three different industries. There was no clean way to say "this course generated X dollars" because no course had ever been sold on its own.

So he built a proxy. He pulled usage data at the course level. Averaged cost per library. Worked backward until he had a dollar sign next to each chunk of the catalog. Not precise. Precise enough.

"I realized about a tenth of the catalog generated 90 percent of our revenue. People looked at me like I was kind of crazy. I'm like, here's the numbers. Go run this yourself."

A thousand courses providing the business and customer ROI. Others had value, but the ROI diminished quickly as you started to account for operational costs. Then there are perception costs when customers see content they don't use or when the presentation level is inconsistent.

The easy button would be to simply draw a line and cut. However, there is value in some titles beyond use. This required a refined approach.

Keep. Question. Kill.

Keep was easy. The courses with the usage and the regulatory necessity. The Accreditation content. The OSHA stuff. The credential-required content that customers literally can't operate without.

Kill was easy, too. The fifth version of the same first aid course from three acquisitions ago, with the instructor wearing a mullet and using what looked like an Apple II.

The work was in the middle. The question pile.

For that, Josh built a second frame. He grabbed the GE matrix off the shelf and rebuilt it for content. Two axes, five parameters each.

The market axis asked, "Is this industry growing more than 5% over the next ten years?" Is it hiring? Is it heavily regulated? Is safety a primary concern? Is the trajectory pointed the right way?

The right-to-win axis asked, "Do we have internal subject-matter expertise?" Can we have peer-to-peer credibility with this audience? Do we have customer referrals here? Can we sell into this segment without faking it?

Then he plotted every contested course into the four quadrants. The top right got kept. Bottom left got cut. The other two quadrants forced an actual conversation instead of a vibe.

"If someone in our top-right quadrant came in and said, 'I need a training course on X,' we'd probably build it — not because they asked, but because it likely had applicability in markets we were trying to grow. If someone outside that quadrant asked, we probably wouldn't."

That's the part most catalog rationalizations skip. The decision isn't just what do we have. It's who we are, and who we are not. The matrix gave Josh a way to say no out loud, with a defensible reason, to industries that had been getting yes for a decade because nobody had bothered to draw the line.

Now he had a recommendation. A thousand courses, not ten thousand. Different libraries, different pricing structure, room to upsell add-on modules instead of stuffing everything into one box. The math worked. The frame held.

He could have walked it into the next executive meeting and asked for a decision.

He didn't.

"I would just shop ideas around early. Knock on a door, get a 30-minute call, and say, 'This is what I'm thinking. Tell me why this won't work. Tell me why this is the dumbest idea you've ever heard.'"

Sometimes the answer came back: "That's not the dumbest idea I've ever heard.” Okay, keep pulling. Sometimes it came back with a flaw he hadn't seen, and he'd fix it before it ever hit a room with stakes.

The executive pushback he was worried about never really materialized. The CPO had already done that air cover work. The pushback came from somewhere else.

It came from the salespeople who'd been selling 10,000 courses for a decade.

They didn't have counter-data. They had a reflex. This was the line that closed deals. This was the demo they knew. The new structure meant new training, new positioning, and new conversations with customers about what they were no longer getting.

"This is what they were comfortable with. This is how they always sold it. They thought they didn't have the data, but their opinion was fairly entrenched."

Josh didn't out-argue them. He just kept showing the numbers, kept the conversation focused on the problem rather than the proposal, and waited for the math to do its work. Eventually, it did. Not with a parade. With a shrug and a "let's try it."

The restructure went in. New catalog shape, fewer SKUs, room to charge more for less. Josh checked the website recently — three years later, the structure he proposed is still live.

He wasn't around to see it through. He'd moved on before the rollout finished. That's a real footnote, not a humble one. You can do the analysis, build the frames, run the politics, and still not get to plant the flag at the end.

When I asked what he'd tell himself starting over, he didn't go to the analysis or the matrix.

"It's your baby. You can't really let it be your baby. What you're trying to do is create a vehicle that grows the business and creates value for customers. If you do that, that's the win."

That's the part that lets the rest of it work. The Pareto. The matrix. The shop-it-around. None of it survives if you walk into those conversations needing to be right. The 10,000-course line was someone's baby, too. Probably several people. They built it, sold it, won deals with it, and got promoted on it.

Telling them the line was now working against them — and asking them to put down the thing they'd been carrying for ten years — only worked because Josh wasn't asking them to admit he was smarter. He was asking them to look at the math.

The line you love to say is worth auditing. Every once in a while, walk into a QBR and listen for the sound of a customer running the division.

The Secret Sauce

The Catalog Audit Framework (Josh's two-stage system)

Stage 1: Keep / Question / Kill

A blunt triage. Every item in your portfolio (feature, SKU, integration, course, product line) goes into one of three buckets:

  • Keep — High usage and/or non-negotiable for the customer (regulatory, contractual, table stakes). The decision is obvious.

  • Kill — Low usage, redundant, or aged out. Often, duplicates are accumulated through acquisition or scope creep. The decision is also obvious, just uncomfortable.

  • Question — Everything in between. This is the pile that needs the second framework.

The trap is treating Question like Keep by default. The whole point of the audit is to force that pile into a real decision.

Stage 2: The Where We Play / Where We Win matrix (for the Question pile)

A 2x2, with five weighted parameters on each axis. Plot every contested item.

Market parameters (X-axis — "should we play here?"):

  1. Industry growth rate over 10 years (>5% threshold)

  2. Hiring trends in the segment

  3. Regulatory density

  4. Safety or compliance pressure

  5. Trajectory of demand

Right-to-win parameters (Y-axis — "can we win here?"):

  1. Internal subject matter expertise

  2. Peer-to-peer credibility with the buyer

  3. Customer referrals and proof points

  4. The sales team's ability to sell it without faking it

  5. Defensibility against competitors

Decision rules:

  • Top right (high market, high right-to-win): Keep. Invest. Build adjacent.

  • Bottom left (low market, low right-to-win): Cut, even if it has some usage.

  • Top left (high market, low right-to-win): Don't build for new requests here. Acknowledge the gap.

  • Bottom right (low market, high right-to-win): Maintain only. Don't expand.

Until next time,

Matt

Keep reading