Welcome to Product Unfiltered, where we talk with Product Leaders about real challenges, how they handled them, and give you the process tools, and frameworks they used to overcome them.
P.S. Did someone forward this to you? You can get this directly in your inbox by signing up for the newsletter here.
Starting a new company is never easy. There are the nerves of excitement, anticipation, and optimism: new job, new opportunities, new problems to solve. Getting plugged in seems simple enough.
Then cracks start to show. Small ones at first. Then they're getting bigger. And they're moving toward you. Now leadership is asking what you're going to do about those cracks. They're holding you responsible. You need to figure out how to seal cracks that were there before you arrived.
You've been here. Maybe you're here right now.
After 20 years of experience in product management, Tiffany Wiley has seen plenty of dysfunction. But this one had a specific flavor that made it hard to diagnose.
The team wasn't ignoring the process. They had standups. They had sprint planning. They had retros. Everyone was in the same room, on the same calls, looking at the same tickets. They'd agree on sprint goals together — not top-down, genuinely collaborative. And then, sprint after sprint, they'd miss.
They'd talk about it in retro. Support issues came up. Tickets got added mid-sprint. Production issue fires that pulled people off their work. Reasonable explanations, every time. But nothing changed.
"Are we on track?" Tiffany would ask mid-sprint. The team felt like they were on track. Great.
They weren't.
The roadmap kept shifting. Four sprints of work turned into eight. Features customers were waiting for slid quarter after quarter. And Tiffany had to deliver the message no product leader wants to: it's going to be later than we said. Again.
"It makes us look bad," she told me. "Not just the product team — the engineering team overall. When you're a SaaS organization, the product is what you sell. When you're not meeting those goals and those deadlines, it reduces confidence across the board. Internal stakeholders, external stakeholders. No one wants that."
Some of the slippage was invisible to customers — backend infrastructure work that wouldn't surface as visible changes anyway. But the bugs they were waiting on? The features they'd been promised? Those were visible. And every time Tiffany had to go back and say, "Your stuff is coming," the credibility account got a little thinner.
Here's the thing she realized: the team wasn't lazy. They weren't sandbagging. They genuinely believed they could hit their goals. They just had no way to prove it — or disprove it — until the sprint was already over.
Everything was gut feel.
So Tiffany did something that sounds simple but took months of patience to actually pull off. She started building instruments to measure what was happening inside the sprint, thereby increasing team accountability and making progress more predictable. Not all at once. One layer at a time.
The first layer was to get the team to estimate using story points—Fibonacci sequence, relative complexity, effort, and risk —not time. The team didn't want it. They wanted hour estimates.
Tiffany wouldn't budge. "That gives a very false sense of precision," she said, "especially with a team that has been consistently not meeting goals. To give that kind of false sense of accuracy because of the preciseness of time — I really didn't want to go that route."
It took months. She'd get to the end of a four-hour planning session and still have unpointed tickets. She didn't let up. "Your gut feel is not working," she told them. "I appreciate your desire and your want to do what you think you can do. But we need something more reliable to back up what we can actually get done."
She celebrated when they got their first fully estimated sprint. Then she started building the reports.
First iteration: a simple stoplight at the end of each sprint. Green if the goal was met. Yellow if it would be done by the end of the day. Red if it wasn't happening. The team saw a lot of red. "You don't want to demoralize the team," Tiffany said, "but we need visibility. If we don't have visibility into what we're doing and what we're not doing, it's hard to make a positive change."
Second iteration: tracking what got added to the sprint after it started. Nine tickets were added mid-sprint on one cycle. Only one was completed. Three still sitting in "new" status when the sprint closed. Now the team could see what the interruptions actually cost.
Third iteration — and this is where it got interesting. Tiffany started tagging every ticket to a specific sprint goal. And the first time she ran the numbers, 63% of the tickets in the sprint weren't associated with any goal.
Sixty-three percent.
Everyone in the same planning calls. Same discussions. Same tickets. And nearly two-thirds of what they were working on wasn't connected to what they'd said mattered.
"It was eye-opening," she said. "That helped reshape the conversation on planning day. Like — do we need to do these? Maybe. But should we be doing them is really the issue."
She kept layering. Velocity tracking came next. Then carryover points — how much unfinished work bled from one sprint into the next. Then she started monitoring tickets that ping-ponged between developers and QA, tracking whether each bounce-back involved an actual code change or just a miscommunication. After seven or eight sprints of watching that metric, the ping-ponging stopped. Observation changed the behavior.
She built Jira automations to capture conversations that were missed. Anytime someone added a ticket to an active sprint, Tiffany got a Slack notification about scope creep. Tickets sitting in the same status for five days triggered an automatic nudge to the team channel. Developers couldn't move a ticket to QA without entering point estimates — a gate that forced the discipline that the discussions hadn't.
None of this was expensive tooling. Jira fields, a Google Sheet connected via a webhook, pivot tables calculating metrics, and results dropped into a Google Slides template. Two or three clicks and you have a sprint report.
The latest iteration tracks what Tiffany calls their say-do score: how many points they committed to versus how many they actually completed. By monitoring this metric, teams can set more realistic goals and improve the accuracy of their sprint planning, leading to better project predictability and stakeholder confidence.
Their most recent sprint? Eighty-one percent.
That's the thing about riding a bronco. Everyone watching from the outside thinks you're out of control. What they don't see is that every small adjustment — one more layer of measurement, one more automation, one more conversation grounded in data instead of gut feel — is slowly calming it down. The bronco doesn't stop bucking all at once. It just bucks a little less each time.
Tiffany's not done. The say-do average is still 49%. The sprint reports keep evolving. But every retro now has real numbers to back it up. When a sprint scores low, the conversation isn't "what happened?" with shrugs and vague explanations. It's "here's what the data shows" — and here's what we adjust.
Two years of not letting up. That's what it took. Not a tool purchase. Not a process overhaul. Just stubborn, incremental measurement — one layer at a time — until the chaos had nowhere left to hide.
The Secret Sauce
Layer 1 — Sprint goal stoplights.
At the end of each sprint demo, go through the goals and mark them green (met), yellow (will be done by end of day), or red (not happening). Simple visibility into whether you hit what you said you'd hit.
Layer 2 — Mid-sprint additions tracking.
Track what got added to the sprint after it started. Break it down by type (bugs, tasks, stories) and show the status of those added items at sprint end — how many completed, how many still in progress, how many never even got touched.
Layer 3 — Bug metrics.
How many bugs completed and closed, broken out by source — support-reported bugs versus bugs found during sprint development.
Layer 4 — Ticket-to-goal association.
This is the big one. Tag every ticket in the sprint to a specific goal using Jira labels. The first time you run this, you might find that over half your tickets aren't associated with any goal. That reshapes what you bring into planning.
Layer 5 — Points and velocity.
This is the layer that takes the longest to get right — not because pointing is complicated, but because getting a team to do it consistently takes patience. Start by picking a few existing tickets as your baseline. A two-point story looks like this. A five-point story looks like that. Compare everything else relative to those, and don't let anyone equate points to hours. Points measure effort and complexity, not speed. Once your team is pointing consistently, you can track committed points vs. completed points per sprint and start building a velocity picture over time. Your early reports will say "greater than" or "approximately" because not everything is pointed yet. That's fine. Imperfect data is better than gut feel.
Layer 6 — Reopened/ping-pong tracking.
Monitor tickets that bounce between dev and QA — how many times, whether code changes were actually made on each bounce or if it was just miscommunication. Track this for several sprints. You might find the behavior self-corrects once people know it's being measured.
Layer 7 — Carryover points.
Track how many points bleed into the next sprint. If your release cycle doesn't align with your sprint cycle, some carryover is structural — but you want to limit it and know how much is normal versus a sign of overcommitting.
Layer 8 — Planned vs. unplanned completion.
Of the completed work, what was in the sprint at the start versus what got added mid-sprint? This tells you how much of your output was intentional versus reactive.
Layer 9 — Say-do score.
Points committed vs. points completed, expressed as a percentage. This is your headline metric — the one number that tells you whether your sprint planning is getting more reliable over time. Recalculate the rolling average every three to six sprints rather than every sprint. If you recalculate every sprint, the number swings too much to be useful. You want to see the trend, not the noise.
The tooling underneath all of it:
You can connect Jira to a Google Sheet through a tool like Zapier so your sprint data flows automatically, or just export your sprint data from Jira into a Google Sheet at the end of each sprint. Build pivot tables to calculate the metrics. Drop the numbers into a slide template. No expensive add-ons required.
Until next time,
Matt