How We Prioritize Incremental Product Improvements
A collaborative, metric-led framework for making small product decisions consistently
Product teams rarely struggle because they have too few ideas. The harder problem is deciding which customer request, reliability improvement, defect, or usability enhancement should come first—and explaining that decision consistently.
For TrueTest, we use a lightweight prioritization framework that connects day-to-day product decisions to a shared customer outcome. It gives Product and Engineering a common language while preserving room for judgment.
Start with the North Star
Number of TrueTest-generated test cases that customers successfully adopt and execute.
The North Star gives the team a practical filter: does this work help customers generate, adopt, and successfully execute useful tests? The metric may evolve as the product and its market fit develop, but it provides a clear decision anchor today.
The framework at a glance
Loading the prioritization workflow…
Prioritization is a conversation, not a calculation
Product and Engineering evaluate each item together. The Product Manager brings customer impact, urgency, proof-of-concept timing, revenue signals, and expected effect on the North Star. Engineering brings effort, technical constraints, dependencies, feasibility, and confidence.
Disagreement is useful when it exposes assumptions. The goal is not a perfect score; it is a predictable and transparent decision that both disciplines understand. When the two perspectives cannot converge, the item is discussed with the broader team and finalized together.
Step 1: filter before scoring
Scoring every request creates noise and gives weak ideas a false sense of legitimacy. We first ask whether the item can create a positive impact on the current North Star.
Items without a meaningful connection—such as purely cosmetic changes or requests that do not currently help customers adopt and execute generated tests—are set aside. This is not a claim that they have no value. It means they are not the best use of the team’s capacity under the current product objective.
Step 2: categorize the surviving work
High severity
The issue blocks or significantly harms a customer’s ability to generate or use test cases during onboarding, active usage, or a proof of concept. Examples include generation failures, missing major journeys, or object-recognition problems affecting a substantial portion of generated tests.
Interpretation: if unresolved, the customer cannot achieve the North Star outcome.
Medium severity
The work improves the reliability, accuracy, stability, or speed of reaching the outcome. Examples include better flow selection, faster map generation, or a workflow redesign that removes recurring friction.
Interpretation: the customer can achieve the outcome, but the improvement makes it faster, more consistent, or more dependable.
Low severity
The item adds usability, future extensibility, or supporting insight but does not directly increase the North Star metric today.
Interpretation: it may be a worthwhile investment, but it is not currently required to unblock customers or deliver the core outcome.
Step 3: score four decision factors
Customer and commercial impact
- 3: strategic customer, major contract value, renewal or expansion risk, or POC at risk
- 2: several mid-sized customers or one meaningful customer
- 1: one small customer
- 0: internal only
North Star impact
- 3: high severity
- 2: medium severity
- 1: low severity
- 0: misaligned and normally filtered out
Effort, scored inversely
- 3: one day or less
- 2: two to five days
- 1: one to two weeks
- 0: more than two weeks; split it
Confidence
- 3: clear problem, proven reproduction, validated feasibility
- 2: some unknowns remain
- 1: unclear or inconsistent evidence
- 0: highly speculative
The inverse effort score intentionally surfaces quick wins. A very large item receives a zero not because it lacks value, but because it needs to be broken down before it can compete fairly with incremental work.
Step 4: turn the score into an action
| Score | Priority | Expected action |
|---|---|---|
| 10–12 | Immediate | Address now; may interrupt the current sprint |
| 7–9 | High | Plan for the next sprint or current delivery cycle |
| 4–6 | Medium | Batch for a future cycle |
| 0–3 | Low | Park and revisit when context changes |
Why scoring is guidance—not governance
A score can make assumptions visible, but it cannot understand every dependency or strategic nuance. A high-confidence quick win should not automatically displace an urgent customer blocker. Nor should a large customer request automatically override product integrity or long-term technical health.
The framework works because it structures the conversation. It reduces recency bias, loudest-voice bias, and unproductive debate while making exceptions explicit. Judgment remains with the team.
The outcome we want
Good prioritization does not mean everyone’s request is accepted. It means stakeholders can understand why the team is doing one thing before another, and Product and Engineering can commit to the same decision.
For incremental TrueTest work, the combination of a North Star filter, shared categorization, lightweight scoring, and clear action buckets gives us that transparency without slowing the team down.