Ron Jeffries, who helped coin the concept of story points back in the 2000s, said recently that he “deplore[s] their misuse.”
In the early days of story points, developers “really only used the points to decide how much work to take into an iteration,” says Jeffries. But some organizations expanded their use, using story points to predict when things will be done and to track actual vs. predicted progress. Many, including Jeffries, have pointed out that it’s problematic to use story points in this way, because
- it’s inaccurate, and
- it distracts from what’s really important, which is delivering value to customers.
This is not to say that story points aren’t useful – many organizations find them very effective. But story points alone aren’t sufficient.
Nowadays, frameworks like SAFe® recommend that you use Flow Metrics to measure and improve efficiency, velocity, and predictability. If you like story points and want to keep story points around, the question becomes, “when is it appropriate to rely on story points, and when should we use Flow Metrics?” In this blog, we’ll answer that question.
Definitions
Before we get into the details, let’s review the terminology.
What are story points?
In the early days of Agile methodology, the amount of work for each story was estimated in “ideal days”– the number of days it would take to complete a story if you had nothing to distract you. Since an “ideal day” usually translated to at least three days in the real world, confusion and disappointment ensued.
The solution: to change the name from “ideal days” to “points.”
Story points became a popular way to estimate the effort required for a software development team to complete a user story. Each work item is assigned a certain number of points based on the estimated difficulty of the task, and these estimates are created collaboratively, by everyone who will work on the task: developers, designers, testers, and deployers.
In its original form, a story point translated into three days of work, so a user story that had three story points was estimated to require nine days of work. And teams would set a story point limit per iteration.
Today, organizations use a variety of estimation conventions, like the Fibonnaci number, but the underlying concept remains the same. Many organizations also attempt to normalize story points such that, in each team, one point represents an equal amount of effort.
What are Flow Metrics?
Flow Metrics, part of Mik Kersten’s Flow FrameworkⓇ , measure the rate of value delivery through a value stream, from the moment the work has been accepted until the moment it is delivered to the customer. They monitor the movement of four types of Flow Items, which are features, defects, debts and risks.
Flow Metrics are designed to get you focused on delivering value. If you track them against business outcomes (value, cost, quality, and employee happiness) you’ll see a close relationship emerge, and this visibility makes it easier for teams to see how their work has a meaningful impact on the business.
Should story points factor into Flow Metrics?
The main difference between story points and Flow Metrics is that the former estimates effort, while the latter measures results. In other words, story points answer the question “how much effort do we think this should take?” and “how much work did we do?”, while Flow Metrics answer the question “how much value did we deliver?”
Unlike story points, Flow Metrics don’t focus on the complexity of work or the effort required to complete it. Rather, they measure the whole system, from end to end, including all dependencies. We can clarify this with a traffic analogy: if story points track individual vehicles down one road, Flow Metrics measure the flow of traffic throughout the whole city.
Both story points and Flow Metrics have their place in Agile development, but they are not equally useful or interchangeable. If you want long-term stability and a value-driven culture, Flow Metrics are essential.
Listen to the Mik + One podcast featuring Dean Leffingwell, founder of SAFe
Story points hide bottlenecks, Flow Metrics find them
Imagine a marathon runner who makes it to the 40 kilometer mark and is then forced to stop, right before crossing the finish line, to wait for a herd of cattle to cross the road. In terms of effort, he’s almost done the race, but being “almost done” doesn’t cut it when you’re on the clock. The herd of cattle blocks all the runners, regardless of their weight or height! Similarly, in a software delivery value stream, constraints to flow impact low-effort items and high-effort items equally.
This happens in software companies all the time. Features get planned, written, tested, adjusted, and tested again, but then they pile up, waiting for approval in someone’s inbox. A lot of effort has been expended to get them there, and if you were only measuring story points, you would think you were doing well. But from a value standpoint, nothing has been delivered and customers are waiting.
A bottleneck at the end of your delivery pipeline, or anywhere else in your value stream, can hold work back from the finish line and make work unpredictable, despite your teams’ hard work. To find your bottlenecks, you need Flow Metrics, specifically Flow Efficiency and Load, which can show you where work is building up in wait-states. 
Example: Flow Load data reveals a bottleneck in the verification state
Story points vs. Flow Metrics in the planning process
For product owners, knowing the amount of effort it will take to push through a work item can help them decide which items to prioritize. For individual teams, these estimates inform decisions about which items get pulled in from the backlog on a weekly basis. Story points also incentivize breaking up work into more bite-size units, which helps flow. But they don’t tell the whole story.
Many teams using story points rely on burn down charts to plan their next iteration. These charts track actual progress against ideal progress by showing how much work has been done and how much remains. A burndown chart can tell you if you can get something done by a certain date, but they don’t tell you how workflow could be improved.
To reach a high degree of predictability and competitive lead times, you need a holistic approach to planning that takes into account more than just effort. This is where Flow Metrics come in.
Flow Metrics help you discover opportunities to deliver more items more quickly! Here are some questions to consider during planning:
- How much capacity do we have for new work? (Flow Load and Flow Velocity)
- What type of work should we be prioritizing to keep our delivery healthy and predictable? (Flow Distribution)
- What constraints and bottlenecks should we be aware of and try to remove? (Flow Efficiency)
- How long does it take us to complete a Flow Item on average? (Flow Time)
Think about it this way: Flow Metrics can be used as key results in your OKRs. For example, if your objective is to improve customer satisfaction scores, shortening Flow Time for features is a valid key result, because it will have a measurable beneficial effect on your customers: you will delight them with new capabilities faster.
Get the ebook on designing flow-based OKRs
Story points can create the wrong incentives
Many businesses use story points as a way to push developers to deliver more, more, more. But this backfires. Under such pressure, workers will cut corners, produce lower quality features, and take on too much work all at once. When this happens, unplanned work rolls in and swamps teams that are already working at capacity, and feature delivery slows down.
Although teams using story points are advised to leave an “interrupt buffer” (a bit of extra capacity in case of unplanned work), the pressures of delivering more points may lead them to underestimate how big a buffer they need. Moreover, most unplanned work items (like production outages, defects, and performance degradations that need immediate attention) are not assigned points, so teams are not rewarded for completing them. As a result, important work gets neglected.
Rely on Flow Metrics to incentivize value-delivery
Flow Metrics can help align incentives with business goals, because they provide data, not just estimates, and they account for all types of work. For example:
- Historical data on Flow Velocity discourages wishful thinking by showing you what your capacity truly is.
- Past patterns of Flow Distribution can show you what happens when you try to deliver too many features all at once.
- Flow Time and Flow Load encourage developers to take on smaller batches of work, swarm around them, and get them across the finish line.
You need this visibility if you want to get developers and business owners on the same page and set realistic goals and expectations. And when it comes to the end of an iteration or PI cadence, instead of rewarding teams for doing more points, reward them for maintaining Flow Distribution targets, or keeping Flow Time under a certain threshold.
In your retrospectives, measure outcomes, not effort
Effort is not an accurate proxy for outcomes. Sometimes, even though teams are working hard, the business results don’t reflect their hard work. Why, if everyone is putting in the effort, are teams still missing commitments?
A common problem in software delivery is that developers start too many things at once and fail to get anything completed. Frequent context-switching hinders focus, and having too many things on-the-go means that low priority items get left behind, half-baked, as higher priority items are continuously pulled in. This leads to unpredictable delivery and missed commitments. It’s like a traffic jam, where cars are continually having to pull over to make way for ambulances.
Again, if you were only measuring story points, productivity would look great, because everyone is busy. Story points can tell you that you’ve been working hard, but Flow Metrics can tell you whether you’ve been working on the right things. The insights from Flow Metrics drive you to work better, not just harder.
Story points vs. Flow Metrics: the conclusion
Story points may be helpful for their original purpose. Once stretched beyond that, they can cause organizations to look in the wrong places to aid predictability and velocity. There are much more powerful techniques for planning and setting expectations and becoming more predictable.
While story points can be useful as a team-level capacity estimation, you shouldn’t use them to measure value delivery. You have only to look at a hamster in a wheel to know that hard work, if applied in the wrong direction, is wasted.
The bottom line is this: customers care about results, not effort. They’re paying for results. And to answer the questions “how well did we do?”, “was what we did valuable?” and “what could we do differently?” organizations benefit from using Flow Metrics.
Learn how to use Flow Metrics with the Flow Institute
Join the Flow Framework Community for ongoing support
In a private Slack workspace built on communal learning and trust, connect with like-minded business and technology leaders to develop expertise in value stream management, Flow Metrics and the Flow Framework®. Join the community to continue the conversation around measuring flow and driving business value!
