At the Agile2015 conference last August, I overheard a tool vendor demonstrating metrics reports to an interested visitor. Wanting to learn more, I lingered by the booth. The visitor expressed interest in lead time metrics and asked if there was any way to exclude weekends from the lead time reports.
I intruded (in my politest, most respectable way) and asked, “Really? Why do you want to exclude weekends from lead time?”
He replied, “Since my team doesn’t work over the weekend, I don’t want that time counted against us.”
“How is it that lead time metrics count against you?” I asked.
“Well,” he said, “Our VP told us to reduce our lead time. It’s listed in my goals, so it impacts my review. My merit increase is tied to my annual performance review.” My sad, stunned expression couldn’t find any words to accompany it. I left the booth and moved on.
Since then, I’ve heard similar stories. People begging to exclude weekends from time metrics reports appears to be a prevalent phenomenon. And it bothers me. I even vented about it on Twitter recently.
The responses included “Yessss!!!!, Plz!” and “Please do!” — so, voila! — I developed a talk for Lean Kanban Central Europe (#LKCE15) about the topic and also wrote this blog post.
Why All Non-Working Time Matters
Here are three reasons why excluding weekends (or other times perceived to be non-working times) from time metrics can get you into trouble:
- Time metrics are based on assumptions.
- The numbers will likely be gamed.
- Customers care about time in terms of duration.
1. Time Metrics are Based on Assumptions
Time metrics can be discredited by questioning your assumptions. For example: Let’s say that you want to exclude time that the team doesn’t work from lead time metrics.
Do you want to exclude all weekends? Does this mean that no one on your team ever works weekends? How about holidays? Do you work on any holidays? If so, which holidays? How about bank holidays? Do your customers only shop Monday through Friday, excluding bank holidays? What about the team in the UK? Should we go with US holidays or UK holidays?
Do people work at all during vacation time? Should we exclude from metrics the vacation days that workers actually take, or all those that are offered? How about about sick days? People don’t work when they’re sick. What about half days? People can go home early if they don’t feel well.
Many people don’t get much work done during meetings. So, maybe excluding meeting time would provide a more accurate metric. What about an emergency that interrupted the team’s work for two days — should that count? Surely, an issue outside of the team’s control (such as Akamai reissuing a SSL certificate) should not count toward your lead time.
Why not go all the way? Why not just measure lead times in units of hours, excluding anything outside of working hours? Would that be a 40-hour workweek? Is everyone on your team working exactly 40 hours per week — every week?
Those are several examples of the assumptions you’d need to discuss: How much time are you willing to spend defending them, and what business value will the debating generate?
2. The Numbers Will Be Gamed
Beyond debating assumptions, time metrics exclusions will generate a workplace culture that harbors inaccuracy and deception. I worked once for a telecom that required workers to fill out weekly time sheets. Every Friday, I entered 40 hours per week. It was a requisite for billing and how I, as a contractor, got paid.
Employees also filled out time sheets, but not for billing purposes. These weekly time sheets tracked their project hours. Based on the time sheet data, management made decisions about resource utilization and capacity planning.
Some people used a 40/60 rule of thumb, where 40% of their time was spent in non-project events such as staff meetings, training, and other administrative tasks. Others attributed all 40 hours to project work, as they feared it would otherwise reflect poorly on them. As a result, the time sheet data was inaccurate and everyone knew it. Regardless, the data was used because project management office processes ruled the floor.
The adage, “When a measure becomes a target, it ceases to be a good measure,” is attributed to economist Charles Goodhart. Known as Goodhart’s Law, it has profound implications for technology organizations. Many measures are easy to collect in high tech data-capture systems. When seen as credible, they can be deceptively comforting.
An arm of a financial services organization I worked with ended up mandating a 50% reduction in lead time. This mandate resulted in people working weekends and holidays to manipulate lead time and velocity metrics. It was a quick, albeit deceptive, approach to meet the target.
Another team in a different department struggled to finish their stories within their two-week sprint. Instead of sizing stories adequately, they acquiesced under pressure and jumbo-sized their stories. The team worked weekends in order to finish the bloated stories. The velocity measures appeared good, but they didn’t take into account the weekend effort. Leadership was delighted with the apparent velocity improvement and asked the team to start finishing bigger stories in less than a sprint. In short: Madness ensued.
3. Customers Care About Duration
If you’re my business customer, and you ask me when a feature will be delivered, and I tell you in 30 days, you expect it to be delivered 30 days from now. You would not be happy if you discovered 30 days from now that the feature won’t be delivered for another 11 days, because weekends and holidays don’t count.
In my experience, customers have never cared how long their request sat in development or test or waiting for deployment. They’ve never cared about the technical lead time. They cared about the time to market – the duration. When understood correctly, lead time represents the duration of time customers wait before they can provide feedback.
du·ra·tion, noun
The period during which something exists, lasts, or is in progress.
i.e. “The subway stop will be closed for the duration of the convention.”
Synonyms: full length, time, time span, time scale, period, term, span, fullness, length, extent, continuation
Duration has a start time and an end time. It doesn’t do start-stop-start-stop-start-stop. That would be process time and wait time — which are important measurements for studying and improving internal times — but they’re not important to business customers. It’s the sequential calendar days that matter.
Using metrics to shame people encourages gaming. Similarly, target-driven metrics that solely focus on the metric and not the real goal of continuous improvement can encourage gaming.
Improve the Odds of Attaining Good Time Metrics
Lead time metrics are a great way to view trends. Tracking the lead time over a period of time can help you evaluate the effectiveness of improvements or the detrimental effect of changes. Knowing the lead time can also help to predict costs and provide estimates for future work. You can demonstrate that your lead time metrics are creditable by using clear assumptions.
There will always be variability in work due to holidays, sick days and unplanned events. Credit your leadership for being smart enough to understand that requests arriving the day before a three-day weekend will have a longer lead time. They might not be thrilled about it, but leadership will likely appreciate the improved accuracy of your metrics, especially if they’re used for forecasting.
If you’re a manager, avoid setting up your workers for gaming numbers. Be OK with truth. Visibility of accurate data arises from transparency. And, keep in mind the calendar start and end time (a.k.a., duration). Your customers will appreciate the improved predictability of time to market.
Understanding the purposes served by metrics — to continuously improve business decisions and to retain happy customers — can help you focus on the heart of your business instead of shallow targets. How you measure your organization’s time metrics reflects your culture and clearly impacts employee job satisfaction, so choose how you measure carefully.
![A Global Collaborative Work Management Blueprint [Video]](https://blog.planview.com/wp-content/uploads/2019/07/A-Global-Collaborative-Work-management-blueprint.png)
