At Tasktop, our engineering teams have been using an Agile Scrum process for years, with great results for our software delivery. This process has helped us to reduce waste and provide predictability and visibility in our integration factory, which delivers 4 on-time releases every year. However, my own team, which had only recently been formed, was experiencing some pain in this process.
End of sprint delivery was becoming unpredictable: it was common at the end of a sprint for one or more of our stories to be “almost done,” causing us to push the remaining work into the following sprint.
This reduced our apparent velocity, as these in-progress stories were re-estimated for scheduling for the following sprint. Additionally, it created a reluctance to pull on unplanned work near the end of the sprint for fear of not completing the additional work by the sprint’s end. The biggest impact, however, was on team morale, which suffered given our seemingly reduced velocity and the lack of freedom to start stories on our own timeline.
One of the key differences on this team compared to other teams was that we were building features in an area that was entirely exploratory. The product was not only a greenfield project, but there was also a lot of technical and business uncertainty about what we were building. We needed a lot of experimentation and short iteration cycles with our product team and users to inform the shaping of product features. This caused some waste during a sprint cycle in which stories were re-scoped or priorities shifted.
With that in mind, the team set out to explore switching to a Kanban process aiming to satisfy the following goals:
Improve team efficiency in planning: reduce the amount of time the whole team had to sit down to do sprint planning by limiting the amount of time spent deciding what should or should not be scheduled for a sprint.
Improve the team’s ability to change focus: allow priorities to be changed quickly with minimal disruption to the team. Ideally, the team’s backlog of unstarted work should be able to be reprioritized with no impact on the team’s workflow.
Improve the accuracy of the team’s internal velocity metric: avoid re-estimating stories across sprint boundaries.
Improve the feeling of completion and achievement on the team: avoid arbitrary deadlines that can cause the team to feel like they failed to meet an objective that didnt necessarily impact release goals.
To integrate more easily with the rest of the engineering group, we decided to start with a Kanban process that included retrospective and demo meetings on the same cadence as the rest organizations sprint cadence. But we dropped the bi-weekly planning meetings in favour of continuously maintaining a prioritized and estimated backlog which team members could draw from as soon as they became ready to take on more work. Initial work-in-progress limits were set both at a story and epic (feature) level to ensure the team stayed focused, but all other barriers to starting new work were removed to improve the teams workflow.
We identified some potential drawbacks and risks using this process:
Ongoing vigilance would be required to ensure the backlog would always be prioritized (at least near the top) and ready to start, in order to avoid team members waiting on work. Without sprint planning sessions, we would need to continuously ensure that new stories are ready to start.
It was not clear how defects and technical debt should be prioritized in the backlog: how would we ensure that we would end up with a proper mix of technical debt, defect and story work without sprint planning?
There would no longer be any end of sprint drive for completion. Would this impact the team’s overall velocity by not having an external motivator to complete stories by a deadline periodically?
We would no longer have sprint themes to guide coherence of work. Without this, wed run the risk of losing focus on the work we are doing which could both slow the team down as well as cause loss of coherence in the code itself.
We are now 6 months into our Kanban process, and its been a great improvement. Team morale has drastically improved, our velocity doubled and stabilized, and waste was eliminated in rescoping our stories. Additionally, moving to a more continuous workflow reduced the waste involved with previously periodically scheduled work. For example, because bi-weekly sprint planning meetings were replaced with on demand (when necessary) feature breakdown sessions, they became much more focused, stress-free and we avoided discussing features that would never get implemented due to reprioritization.
But, it hasn’t been flawless. We’ve had to make the normal adjustments along the way that we discovered during retrospective meetings, but we’ve also had some challenges both with tooling and with forward looking predictability and visibility around our work. Particularly around meeting external expectations on the team that assumed, like the rest of engineering, that we were on a sprint-like cadence; it became difficult for stakeholders to easily understand what was expected to be delivered by a certain mid-release point, and it became difficult for the team to have visibility into whether the release was on track.
However, this process made us more agile. We were able to respond quickly to feedback from customers with minimal disruption and waste to the team. Reprioritizing and rescoping stories on the backlog, even those that were up next, had next to no impact on the team (as long as it hadn’t been started yet), as we avoided having to have discussions about what would come on and off of the sprint.
Watch for a follow up post about our concrete implementation of Kanban, how we addressed the potential drawbacks, risks and real problems that we encountered to improve our flow, and how we manifested this process in our tooling (primarily JIRA).
