ALTERthought Blogs

15 July 2009

Beyond velocity: issue metrics

Continuing our series on beefing up the Agile dashboard with ‘leading indicator’ type metrics, let’s discuss project issues. First, let’s get on the same page on the definition. A good way to understand the definition of an issue is to compare to that of a risk. So, what’s the difference between and issue and a risk? An issue is something that is currently impacting or will certainly impact your project. A risk is something that has yet to materialize, but has a probability of materializing in the future and impacting your project. Further, what is the significance of analyzing issue metrics in an Agile setting? Well, issue management is a prime indicator of how mature an organization and its processes are. This category (and others we’ll cover) gives insight into the mechanics of how well projects are managed in a shop. In our experience, it is the most significant indicator of whether a project is going to experience overruns. It is the “organization’s project environment.”
As we’ve discussed before, understanding what’s happening with issues on your project could serve as a forewarning of impending hiccups in project velocity. But, before we discuss the handful of things to quantify and track, a couple of housekeeping notes.

An Act of Congress to get this info?

Most organizations we’ve worked with have some tool they track issues with: jira, bugzilla, trak, good-old-excel. You name it. So, in general, the mechanism for collecting issues should not be a big stretch.

On the other hand, the process of being diligent enough to log issues is another matter. From our experience in the Agile setting, those issues which get ‘parked’ by project members during a stand-up/scrum for further discussion are obviously clear candidates for further analysis and tacking. Clearly, during the scrum is not the only time that issues arise. The key point here is to be diligent about tracking issues. That being said, one of the key roadblocks to being diligent is insisting on excessive details about the ‘attributes’ of an issue. Now, there is always the respectable goal of having super fine-grained data that you can report and track progress against, but we all know this fine-grained reporting rarely happens. Which leads us to …
Issue “Priority”

Many teams go overboard on trying to assign priority to an issue. ‘Low, Medium, Medium-High, High, Super Duper High, Nuclear’ and variants are not uncommon to see in a priority field. At the end of the day, these designations tell us little-to-nothing about the significance of the issue. In our opinion its more fruitful to use a few meaningful priorities that convey the impact of an issue. For instance: full stop (all work being blocked), progress slowed (work continues, but decisions are needed fast), work continues, impedes/complicates some other function outside of development. Again, from the Agile lens, we care about things that are going to affect progress (i.e. velocity). There’s no need to split hairs by having umpteen variants of “work is being impacted.” If its in the sprint plan, its important; period.

Issue “Statuses”

Status is another area where teams tend to get lost in trying to convey information that is — well — better left for a status report. From an Agile perspective, an issue is either: impacting this sprint, will impact the next sprint, will impact some other future sprint, is resolved, is reopened. So if an issue has priority of ‘full stop’ with a status of ‘impacting this sprint,’ its logical that it should get a lot of attention because you can be guaranteed that velocity is being affected as you speak.

So, when its all said an done, the attributes associated with an issue should basically boil down to: who’s responsible, which sprint, the open date, the close date, the priority, the status, and description. Anything else that is made a requirement for entry is either ‘nice-to-have’ or is getting in the way of effective issue management.

So, what to measure?

Okay, “step down from the sopabox, Sunjay.” So, what is it that we’re analyzing. Well, there are fancy names (in bold) for these things, and there are common sense (in italics) descriptions that help understand why its important to track. We’ll give you both.

Issue Resolution Velocity – how fast issues are being closed: what’s the sprint/project average, mean, moving average? What is the open count of high priority issues versus all logged to date?
Issue Leakage – are issues like zombies — being re-opened?: what’s the sprint/project average, mean, moving average?
Issue Growth Rate – how is the log growing/shrinking?: how many new? how many outstanding, what’s the ratio of new to open?

So at the end of the day, the ‘issue’ category and the data that comes from analyzing it provide important fodder for the agile retrospective. Its a leading indicator that can be used to assess why velocity is slipping or what might happen with velocity in the future. If you can, its probably a good idea to add it to your toolkit.

2 Comments currently posted.