Which do you use and Why?
I have already blogged about some common metrics but here are ten that I use or have used most frequently. It is my aim to summarise them here and expand each in greater detail over the next few weeks. (I will come back and add links as I do)
Warning!
Before we begin, the first piece of advice that can be said about metrics is that before you record data or use metrics you should really ask why you are recording the information and how it will be used.
If you cannot be sure that the metrics will be used for the benefit of the team then you shouldn’t record the metrics and certainly shouldn’t use them. If ever metrics are misused they become worthless as they are very easily gamed, especially if the team feel they will be abused.
Metrics are almost always reliant on honesty and there are not many metrics that couldn’t be ‘gamed’ if someone was motivated to do so. Never attribute reward or penalty for a metric or even multiple metrics. As soon as you do they are broken.
Have you ever heard tales of teams that were paid for number of lines of code written or for number of bugs fixed. I have worked at companies where the annual bonus was dependant on a high management approval rating in an annual employee survey.
Any environment where you are effectively penalised for honesty loses all hope of trust, transparency and integrity and your metrics become worthless.
If there is even the hint or suggestion that any of the metrics will be used for anything other than self-improvement and introspection again they immediately become worthless.
But that being said as a Coach measurements can be very useful. Measurements, metrics and reflection and retrospection are hugely valuable for coaching an individual or team on how to improve. An Agile Coach or Scrum Master has a variety of metrics available – and these 10 are by no means a comprehensive list, actually it is a very limited list, Each project or development environment is different and so tailoring appropriate metrics to your particular environment is important, not all metrics apply to all projects.
One last note is that there are myriad tools for tracking projects, some will measure these metrics but many won’t and it will be necessary to manually record the data. It will often be difficult to backdate if you haven’t been recording it.
In projects I work on I try to keep a lot of metrics just for myself, much more than the team uses. If at any point I feel there might be value in a metric I will bring it to the team and allow them to decide if it has value and if they are interested. But too much becomes confusing. As a rule of thumb they team only really have interest in a few of these, although which ones may vary over time. Keeping metrics without using them is also a good way of avoiding them responding to inspection.
Beware of context
I am also aware that metrics can be misleading if taken out of context. Information can be dangerous if misunderstood, if I choose to publish metrics in reports or in the office on a wall I will work hard to make sure the context is clear and if appropriate there is an explanation. But even then you can be sure that someone will misuse the information, so be prepared!
1 – The Burndown:
This is probably the most often used Scrum metric. In essence it measures work outstanding at the start of the sprint and a line is drawn converging on zero at the end of the sprint and each day the progress or burn-down is measured. The goal being to complete all planned(and emerging) work by the end of the sprint.
2 – The Velocity Chart:
The velocity chart can take many forms and can be used to track a number of key factors. In the most simplistic form it is a bar chart showing the number of points or sometimes the number of stories completed in the recent sprints.
3 – The Velocity Committed/Actual Chart:
Sometimes the velocity chart can be further enhanced to show committed points compared with actual points, this is useful when the team is not consistently delivering on commitments.
4 – Release Burndown
The principle is that for a given product we measure the Burndown of the entire product in a similar way to a Sprint Burndown, it may take the form of a cumulative flow diagram or be similar to a velocity chart.
5 – Cumulative Flow Diagram
A cumulative flow diagram tells all sorts of stories about a project, essentially it measures the number of stories in various states as the project progresses.
6 – The Burn-up
The Burn-up measures the actual effort expended in the sprint. In a perfect world it would be an exact inverse of the Burndown. But rarely do we predict all tasks in advance nor estimate the ones we know accurately. It can help see where time went and improve our estimates.
7 – Flow time, Lead Time, Cycle Time
There are some varieties in how these terms are used but I use a simplified version as follows:
Flow-time is the time from when a story is started to when it is completed by the team.
Lead-time is the time from when a story is added to the backlog to when it is completed into production – from conception to getting the value.
Cycle-time is the average time from when a story is started to the point where it is completed – we could be working on multiple stories at once, or overlap them. e.g. each story could have a flow time of 3 weeks but we could stagger them to deliver 1 per week.
Using averages for these allows us to measure whether we are improving over time.
8 – Health check survey
A health check survey is more touch-feely than the other more raw metrics but can be useful for gauging the team’s own opinion of their progress and behaviour. You can measure opinions on anything, from software practices to interactions with other teams, to quality of stories, anything the team has an opinion of can be measured.
9 – Predicted capacity : Actual Capacity
Each sprint during planning – stories can be broken into tasks, each of those tasks can be given a time estimate. When totaled up and the team has committed to the Sprint content, this total effort estimate is the ‘predicted capacity’ at the start of the sprint. This is the important number. Tasks will almost always be added as the sprint progresses and many of the task estimates will be off, but the initial predicted capacity is a measure of what the team is anticipating the workload will be for the sprint.
In theory the actual capacity of the sprint is simply the number of team members multiplied by the number of hours available in the sprint, excluding, planned holidays, planned meetings, predicted time on non-sprint activities.
Tracking these figures allows the team to gauge whether the capacity is realistic in relation to the previous sprint, the end result will be a (hopefully consistent) ratio.
10 – Scope creep
This can be covered by the cumulative flow diagram, and by the release burndown, but sometimes it is useful to explicitly report on what has been added or removed from the Scope each sprint.
John Yorke 