One of the things I have noticed over the last few years is a drastic increase in the use of metrics. Particularly with the surge in the use of electronic boards for teams data is much more readily available and so is much more frequently used.
If you can’t measure it, you can’t improve it.
Peter Drucker
Previously there was an effort required to collect and process data to be meaningful, but now much more is easy to do, often it is done for you. But is this a good thing?
Problem 1: Lack of understanding the data or the tool
There are two problems I frequently see, the first is that we have access to data and tools but that doesn’t mean we know how to read the data or understand what it is telling us. Monte Carlo simulations are a great example here, I regularly see them being used without understanding what the data is telling you or understanding the limitations of the information. Instead the results are copied into forecasts and endorsed like they are gospel. The information is too easy to get that there is no longer a need to have an understanding to use the data.
Problem 2: Lack of understanding the consequences
The second problem is that our behavior changes as soon as we are measured, sometimes consciously sometimes not. Generally speaking this is the purpose of measuring, we measure what we want to improve, be it our weight or lap time or velocity.
However, there are nearly always unintended consequences, and they can be unexpected and unpredictable.
The Cobra Effect
The Cobra effect is term that originated in a story from British colonial rule in India. The British rulers were worried about the number of venomous cobras.
So they created a policy of putting a bounty on dead cobras. People were paid handsomely to deliver dead snakes for a payment. What could go wrong?
Well at first the reward system worked well, the number of snakes seemed to decline, but the bounties paid continued to rise. The British became suspicious and investigated, they discovered that the reward was encouraging enterprising locals to start to breed snakes for the reward, it was easier and safer to breed snakes than catch wild ones, and they were breeding them in large numbers.
The British rulers were shocked at the dishonesty (It is simply not Cricket, old boy) and were so outraged they cancelled the program and cracked down on the breeders, who released the captive snakes in to the wild. The result was a massive increase in snakes rather than a decline.
Republic of Congo
There is a much more horrific version of this situation from Leopold II (of Belgium) rule in the Congo. If your quota from your rubber plantation was not met the overseers were expected to execute the wives and children of poor performing farmers and send their hands (as proof of death) in lieu of the rubber quota. – a hand equated to a quantity of rubber.
The problem was, some of the villages found it easier to supply hands than to supply the rubber, so were meeting their quota with a mixture of hands and rubber, there were some reports of entire quotas being met just with hands.
Bad as this policy was, it then got worse. The hands were not necessarily of poor performing farmers or their families. Some villages and soldiers that would collect quotas were attacking other villages, and killing each other just to chop off hands and then using the hands instead of rubber as payment.
Such was the extent of this practice some reports suggest that the population of the region reduced by as much as half as a result of this practice. Estimates suggest that 10million people were killed for their hands.
Parallels
Now those were extreme cases and it seems unfair to draw parallels given the seriousness but nevertheless it is human nature to respond to how we are measured, and by implication rewarded.
I hear tales of management that expects velocity to rise every sprint, and guess what it does. But does actual value delivered go up – I very much doubt it.
I see teams wanting to mark a card as done and open a new identical one, because the cycle times are looking bad.
But the worse is the measure of being busy, far too many organizations have a fixation with everyone being busy 100% of the time, regardless of whether what they are doing is contributing to delivering value to the customer or if having a little slack would allow the team to deliver far more.
Suggestion
This sounds like a message that metrics are all bad, it is not at all. Metrics can be very useful and as many say you cannot improve what you don’t measure. The key though is to have those being measured understand the desired outcome and share or support it.
Back to weighing yourself for a diet, your goal is to be healthier, your weight is a measurement. tampering with the scale may impact the measurement but will not help you to your goal. You may fool yourself for a while but it is only yourself you are fooling.
A clear goal, and a variety of metrics that support that goal is often more valuable than a single goal, especially if you are supportive of the desired outcome.
“Tell me how you measure me, and I will tell you how I will behave”
Eli Goldratt
Beware of the measurements you use, you may very well be defining the behavior you get, even if it is not your intent.
John Yorke 