I seem to have had a great many conversations of late to do with estimating, not story estimating which is now pretty well understood and accepted, but now the discussion is about project level estimation.

Project level estimation is like a hydra, every time you think you have dealt with it, it comes back with two more heads.

I think part of the problem is that we don’t have a common language, we use the word ‘estimate’ to mean so many things. I get the impression that there is a great deal of confusion between “Accuracy and Precision” or between “A Plan and an Estimate”, or “An estimate and a foretelling of the future” and worst of all, the difference between an “Estimate and a Commitment” I also wonder if the question being asked is the right question, are you asking “if the project is viable” or “will it give a good return on investment”.  And finally there is a notion that if you do not deliver in line with your estimate then you got it wrong and that is bad.

What is an estimate?

Roughly (and generally quickly) calculate or judge the value, number, quantity, or extent of something.

Was my estimate wrong?

Let’s start with an easier one.  I estimated a project would cost  £1m and it ended up costing £2m, did I get it wrong?
Superficially this may seem obvious, for a commercial organisation to consistently underestimate projects would result in bankruptcy, clearly it must be wrong?  But an estimate in most practical situations isn’t wrong simply because the outcome doesn’t match the estimate, an estimate is made quickly and roughly based on incomplete information and the true answer is not known until it is done. You can overestimate or underestimate, even wildly, but you can’t be wrong.  Sounds confusing but let’s take a practical example.

I estimate the time it takes to drive into town,  past experience says that it would take between 10 and 20 minutes.  So I estimate 15 mins.    If the journey actually takes 30 minutes, that doesn’t make my estimate wrong.  I most certainly did not correctly predict the future, but I wasn’t wrong.  If I was asked the next day to estimate the same journey and I felt that the previous day was abnormal I may estimate 15 mins again.   I still believe the basis for my original estimate holds true and one abnormal example doesn’t change that.  But let’s say I discovered there were road works on that route, I may alter my estimate to reflect the new route or to reflect the anticipated delays.

I’ll give a second mathematical example to hopefully simplify things.

If I roll two fair six-sided dice, can you estimate the total of the face values?

Mathematically speaking the most likely outcome is ‘7’ the probable average of doing this many times is also 7.   So mathematically speaking it would be sensible to estimate ‘7’  but 5 times out of 6 the result would not match your estimate.  Our estimate was ‘correct’, we can mathematically prove it is ‘right’ and yet we can get the ‘wrong’ outcome 5 out of 6 times.

Conversely if I estimate ‘4’ and roll the dice and get it ‘right’ that doesn’t make my estimate right, it makes me lucky.

That doesn’t help the poor businessman that has just gone £1m over on his current project, but hopefully over time he will get better at estimates and he will on average get it ‘right’.

My point is that an estimate is a useful tool to help guide our decisions; it is not a method for foretelling the future.

Are you asking for an Estimate or a Plan?

Back to the car journey example, I have made the journey 10 times and the shortest time was 10 mins the longest was 30 mins and the average time was 15 mins.   If our goal is long term consistency then the best ‘estimate’ is probably 15 mins, because the likelihood is that If I did the journey 10 more times then the average would again be 15mins.

Easy!  But I have an appointment at 4pm that I mustn’t be late for, what time should I leave? My estimate is 15 mins so I leave at 3:45pm. But at that rate I’m likely to miss my appointment a high proportion of the time.  What went wrong, my estimate was perfect?

My mistake was that I confused an estimate with a plan.  My estimate is useful information, but whilst my journey is likely to average 15 mins I need to build contingency into my plan if I have a fixed deadline.  This all sounds obvious but in your own experience how many times has someone taken an estimate and put it directly in a project plan with other dependencies relying on it?

My point once again is that an estimate is a useful tool to help guide our decisions, it is not a foreteller of the future, and used in isolation without understanding it can be confusing or damaging.

Flexible or fixed

But let’s go further, if my estimate becomes a plan then my 15 minute estimate needs to become a 25 minute ‘plan’ so I can offer reasonable confidence in completing by a fixed deadline.   Anyone notice that to have a confident plan then in this example on average it contains 10 mins of contingency, and on average 10 mins of potential waste when a fixed deadline is imposed?   If I planned on doing this journey 10 times and had to be confident of meeting an agreed time each time, I would need to plan for 250 minutes, whereas if I simply allowed my plan to flex and take as long as it took I would likely do the same 10 journeys in 150 minutes, clearly that is a huge difference – by removing a fixed commitment I allow contingencies to be shared and am far more likely to achieve more. It is a contrived example but illustrates the amount of waste that is necessary in a fixed plan.

One last point is that even with significant contingency there is still no guarantee, I could still break down or there could be something unexpected that causes me to be late.

Are you turning an Estimate into a commitment?

Let’s take the journey and appointment a little further.   I use the estimate of 15mins and I add a sensible and realistic 5 mins contingency and then you say… “Can you guarantee you will be on time?”

No!  You didn’t ask for a guarantee or a commitment, you asked for an estimate, my estimate was 15mins and I feel pretty confident in that estimate.  If you want a guarantee, then I can’t give one, I couldn’t guarantee 45mins or even 60 mins, I cannot offer a 100% guarantee because I cannot foretell the future I can only offer an estimate based on my experience of the past.

Had you asked me whether I could be ‘confident’ I’d be on time or if it was ‘probable’ I’d be on time the majority of journeys then I’d be much more comfortable with the assessment.

My point is that an estimate is a useful tool to help guide our decisions, it is not a way to foretell of the future, and certainly the future does not come with a guarantee.

“Accuracy and Precision”

Another oddity with estimation is that I am often asked for a more accurate estimate. Let’s say I’ve estimated approximately 6 months, or given a range 4-8 months.   The response is quite often “can you be more accurate?”

What do they mean?   An estimate is by nature a rough approximation, hopefully accurate. But clearly they are expecting something else. They will only know if it is inaccurate after the work is done.

Take two estimates:  A) 2325.4 seconds.  B)  1 hour.

Which is more accurate?

I don’t have a clue, accurate is in reflection to the outcome which is as yet unknown. One estimate is more precise than the other but that doesn’t make it any more accurate.

My guess is that when they ask you to be more accurate what they mean is either, “What is your confidence level for this estimate?, what could you do to increase your confidence level?”. Or as is more often the case what they actually mean is “That is higher than I was expecting/hoping”

Can you estimate that again…

This is a fun one, whilst it is true that the more information you have the more confident you can be of your estimate. However, there is a law of diminishing returns, and in software the domain is generally so complex and there are so many variables that the effort in investigating is disproportionate to the gain in understanding and confidence in your estimate.  For example I’d suggest that spending a couple of days investigating for a 6 months project could lead to a reasonable level of confidence in an estimate, spending a further 2 weeks investigating often as not will not alter the estimate by a significant degree and will only lead to a marginal increase in confidence in the original estimate (I am not saying that you will not gain a lot of understanding, but the issue is whether it materially impacts on the estimate). Usually it results in more clarity of what you had assumed, and highlights how much you still don’t know.

Summary

That may sound defeatist, but the reality is that in most software projects, over the course of the project requirements will change, new requirements will emerge and priorities will change. The team may not remain the same; external resources may not be available when needed. No amount of investigation can predict the future with any degree of certainty; the best we can do is identify potential risks and be prepared to adjust. If you must have a fixed deadline then you really should have a significantly flexible scope. Have confidence in rough estimates and be agile.

So if we can learn to create plans based on rough estimates; prioritise work sensibly and be prepared to adjust, we are far better able to deliver, and are likely to deliver more than if we insist on fixed plans based on what will always be incomplete information.  A fixed plan by nature will include contingency and contingency generally gets used, there is huge potential waste and the cost of change is high.  A flexible plan, should flex to the amount of time needed or the scope should flex to the amount of time allocated.  If you cannot do this, however good your intentions, then directly or indirectly you will need to introduce wasteful contingency, to enable you to fool yourself you are delivering to a fixed plan.