Is showing-up half the battle?

In the office we have been discussing the possibility of developers Working from home recently. With mixed reactions, and mixed experiences. Judging by this comment from the Yahoo! boss a couple of years ago we are not alone.

http://money.cnn.com/2013/02/25/technology/yahoo-work-from-home/?hpt=hp_t1

Marissa Meyer’s view on home working was:

“To become the absolute best place to work, communication and collaboration will be important, so we need to be working side-by-side. That is why it is critical that we are all present in our offices. Some of the best decisions and insights come from hallway and cafeteria discussions, meeting new people, and impromptu team meetings. Speed and quality are often sacrificed when we work from home. We need to be one Yahoo!, and that starts with physically being together.”

My personal experience was that as a developer when I was working on a clear piece of work I found working from home far more productive, less distractions, I enjoyed my own domain and often it seemed like coming to the office was unnecessary. I am by no means someone that has a negative impression of remote working.

But Scrum is a very different way of working. Pair programming is common and even when not pairing there are continual short conversations; clarifications; general evolution and interaction when implementing and testing stories. The to-and-fro is never ceasing, there is a buzz to the team process. My experience is that even the need to pick up a phone breaks that flow and interrupts the spontaneity. A dev may have a very quick trivial question for the Product Owner and may feel comfortable asking it in the context of a team room, but if they need to make a phone call they may not bother and make an independent assumption – I have seen it happen regularly, and assumptions are the path to the dark side.

I know there are a lot of teams that work remotely, and some successfully, I am not saying it can’t work, but my preference for a truly productive team is to have the whole team together in one room all working together.

That doesn’t exclude consideration for flexi-time, family emergencies, general common sense, good-will and the practicalities of life, all of those are far more important than rules on attendance. But in my opinion the norm should be everyone in the office in one room all together working towards a common goal.

Like Yahoo! we want to create an environment where we can be successful, we also want to create an environment where employees feel valued and respected, where their needs are considered important and ultimately a place where they want to work. But in my idealist vision at least Working from home is a ‘perk’ that doesn’t fit easily with Agile/Scrum. One of the core principles of Agile is The most efficient and effective method of conveying information to and within a development team is face-to-face conversation.

Whilst it is possible to make Scrum work remotely anything that gets in the way of face to face conversation is an impediment that I would prefer to remove if I can. There are many tools to reduce that impediment but I’d rather remove it entirely and have everyone in the same room.

Should we re-estimate stories at sprint planning based on better understanding of how to implement a solution?

Estimates should be based on the relative size of the story. E.g. If our story is to do a 1000pc jigsaw puzzle and we have estimated it as an 8 point story. The story takes us 6 hours to complete. We break up the puzzle and put it back in the box.

We are then asked to do the exact same puzzle again as a new story. We have just done it, so we know the difficult bits, we have fresh knowledge and recent experience, it is highly likely We’d complete the story in much less time. But the story is identical, We still have to do the same puzzle. Last time it was an 8 point story, this time it is still an 8 point story.

In other words our experience changes our ability to complete the story it doesn’t change the relative size of the story. We estimate using relative size because we don’t know who will be doing the story or when it will be done.

Hopefully we learn and get better, equally it is likely a more experienced or senior developers will complete stories quicker, but none of this changes the relative size of the story.

Story point estimates are to offer the ability to forecast, they are accurate in that context and over the long-term only, Think of stories like rolling a dice. A three point story is like rolling a dice 3 times and totalling the results, an 8 point story is like rolling a dice 8 times and totalling the results. Sometimes a 3 point story will take longer than a 5 point story. But in the long run the average will be 3.5 per roll.

I could never guarantee the next roll of the dice will result in a value of 3.5 but what we can offer probability not predictability over a longer period, by the time you roll the dice (take the story into sprint planning) the story points offer no value or interest to the development team. The forecasting value is gone. The story will take as long as the story takes, we must trust the team to do their job.

The value of a ScrumMaster

In June 2014 a new ScrumMaster was brought in to work with an existing and established development team

At the point of joining the team had already raised concerns about environments and tools but the team had been unable to express specific issues that could be resolved, these problems had been intermittent for most of the year. The team was composed of a BA acting as Product Owner, a Lead Engineer, three other developers and two testers with the addition of a ScrumMaster this made a team of size 8, which equates to an approximate cost to the business of around £600,000/$1,000,000 a year.

The team measure their productivity using a term called velocity, which is how much work measured using a relative scale is achieved over a period of time. The amount of work fluctuates for a variety of reasons, holidays, sickness, bugfixing, mistakes, environmental issues, focus or context switching, support requirements, spikes etc. It is an anecdotal estimate of productivity only. But has become the industry de facto standard for measuring improvement. This had been recorded over the previous year and the documented average for the 12 months preceding the new ScrumMaster is noted below as 100 basis points per week.

Velocity can only be used to compare a team with it’s own past performance, i.e. it is a relative scale, not an absolute measure but in that context it is very useful. But should be used with caution. In this case the team had an existing Datum and a set of benchmark stories. The Datum was not modified and the benchmark stories remained in this period, thus there is a consistent base level for comparison.

Year 1 (Prior to ScrumMaster)

The new ScrumMaster, spent some time with the team, observing and evaluating. He sought to identify problem areas and any behaviour in the team where improvements could be made. But unlike a conventional manager or trouble-shooter he did not issue directives on changes to behaviour.

The team scheduled regular ‘retrospective’ meetings where they review the past time period and look for ways to improve. The ScrumMaster used a variety of techniques, to coach and guide the team to focus on areas identified either by the team or by the ScrumMaster as problematic or where improvement could be made. The ScrumMaster collected and compiled metrics to aid this analysis, and facilitated meetings to be productive in deconstructing problems in a non-negative manner.

By using this approach the ScrumMaster did not solve the teams problems, he did not offer ‘his’ solution to the team, nor issue directives for improvement. The ScrumMaster worked with the team to enable them to become aware of their problems and the ScrumMaster created an environment where the team were empowered to solve their own problems. The ScrumMaster uses his past experience to identify problems and may steer the team to suitable solutions, but relies on the team to solve their own problems.

In the following quarter there was a notable upturn in velocity (productivity)

Year 2 (First quarter after ScrumMaster)

The ScrumMaster is responsible for creating an environment where the team can reach a long-term and consistent performance – a spike or a blip is not considered sustainable, so it is important to not seek to boost productivity with short-term fixes like overtime or cutting quality. The aim is sustainable pace and sustainable quality over the long-term.

The ScrumMaster continued to work with the team identifying more bottle-necks and waste, removing impediments and coaching the team how to work around impediments or resolve them themselves, there is always room for improvement in a team.

The ScrumMaster also spent time coaching the Product Owner and Programme Manager on expectations and in their interactions with the Scrum Team, including challenging the Programme Manager on how his priorities are fed to the team so that the team could focus to be more effective.

Year 2 (First full year with new ScrumMaster)

Velocity is only one measure of productivity and is largely anecdotal, however it is the only real measure we currently have available. The productivity improvement should be considered in that context. However, there is a clear and notable improvement in the productivity of the team over the 12 month period. The team deserves the credit for the improvement as they have improved themselves. But by adding an experienced ScrumMaster to an existing team he has been able to coach the team into identifying ways they could improve and in doing so doubling their productivity – a significant change in just 1 year. In this instance it could be argued that the ScrumMaster’s coaching of this one team has resulted in £600,000/$1,000,000 worth of added value to the business and assuming the team does not regress this is an ongoing year on year gain.

It is not often possible to measure the impact on a team, and velocity is a fragile tool for that, but I hope it is clear from these metrics the value that one person can have on a team. It is highly likely that Year 3 will not have the same growth but even if the new velocity can be merely sustained the value to the organisation is significant.

Estimating at a project level.

One of the most difficult aspects of the transition to Agile is the confusion over how estimation is done.

Estimation is difficult, the experts suggest that even with a full picture of what is required and with clear detailed and fixed requirements, the best estimators cannot realistically estimate better than within a 25% margin of error. It’s easily possible to do worse than this. But It isn’t possible to be consistently more accurate; it’s only possible to occasionally get lucky.

But in agile we start without clear requirements, we don’t have a fixed scope and chances are the requirements we do have are at a high level and there are many unknowns. I could talk about the cone of uncertainty but I’m not convinced most businesses will accept that level of uncertainty even if it is based on sound reasoning. In my experience they would rather base decisions on a specific guess than an accurate ranged estimate especially a wide range. Sounds daft when I say it like that but I bet you have experienced it.

Nevertheless it is still often necessary for commercial reasons to have a solid estimate before starting a project (Agile or otherwise), in many situations projects need to provide a good ROI or are limited by a budget. In some situations the ability to estimate reliably could be the difference between the success and failure of a business. These estimates can be crucial.

So how do projects provide reliable and useful estimates?

First of all it is worth noting that estimates are largely misunderstood in general, they are misused and can often be damaging to the project. But still estimates are asked for and used to make important decisions.

In a survey from a few years ago*, a range of IT companies were asked about estimation strategies, the results were both worrying and yet reassuring that difficulties were universal.

* http://www.ambysoft.com/surveys/stateOfITUnion200907.html

Around 44% of the project teams in the survey described themselves as ‘Agile’ so this is a balanced pool of projects and should give an idea of estimation across the board.

When asked to give estimates to the business for project delivery around 65% of teams were asked by the business to provide estimates within the 25% margin of error range that experts in the field say is ‘impossible’. 11% were allowed no margin of error at all they had to specify a single date or a specific cost for the project, conversely 21% were not asked to give any estimates at all. The rest allowed a margin of up to 50% on their estimates.

So how did that pan out for those companies?

Well 40% never even tracked whether those initial estimates were correct, it is difficult to draw any conclusions from this, but 40% came within that magic 25% of their estimates, which frankly is an incredible statistic, when I first read this I started questioning the validity of the survey. 40% of software project estimates were more accurate than the ‘experts’ say is possible to achieve consistently, 40% is more than just getting lucky it is frankly unbelievable. At this point I was about to dismiss the survey as nonsense, but I read on…

How is it possible?

In order to achieve the 25% margin of error the projects did the following:

18% admitted they had padded their original estimate
63% de-scoped towards the end of the project to deliver on the estimated schedule.
34% asked for extra funds to complete the projects on the original estimated schedule
72% extended the schedule to deliver the promised scope (effectively revising the estimate and success was then measured on the revised estimate not the original)

It is impossible to tell from this how many of the projects matched the original estimates, but clearly it wasn’t very many, it is not a stretch to conclude that the vast majority of respondents de-scoped and/or extended the original estimates, including those that had already padded the original estimates.

Moving goalposts is the key

My reading of this survey is that very few if any delivered what was estimated in the originally estimated time-frame/budget. It makes very bleak reading and regardless of whether the project was or wasn’t Agile the estimates did not deliver what the business asked them to.

If we take the stated purpose as being simply to plan and budget and assume the estimates were not padded or interpreted then they hold very little value based on the lack of accuracy.

In my opinion if any of the businesses that demanded such specific estimates went on to actually base business decisions on the accuracy of those estimates, then they were just setting themselves up for disappointment and future problems.

There is no way from this survey to conclude what the accuracy of the original estimates actually was other than to say that even with padding, de-scoping and extending schedules they were still unable to meet the original expectations and were overwhelmingly wrong and seemingly nearly always underestimated the true time/cost. This reads like a recipe for disappointed customers and shrinking profit margins.

That is a very long winded way of saying that (according to this survey at least) no one in the industry, Agile or otherwise is producing reliable estimates for software projects, we consistently get it wrong, and more worryingly fudge the figures so we never learn from our mistakes. So any suggestion that estimating Agile projects is more difficult is not based in fact, estimating for software projects is difficult full stop.

Do estimates have value?

Now that is a different question, if I was running a business and I received a project estimate of 6 months, I would be foolish to consistently believe it will be delivered to the defined scope in that time-frame. But that doesn’t make the estimate useless. If one project estimates 6 months and another estimates 3 months. I can conclude that the first is likely to take longer than the second, especially if the same person or group has estimated both. Both estimates are likely wrong but chances are that on average and over time they will be wrong by a consistent margin, which makes them predictable.

If I check historic records I might be able to see that projects estimated at 6m generally take 8-12 months, or better yet I could ask the estimators to compare the current proposed project and identify a previously completed project that is closest in size and scope and use the actual figures from a sensible comparator. Empirical evidence is so valuable I’m surprised more emphasis is not put into keeping track of past estimates and actual delivery costs and schedules.

Estimates are not commitments

Essentially we need to accept estimates as simply estimates not as a plan or a commitment. Any PM that transposes an estimate of a software project straight into a plan is nuts, and it happens so often that in my experience developers turn white and have panic attacks when asked for an estimate, painful experience says they will be misused and ultimately the one that gave the estimate gets blamed. If the business could be trusted to accept that estimates are not an exact science and factor in realistic contingency based on empirical evidence then developers would be less afraid to give estimates.

So how should we do it?

I have two suggestions, the first is to use an extension of the Planning Poker process. Take a group of people that are experienced with software delivery and relatively knowledgeable about the scope and complexity of what is being asked. E.g. Product Owners, Business analysts, Project managers, representatives from development and testing. Ask them to give estimates of a variety of projects relative to each other. I’d use Fibonacci numbers or T-shirt estimates, to keep it at an abstract level. If possible I’d try to include a benchmark project (or more than one) where the actual time/cost is known.

If we accept that the best we are going to get is a granular; relative; ball-park estimate of a project then this should give you that and more. In fact for budgeting purposes a reliable granular estimate is of far more value than an unreliable specific figure, and far more valuable than the estimates in the survey. Over time it is likely that the estimation team will learn and improve, they will get better with every completed project. I’d have far more confidence saying a project is either a Medium or Large T-shirt. The T-shirt sizes could map to high level budgets.

My second suggestion which could be used in conjunction or independently of the first is to set a budget and ask the team to provide the best product they can within that time/cost. A good Scrum team will be able to prioritise stories and features to ensure you get the best value for money. If that budget is based on the poker estimates above it is more likely that the budget chosen is realistic and you will get the product you want. You will also very quickly be able to tell if the project will be unable to meet the goal and can cut your losses early, rather than having to pour more money into a money-pit project that is over-running but too far down the line to cancel.

Estimation is a difficult skill to master but a group is far better than an individual.