Is Efficiency the killer of projects?

I have worked in software delivery for over 25 years now and in that time I don’t recall ever hearing of a project being cancelled due to lack of efficiency. By contrast I have seen numerous project cancelled because of misguided decisions in the name of efficiency. The most common is where team decide to build out infrastructure or data layers or business logic that they “know they will need later” and it is more efficient to do them now. The team works really hard but the client sees little or no progress and cancels the project. This sadly is something I have seen far too often.

This thought process is anti-Lean and it is anti-Agile but still it comes up daily. Teams anticipate future work and prepare for it, they over engineer stories or add elements that were not requested, “while I was here I added somethings we will need later”, all the time giving the impression to the customer that progress is slow. The team themselves feel they working hard and are adding value by being efficient. This lack of alignment between team activity and customer priorities and expectations almost always results in customer disappointment and teams feeling underappreciated. Since it is the customer that pays the bills misalignment with them is destructive to a project. That perceived future efficiency is never realized because the project doesn’t last long enough to benefit from it.

So why is it that we see efficiency as such a powerful virtue despite evidence to the contrary?

The sooner begun, the sooner done
proverb

A lot of it stems from misguided definitions of efficiency. From an early age many of us are taught that the sooner something is started the sooner it is finished. Or if you don’t start it, it will never get finished. We have a bulk buying culture and a bulk buy mentality. The belief that over time we will save money or make efficiency gains. What we don’t think about is opportunity cost or what the trade offs are for our actions. These costs in general far out-weigh any efficiency gains.

Cashflow is king

82% of small business closures are due to cashflow rather than profitability. It is not that they are not profitable but that they don’t have the cash available immediately often because it is tied up in unsold inventory or materials waiting to be processed. This is essentially the same manifestation of what is happening in software. We are tying up our time and energy in future potential and we find the project gets cancelled before we get to profit from our efforts. Value to customers is like cash is to businesses. It is no good telling a customer we are accumulating value for later when they need and expect the value now.

So the claim that the best way to get something done is to begin is only half right. The best way to get something done is to finish what you started, before starting something else. If we pay more attention to starting than we do to finishing we end up with many started jobs and very few finished jobs. We need to reverse our sense of efficiency to see value in finishing jobs and efficiency in completing work sooner rather than starting more jobs or extending them.

Stop starting, Start finishing

Whether it be software or manufacturing the efficiency we should be striving for is reducing the number of activities in progress. Be efficient in how many we complete, not how many we start. Starting on something that will not be finished until much later will more often than not create additional work in maintenance and confusion, not to mention the more valuable work that could have been done while you were being ‘efficient.’

Let’s reevaluate our understanding of efficiency and let’s stop starting and start finishing.

Henry Ford – Master of flow

I have recently been presenting a talk on WiP limits and as part of my research I spent some time looking into the history of Henry Ford as he was one of the most influential pioneers of what today we would call Lean or Kanban. He was the pioneer of almost all of our current thinking in industry, and I think was a thought leader in how Agile Software Development is now done.

Theory of Constraints in action

What really struck me however, was how he exemplified the thinking processes behind the Theory of Constraints in all his actions. Everything I read drew me back to that thought.

Ford reduced the average time to produce a car from over 12.5 hours to just 93 minutes.

Ford and his team (I won’t get into the debate as to who had the ideas) created a production line and improved upon it, first by making it a moving production line to keep the focus on flow. Initially simply a winch and rope to pull the vehicle and conveyor belts to deliver parts to the workers. This simple change alone saved hours, previously the workers had dragged their tools along the line of static vehicles as they were assembled.

At one stage the production line for the Model T took 12.5 hours but over the next 5 years Ford reviewed every procedure and managed to cut the production time to just 93 minutes, he cut 11 hours of waste out of an already efficient and profitable system.

Ford cut nearly 90% waste out of an already efficient and profitable system.

Was it a big deal?

Was it really that big of a deal? Yes! In 1914 Ford produced more cars than everyone else, not just more cars than his competitors but MORE cars than all of his competitors in the world combined.

He also produced more with far less, Ford employed 13,000 employees, his competitors combined had 66,000 employees, so the productivity of his employees was 5 times the rest of the industry average. That fact alone makes you sit up and take notice.

In 1914, Ford’s 13,000 workers built around 300,000 cars — more than his nearly 300 competitors managed to build with 66,350 employees.

Continual Improvement

It seemed like no improvement was good enough and Ford was continually pushing for the next improvement, 5 years of asking What’s next. Identifying every bottleneck and then the next bottleneck, his obsession for improving flow must have been relentless.

Fighting against local optimization

But Ford was not understood by his peers who’s focus was on Local Optimization, and even his own sales team who couldn’t understand his desire to simplify the design or reduce the price, they wanted options and variety. Ford wanted a car that was affordable to everyone.

Over the years Ford reduced the price of the Model T from $850 (approx. $22,000 in current terms) to just $265 which was less than 3 months wages for his workers.

The price of the Model T reduced from $850 to just $265 as a result of improvements.

His vision was to have a car available to everyone, but especially farmers, and the Model T was designed to be an effective Farm tool, and could easily convert to farm equipment.

I will build a car for the great multitude. It will be large enough for the family, but small enough for the individual to run and care for. It will be constructed of the best materials, by the best men to be hired, after the simplest designs that modern engineering can devise. But it will be so low in price that no man making a good salary will be unable to own one — and enjoy with his family the blessing of hours of pleasure in God’s great open spaces.

Henry Ford

People Problems

One of the biggest issues that Ford faced was with his workforce, factory workers were unreliable and his new method of simple repetitive tasks and his desire for rapid growth meant he had high turnover and lower quality workers.

The manual processes Ford devised for assembling the Model T were not complicated but benefitted from training and consistency, so the lack of reliability and consistency of workers was an issue for him.

His solution was to double the average pay of factory workers, Ford offered $5 per day, he reduced the working day to 8 hours and the working week to 5 days, he also offered a form of tenure (on his terms) to all employees.

Ford quipped that it was the best cost saving decision he ever made, with a massive reduction in turnover the cost of training plummeted and productivity soared, and what was really a marketing bonanza his workers could afford his cars.

Blunder or Crime?

This decision was national news. But much of the press despised him helping the poor, seeing it as a social policy rather than a sound business decision. The press too seemingly had no comprehension of the Theory of Constraints or system level thinking.

They believed he was hurting business, with one major paper calling him a ‘class traitor’, and commenting that he shouldn’t bring “biblical or spiritual principles into a field where they do not belong.” going further suggesting that paying factory workers that much “was a blunder, if not a crime … against organized society“. That is pretty harsh for a man just wanting to pay people a little more so he could build cars.

Ford had the last laugh though, employee turnover declined radically, and profits doubled to $60 million in 1916 two years after the policy was introduced from $30 million in 1914.

Any customer can have a car painted any color that he wants, so long as it is black.

Henry Ford

Was the Model T only available in Black?

This for me is the most interesting story of all. Some say it is simply a myth, as the Model T was available in many colors over the years, others say it was a metaphor for his policy of being lean, others that it was a metaphor for his dominance in the field and how he could dictate what the customer wants.

Ford himself claims he made the statement in 1909, and yet he didn’t limit production to Black until 1914, and oddly in 1909 you couldn’t get the vehicle in black, it was not one of the 6 color options available. So the quote does have an enigma quality about it.

However, In 1914 Ford restricted the option to only Black and it remained that way for 14 years before expanding to 14 color options shortly before the Model T was replaced with the Model A.

Cost saving?

I have read that the decision to limit to black was a cost saving exercise, with a suggestion that the unit cost of black paint was less than the color options but I have been unable to validate that claim and frankly I find that very hard to accept. Unit cost considerations have not played a part in any of his other major decisions – the wage decision being a clear example of how flow was far more important to him than unit cost. Cost savings were a consideration at a system level only and certainly not a factor if it impacted on flow.

It is all about flow

From what I can deduce from applying lean thinking myself to the situation, Ford would have switched to black even if the unit cost had been significantly higher. Japan Black paint was completely different to the other paint methods and had two distinct qualities that would have appealed to Ford, the first was that it touch dried very quickly and secondly it baked hard in 48 hours, compared to 14 days for the other colors.

Increase flow and reduce Cycle-time

Because the paint could dry quicker it would improve flow and so a production line was able to complete a car every 3 minutes (with 3 minutes effectively being the slowest process on the line).

Reduce Inventory and reduce Lead-Time

For Ford, being able to ship cars from his factory 12 days sooner than previously was massive, he could reduce WiP, but more significantly it enabled him to reduce his inventory of finished goods by 85%. in 1914 at any given time he would be sitting on over $5 million (retail value) of stock, by switching to Japan Black he was able to ship more and more-sooner. That decision would have been an instant injection of over $4 million dollars (that is over $100 million dollars in 2017 terms). And by 1923 if he had still been offering colors that inventory cost to him would have been over $30 million ($750 million – 2017 value). Not to mention the space needed to store 80,000+ vehicles while the paint dried.

Using only black paint was a $750,000,000 Decision

In essence the decision to ONLY offer the Model T in black was a decision worth far in excess of $750 Million in todays terms, and yet very few people understood the ramifications of that decision and many opposed it or ridiculed it, and many still don’t understand it, even with the results being self-evident.

What is Productivity?

Business is a lot of numbers, and I don’t want to bore you with figures but the improvements that Ford made to flow and cycle-time and lead-time all went straight to the bottom line, by focussing on flow rather than local optimization, and by focusing on the throughput of the whole system rather than on keeping one worker busy, Ford was able to get his workers to be 5x more productive than the competition by doing less work.

Thinking is the hardest work there is, which is probably the reason why so few engage in it.

Henry Ford

Ford showed the correlation between effort and productivity is a myth, and it is about working smarter not harder. He passed his efficiencies on to the customer, as his productivity went up the price of the Model T came down. Eventually to just $260 in 1925 which is a mere $3600 in 2017 terms. A car truly affordable to the masses.

System Thinker

Ford seemed to understand Systems Thinking and the Theory of Constraints long before either were recognized, and he did so to a level that few of us will ever be able to comprehend and in the face of public and private pressure against this way of thinking and he was often vocally opposed for his decisions.

Ford changed not just his own organization but his actions changed an industry and likely even the economy of a country. He balanced profitability with altruism, although some of his values and politics were questionable and some of the rules he imposed on his workers would be unthinkable today. But for me he is the pioneer of Lean, Kanban, and the pioneer of the Theory of Constraints, everything since then seems to be built on his shoulders.

Do you know ‘what’ your problem is?

The title is a little tongue in cheek, because in my experience it is the ‘what‘ that is likely your problem. We focus far to much on what we are doing both in our work and in our product, and far too little on why we are doing it.

Failing to understand the why

The obsession with local efficiency and and maximizing utilization of people is crippling customer focus and value.

There is nothing so useless as doing efficiently that which should not be done at all.

Dr Peter F. Drucker

Peter Drucker has some very inspiring quotes but that one is by far my favorite and one that I wish was better understood and adopted in practice

We look for work to stay busy, whether it be setting up grand databases or magnificent CI solutions. I am not saying either is unnecessary, but does the solution fit the need? Were they built when needed or in anticipation of a possible future need?

The focus on lead/cycle times can result in stories becoming rushed, the focus becomes on getting work ‘done’ as quickly as possible, rather than a focus on getting the work ‘done right’. We are completing work quickly at the expense of adding value. It also leads to ‘creative accounting’ where the desire to reduce cycle times results in the definition of done getting corrupted.

Refactoring gets sacrificed when throughput is the priority.

Stories focus on the what not the why.

We are often in such a rush to complete we either skip the why entirely or we take the easy route and look for a ‘why’ that satisfies our ‘what’ rather than actually asking why or giving it proper thought and effort into understanding the real why.

Of course the “Why conversation” can be hard, or time consuming and sometimes it seems obvious but can be hard to articulate. Story writing is a time consuming process but when the process becomes rushed there can be a tendency to prioritize stories according to what is written or what is easy to write rather than what is the next priority based on value or other prioritization methods. We tend towards keeping people busy and the process flowing rather than ensuring we are delivering value and working on the right thing.

It is far too easy to get into a cycle of business as usual, all the time running to catch up but not really knowing if you are working on the right thing. But because everyone is busy and something is getting done “Asking why?” drops below the radar and the hard question of value does not get asked.

Step back and evaluate if you are working on the right thing. Sometimes going slower is actually going faster.

Taking a little while to do it right could well pay dividends. Our goal should be to give our customer the most value, not maximize how busy we are. And whilst logically these two should be related, in reality the opposite can be true.

Learning to write good stories with an emphasis on understanding the why this story adds value, learning to prioritize with the emphasis on why this story is the one we work on next can make a huge difference to a project delivery. Incorporating story maps; road maps, story boards and utilizing other prioritization techniques can help your product deliver the right thing.

The customer should be your focus

Your product owner should be able to articulate clearly and confidently why the next story is next and that justification should be customer centric. Your goal is to make the customer happy NOT to keep your team busy.

When lead/cycle times start to become more important than refactoring it leads to a degradation of the system, either in terms of a reduction in the quality of the code, or an increase in technical debt, but more often a reduction in quality of the user experience.

UX is an afterthought

When we teach story writing we teach the INVEST method, the I is independent, and V is valuable, but I wonder if the independent is misunderstood, especially when it comes to UX. The goal of a story is NOT to minimize Effort, it is to maximize Value. Sometimes we can maximize value by minimizing effort (ROI: Return on investment) but our goal is still the value and NOT the reduction in effort.

The goal of a story is NOT to minimize development Effort, it is to maximize customer Value

A story that adds new functionality could easily be tagged on to the existing system with little redesign of what already exists, e.g. We add a new tab or a new page or a new button, just append it to what is there. In some circumstances this may be the right outcome, but it should be a conscious decision not a default lazy way out.

A new story that is adding functionality should be assumed to be incorporating that extra functionality into the system where it fits best for the user and maintains the user experience and flow NOT where it is the easiest to integrate by the developer. If we just tag on new functionality without consideration for the user or the existing system it can quickly become ugly and cumbersome. Sadly the prospect of changing completed work – especially visible work can be daunting and many teams are resistant to make changes to the UI, instead preferring to add new functionality where it is least disruptive.

Each new story especially UI stories should require some reflection and a conscious decision to maintain flow and look and feel and maintaining the user experience, maintaining the user experience may very well be more valuable than the new functionality adds.

Acceptance Criteria is a substitute for thinking and conversation

We can get so focused on delivering quickly and meeting AC for a specific story that we become blinkered, we know we need to maintain quality in the sense of stability, reliability and security, but we can easily forget about the less quantifiable quality of the user experience of the system as a whole. We can seek to address the acceptance criteria without thinking whether what you are doing truly adds to the value of the product (not story) and when A/C is specific we don’t question whether it is right, we skip the conversations.

Being busy is not your goal

It can be a very hard adjustment especially when you are paid by the hour, or even bill by the hour, for us to accept and understand that sometimes the most effective use of our time is not keeping busy. It may be to do something inefficiently or to not do anything at all. Keeping you busy is not the goal, delivering the best product to the customer is.

I think you are missing the point…

A brief story….

A team found that many of their impediments and problems with delivery were caused by a lack of access to IT operations or lack of responsiveness, they requested a member of the IT operations team be co-located with (and become part of) the delivery team. This was approved and trialled as an experiment.

From the perspective of the development team this was a major success, delivery times improved, delays were reduced and the major constraint to delivery was aleiviated. In short software projects were delivered sooner and with less complications, the overall cost of delivery went down dramatically.

However, the management team decided to remove the IT ops team member and retrurn them to their old team on the basis that they were not being fully utilized. The notion of a human resource not being fully utilized was too much for them to bear.

[Please note when I say utilization – that is on IT Ops tasks – they were using ‘slack time’ to support the team in other ways]

When I challenged this decision, I was told that they were unable to increase headcount for IT operations staff unless it could be demonstrated that existing resources were more than 100% utilized on IT Ops tasks over the course of a normal week. So anyone not 100% utilized was considered a problem for them and became a burden on other team members.

This same IT operations group has a huge backlog of work and lead times were extremely long, they were a major bottleneck for almost every aspect of the business operations. Sadly the management team did not see any correlation between the absence of slack time and the long lead times.

Too often the dependency on IT Operations and their perceived hindrance to business operations, leads to unnecessary conflicts and frustrations. Usually felt by the staff on the IT Ops teams. But it is because IT Operations are so important that they get this reputation.

How can you justify extra resource if you are not fully utilizing the ones you have?

I have no doubt this is a common story, “how can you justify extra resource if you are not fully utilizing the ones you have?” In principle this would appear to be a perfectly reasonable and logical statement, and I can understand why many management teams fall into this trap. But it is the wrong question.

What troubled me most was that we had demonstrated that the impact and delays suffered by the delivery team in just this one example were actually costing the organization so much that they could easily pay for 5 or more additional IT operations staff with the benefits gained. And more so that this situation could be repeated all over the business. The response from the IT Ops manager was that I was missing the point, and they couldn’t justify underutilizing staff. Their measure for success was maximizing utilization of staff, and not supporting business goals, like say profitability.

I see this mainly as a failure of the business, for not making clear to management where they contribute and how they impact on the larger business objectives. That combined with a manager that either doesn’t see or doesn’t challenge this omission leads to perpetuation of dysfunctional behaviour.

Managing resources effectively not efficiently

I apologize for comparing people to resources, but in this context it is applicable.

Could you imagine if that same principle was applied to other resources? Your PC would be taken away if you didn’t use it 40 hours a week, you would be forced sell your car because my guess is that you only use it 10% of the time. You would wear the same clothes every day. Cooking would be a nightmare if you could only have food where all ingredients took exactly the same amount of prep time and cooking time. I’m getting silly now but it is the same principle, by totally ignoring the reason why we have a resource and focusing entirely on maximizing it’s utilization we behave in really rather perverse and contrary ways.

In any other context the notion that anything less than 100% utilization prohibits buying a second item (regardless of how beneficial) is plain nuts.

Please do not misunderstand, I am not encouraging waste by this, I am encouraging understanding of how resources are being used. When you are meeting the business goals and your flow is optimized, that is the time to look at whether reducing waste is possible but to do so in a way that ensures that any reduction in waste is not hurting business goals. It may turn out that just like your car, the optimum utilization is less than 100% and availability and responsiveness are valued higher than utilisation.

Summary

In this situation, I am very proud to be “missing the point” and I wish far more managers would take the time to understand how their role impacts the larger business and perhaps they too will miss the point and start asking important questions. IT Operations in particular are the lifeblood of so many businesses and their actions can make or break a business, understanding all the areas of the business that IT operations supports and how the cost of delay is often significantly higher than the individual cost to operations of that task or that ‘resource’. The ability for IT operations to prioritize and react promptly to the needs of a business is a far better measurement than measuring utilization of staff.

I’d highly recommend that anyone in IT Operations reads the book The Phoenix Project by Gene Kim, and ensure that anyone that benefits from the service they provide also read it.

Understanding how you contribute to the goals of the business is so important, and if after that you truly believe that your role should be maximizing the utilization of resources and not supporting the business achieve it’s goals, then I would suggest it may be you that is missing the point.