Advice on splitting stories

One of the most common reasons we reject people interviewing for coaching or product ownership related roles is an inability to grasp the purpose and value in splitting stories effectively, especially lacking an understanding of vertical slicing.

This is also a commonly requested topic for me at the meetup or speaking engagements. Yet it is a topic I have struggled to effectively explain. The conversation often ends up as a narrow technical example on certain techniques, or difficult stories or becomes too abstract for people to apply. In short it is a large and complex topic.

But this video sums up the notion of story splitting and in particular vertical slicing and the ‘why’ behind the method so perfectly that I felt I had to share it.

“Successful problem solving requires finding the right solution to the right problem. We fail more often because we solve the wrong problem than because we get the wrong solution to the right problem.”

Russell Ackoff

Dr Ackoff sums up the issue with the analogy of the parts of a car, if you assess the purpose of a car to get you to a destination then an engine alone is worthless, even the best designed and most efficient engine cannot get you to your destination. Until it is connected to the minimal set of features to achieve the user’s purpose it is useless and remains useless.

Building any feature that does not work end to end adds no value, and building any feature that does not support the purpose of the user also adds no value. But more crucially it is often the interaction between layers or between components that is the most complex aspect of any development, be it a car or software. and the notion that we can build an engine, and a gearbox and fit them together later and expect them to work seamlessly is laughable. But I hear it all the time in software design.

A system must have an aim. Without an aim, there is no system.

W. Edwards Deming

We’ll build the database first then add the other layers, or we’ll work on an API layer 3-6 months in advance of the front end. It is as if we assume that the integration is the easy bit and worse is the assumption that we have anticipated every need of the user (and omitted everything they don’t need) before we design and build the interface, and before we ask for any feedback. And yet as software designers; planners; and project managers we repeat this error over and over, never learning from the pain of not using vertical slicing for splitting stories.

I believe our fundamental attribution error is the focus on the blocks of functionality (the components of the car) rather than the interactions, and rather than focusing on the purpose of the tool and user feedback we plan for efficiency of the workforce. The result is an optimized workforce and an inefficient workflow. We create a sub-par product that has efficient working components but do not effectively work together, and generally this results turf wars over interfaces that do not match the use cases and last ditch efforts to fit square pegs in round holes.

We can learn so much from Dr Ackoff, software alone is not the system, software is a tool, it becomes a system only when it is in use. The only way we can know if the software is efficient is by putting working software in front of a user and for them to use it and give feedback. So the only good way to split a story is in such a way that you are able to get feedback from the user that helps shape the design or to assist in making decisions.

If a story cannot lead to feedback or use, then it has no value, it becomes inventory or work in progress, it is a liability rather than an asset. That Database or API layer that is built with nothing utilizing it is not benefiting you, it is waste, it is an over engineered liability and the pain comes when you integrate it with other components. This extends to unused data fields or unused end points, “we know we will need them later” is a poor excuse for creating additional WiP (work in progress).

Learning is not compulsory… neither is survival

W. Edwards Deming

We as Agile practitioners can learn so much from Dr Deming and Dr Ackoff we are building systems, and the development process itself is a system, if we applied a little more systems thinking I believe we could be far more effective.

But as Deming said “Learning is not compulsory… neither is survival ”

Practically Agile

As a coach it is very easy to get wrapped up in theory rather than practice, the topic itself is challenging because it is so simple to understand but so difficult to execute. We see so many Agile Transformations fail, so many poor implementations of Kanban or Scrum and at times it feels great other time so futile, the concepts are neither complex, nor new, but they are very difficult to implement effectively and in a lasting manner.

Successful Agile Software Development is in my opinion based on three similar but intertwining thought processes and if any one is absent the strength of the whole is significantly diminished.

  • Systems Thinking
  • Community Context
  • Reflective Practice and Application

Sometimes we get focused too heavily on the principles and the values but the Manifesto begins with what I think is a statement more important than the rest.

Manifesto for Agile Software Development
We are uncovering better ways of developing
software by doing it and helping others do it.

At the very heart the manifesto is about getting better at delivering software.

We are uncovering better ways” – it is a journey of discovery, we do not have all the answers.  “by doing it and by helping others do it” – It is not just theory, and we share our successes with others so they can benefit from our past successes and failures.

Systems Thinking

It sounds grand and perhaps I would be better served coming up with a less grandiose title but essentially the issue here is that YOU are not the center of the universe. You could mean you personally, or your team.

Systems Thinking = YOU are not the center of the universe

The goal is effective Agile Software Development, solving a problem for a user, with software.  Our system is the whole process from identifying a need, through to the delivery of a solution, and that solution being used to satisfy a need.

Our system is not coding; it is not moving a card from one column to the next.

Having great code on a branch that will not get to the user for months or years (if ever) is not satisfying customers, however proud you are of it. The same is true for designs or perfect architecture for features not needed .

Transformation failures

In my experience I would attribute a significant majority of unsuccessful transformations (and many business failures) as a result of a failure by members of the team to grasp that they are contributors to a larger system, it sounds insensitive, but you are just a link in a chain, a cog in the machine. Focusing too much on one small part is not helping the organisation or the system as a whole.

And yet we expend a lot of effort on improving local efficiency, and at the same time failing to grasp that your efficiency is irrelevant without context.  By focusing on your own local efficiency is at best doing nothing for the larger system and at worst making the larger system less efficient – by not focusing on what is needed.  The obsession with coding efficiency in particular kills a great many software products. I see teams actually proud of a growing pile of stories in need of testing, or a team dedicated to front end UI proud of having endless features complete against mocks and how the back end teams can’t keep up. Sadly these teams seem oblivious to the fact that they are not adding value to the system.

Community Context

Systems thinking refers to the context and the domain but within that is a team – often many teams. The teams are a collection of individuals – all distinct and all different, and your ability to work together (or not) can determine how well you are able to produce software. Teams that bond and grow together achieve amazing things, teams that fail to establish trust can and do churn without making much progress.

An understanding that software development is about people first and foremost, may sound an odd statement when media presents it as a lonely and socially awkward enterprise. But all aspects are about interactions, within the team, with users, with stakeholders, with other teams, the list goes on, but effective communication drives software development.

Those that master the understanding that product creation is a people centered activity and overwhelmingly a team activity will thrive, we build cross functional teams in the belief that self-organisation and motivated groups produce great software – and they do.

It can be tough adjusting from thinking about you as an individual to you as a member of a larger community, but when we can act in a way that best serves our community rather than ourselves we start to become a high performing team, it is worth noting that you do not create a high performing team by simply grouping a number of high performing individuals, often that is a disaster.  High performing teams arise when we learn that we are more that the sum of our parts.

The second part of this is that part of being in a community is sharing knowledge and offering support. The manifesto calls out that part of being agile is not just doing it but helping others do it.  We find ways to grow as individuals and together.

Reflective Practice and Application

Finally and this is the pillar that binds it together and makes it so strong.  We take time to reflect often, so much so that we become skilled in self-reflection and in giving feedback to others to aid their self-reflection.

I believe every team – not just technical teams should take a break and reflect regularly, no matter what you do, you can improve, but you need an environment conducive for that thought process. An hour away from your normal environment may end up being the most productive hour of your week if you can learn to reflect effectively.

Facilitating Retrospectives

Facilitating retrospectives is a difficult skill, getting past the noise to get at the real issues can be difficult and takes skill and practice, but both the facilitator and teams get better at this over time, especially if they reflect on how to improve this skill.

As a group and individually we should take time to identify learning opportunities, we continually observe ourselves and our teams looking for ways to improve. We learn to give and receive feedback in a way that helps us grow – not to diminish us.

We challenge our thinking, we question our beliefs and we look for ways to grow.

We learn how to experiment in structured ways trying new things and observing, we learn the value of metrics and measurements, both in our team and our products.

But the learning and reflection is for nothing if we do not apply what we learn, the application becomes yet another skill we can develop, may of us can become analytical and observant but continue to do the same things despite what we see.

If you always do what you’ve always done, you’ll always get what you always got

Learning to apply our observations and reflections in a structured an effective way is another challenge, and bringing this back full circle to systems thinking we should focus on the area that is holding us back the most and deal with that alone, and then repeat the process.

5-focusing-steps-113297

Trying to fix too many things or focus on too much at once can lead to confusion and particularly if you are measuring impact of changes if you change more than one thing it can be very difficult to know which one had impact.

Summary

All three of these thought processes overlap, intertwine and often depend on each other, but when bound together are extremely strong, and we can and will get better at delivering software and helping others to do it.

 

Applying the Theory of Constraints to Agile Software Delivery

In ‘Agile’ circles there are certain topics that come up frequently, such as Kanban and Scrum, so much so that we often forget that Agile is a mindset rather than a small subset of methodologies.

As the saying goes there are a great many ways to skin a cat, but limiting ourselves to only a few may constrain us from opportunities to improve.  The Theory of Constraints is one of these overlooked options.

The Theory of Constraints (TOC) is often assumed to only apply to industry and manufacturing, which is very sad. TOC is in my opinion the very example of an Agile mindset applied to industry and the techniques can be applied very effectively to Agile Software Delivery.

TOC5

Some common applications of TOC

  • Ongoing Improvements
  • Improve flow
  • Incremental delivery
  • Smaller batches of work
  • Reduced Work in Progress
  • Focus on maximizing customer value
  • Reduce firefighting
  • Reduce inventory of undelivered ‘finished’ work
  • Reduce speculative work
  • Reduce reaction time – ability to pivot sooner
  • Identify potential productivity improvements
  • Expose underutilized capacity
  • Identify skills shortages
  • Identify blocker clusters
  • Identifying new markets or new ways to engage with your market
  • Using cost of delay principles to prioritize work

TOC1

System Thinking

TOC starts and ends with system level thinking, your organization as a whole, not just your team, the whole market, not just your current customers or your current market. TOC requires you to look up and around rather than just looking down.

The Theory of constraints uses the term ‘theory’ in the scientific sense, it is a well defined set of practices that have proven to be very successful when applied appropriately.

The guiding principle is that improvements to anywhere that isn’t your bottleneck is worthless. Think of adding a third row of seats to a car that only ever has one passenger. and that improvements to your bottleneck improve your entire system as a whole so should not be measured in isolation.

This can often feel confusing and counter intuitive, it can sometimes result in decisions that increase local costs and even reduce local efficiencies.  Say shipping by air may seem far more costly per item, but results in an increase in sales that far out strip the increase in costs.  Shipping got less efficient and costs went up, profit per unit went down. All seemingly unwise decisions when considered in isolation.  But the result was that sales went up and so profits increased faster than sales.

And that is the heart of the problem, most organizations create silos and fail to consider the impact of opportunity cost on the organization as a whole. This mindset leaves the shipping department seeking to reduce costs without considering the impact on sales.

In IT this may be seen as saving money by sharing an expert between two teams which means their utilization is high (they are perceived as efficient) but they are not fulfilling their purpose in a timely way to their teams which actually costs the company far more in opportunity cost.  Sadly opportunity cost does not show on a balance sheet and so cannot be seen unless you look for it, and understand the significance of it’s impact.

Having dedicated resources with built in slack often causes traditional managers and PMs to have kittens, they cannot bear the thought of costly resources being underutilised, they only focus on cost, not on the value that people bring and that thinking could be crippling your organisation.

TOC3

Contrary to popular belief your company’s goal is not to keep all workers busy all the time,  it is to be the most profitable it can be now and in the future.  That means effective use of people and not maximum use of people. The two are not the same and until that is understood your company’s profits will suffer.

In agile we suggest dedicated teams and to minimize multi-tasking, limit WiP and so on, this often leads to under utilized resources(people) but far more value to the customer and generally more profits.

Dependent Events

Another key principle is the impact of statistical variation on dependent events.  That is a bit of a mouthful but basically it means that in any given time period, output from a person or a machine varies, neither people nor machines are consistent, we have good days and bad days, some tasks take longer than others, we encounter problems and so on.

The results is that variation in output of one item in a chain impacts on the next and so on, and the longer the chain the more stages are impacted and the more variation disrupts the flow.

This is hard to explain without a demonstration and few people get it until they see it for themselves. Goldratt explained it really well with a game in the book The Goal, but sufficie to say balance is a fallacy.

Balancing the items in a chain (people, processes or machines) does not solve the problem, TOC explores techniques for managing flow to mitigate these problems.

Agile teams apply many of the flow management techniques TOC advocates – training T-shaped people, reducing hand-offs, buffering work, smaller stories, more paths through the system, continuous delivery. These are all techniques that come directly from the Theory of Constraints and have been adopted by Agile development teams.

littles-law-queue

Summary

You would be surprised at how many good ‘Agile’ ideas have come from the Theory of Constraints and yet we continue to consider it to be for industry rather than software. Think how much more we could be missing by not applying the thinking processes.

I’ll try to cover some of the aspects of the Theory of Constraints in future articles with some ideas of how they can be applied to software development.