Goldratt User Stories

As a product owner we should be building products to deliver on the Vision and a set of  defined objectives or success criteria. But often our plan is not closely mapped to those criteria or it can be difficult to measure whether we have achieved our Vision or delivered on our objective.

Let’s change the way we write stories

So I wondered if we could expand on a desire for delivering measurable value by changing the way we both plan projects/product and write our user stories?

 

Tell me how you measure me and I’ll tell you how I will behave.

Eli Goldratt

 

The notion of this is that rather than defining our User Stories in terms of behaviour, or actions or activities –  in which we primarily focus on the ‘What’ we should change, our start point should become us defining the “needle we want to move”  Let us start with the measurement of success. Understanding how we will be measured and identifying behaviour that impacts the measurement.

photo_of_the_day_ferrari_458_italia_dashboard

E.g. our goal is to increase the number of items per basket for online sales, our measurement is therefore the average number of items sold per transaction.

Equally it could be to increase the average value of the basket per transaction, or increase the number of units sold of high profit products

Hypothesis rather than Story

Our ‘story’ then would be to ask the Product Owner and Team to identify one or more hypothesis for ways that could impact that measurement. The notion being that a hypothesis must be tested before the story could be considered done.

This could take the form or a brainstorm session to identify the more traditional stories or it could simply be left for the team (in conjunction with the PO) to determine the best way (possibly two or three alternatives) to achieve the desired outcome after the story has been pulled.

This is building on the principle:

The best architectures, requirements, and designs
emerge from self-organizing teams.

Already happening?

You could argue that this is already the role of the Product Owner and that the Product already does this, and in identifying and prioritizing stories they have already considered the impact.  In some cases I would agree with you, in a few cases I have seen products or projects with clear objectives in terms of the goal of the product and even sometimes a measure of success in terms of measurable objectives.

I have yet to see the end goals mapped even at a high level to the stories and features, in most cases success criteria are wooly or undefined and more often than not the Product Owner and team will brainstorm stories (story mapping) without reference to the goals, and even Impact Mapping often will not extend to measurable goals.

That is not to say that the Product Owner is not considering this holistically, the measurements are generally part of the consideration but many stories creep in that have no material benefit or impact on the success criteria of the product.

From-the-book-User-Story-Mapping-by-Jeff-Patton

Measurements for Goals as the basis for Story Mapping

For reference I love Story Mapping, I consider it my favourite tool for Product Ownership and have been known to rave a little too much about it on occasion.

The start point in story mapping is typically to identify themes of user activities (you could call them epics but don’t get hung up on terminology they are essentially big rocks and then we will progressively break them down in to smaller rocks arranged underneath the big rocks.

We can then prioritise the work in the context of the whole rather than being constrained by a linear backlog which makes the context difficult to visualise.

It is also a fantastic tool for explaining the product for release planning and forecasting and so on. Like I say one of my favourite tools.

But what if instead of starting with activities or big stories, what if we started with our measurable goals or success criteria.   e.g. new subscribers per month, revenue per visit, time to achieve objective using application etc.  And we then identify stories that best enable us to achieve that goal.

There may very well be some overlap so there maybe the notion of tagging a secondary objective, but imagine if this helps us weed out stories that are not taking us to our goal or have limited value in the context of our goal.

We would ask which of these stories has the potential to move the needle the most? and prioritise accordingly.

download (9)

Make it measurable

Focusing our efforts on measurable impact to the product should implicitly be the goal of any Product Owner so why not make it our explicit goal?

 

 

 

 

 

Advertisements

How do I know I am delivering Value?

Many of us are relatively familiar with the notion that stories should be expressed in terms of value delivered (Why) and how important the “Why” is for being able to maximize the outcome for the customer–

 

Who: As a …

What: I want to :

Why: So that …

 

When we talk of good stories we refer to INVEST as a means of helping validate that our stories are well written, I think this is a great tool for helping write user stories, we may even include Acceptance Criteria to help the team identify that the story has been completed in such a way that the value is best able to be realized, but I’d like to propose going a step further.

success

Expected Vs Actual

Whichever way the story is written the assumption is that the PO has determined the value of the story and prioritized it accordingly, but value is a very nebulous term and encapsulates all sorts of things many of which are assumptions or just plain guesses or personal preferences. We are also making the assumption that the story will successfully deliver the value we intend.  Rather than accepting that it is a hypothesis that it will achieve our goal.

Up to this point we are assuming that the Product Owner is always making the right decisions, and their assumptions of the value delivered by a story are infallible. Speaking as a Product Owner, that is a rather hopeful assumption, often value judgements are little more than educated guesses, certainly they are very subjective opinions on value. Even market research is guesswork to some extent and particularly with new products or internal systems there is little opportunity for effective predictions of value.  I don’t want to take anything away from the PO and their authority on making these judgements that is after all their role.  But as a PO I would very much value a feedback loop which would enable me to validate that my decisions were right or wrong and give me the opportunity to course correct accordingly.

In other words it is necessary for us to make judgement calls but getting feedback on the accuracy or otherwise of those decisions would be hugely beneficial.

download (5)

So what can we do?

We could add to the Acceptance Criteria to include some additional validation. Our Acceptance Criteria helps us validate that a story is implemented the way we intend it to be implemented, it does not however always enable us to measure whether the value is fully realized.

For example:

Assumption: We believe that adding a picture to listings on a product website will increase sales by 10% (our market research says so).

As an online customer,

I want like to see pictures of products

So that I can make more informed buying decisions (and thus buy more products) –

(Business Value: Marketing estimates sales increase of 10%)

 

Our Acceptance Criteria may stipulate positioning of the picture, the size and what to display if a picture is not available. We may even add some performance Acceptance Criteria such as average page load time. But that is not enough to validate that the value was achieved.

missing3

How do we validate that a story delivers Value

But how do we validate that this story does actually deliver the value we expect – whilst we can be confident that having a picture fulfils some aspects of the value – the better informed decisions, it might be that we are missing out by not having the ability to zoom on the picture, or it may be that our users are not bothered by the picture at all and would prefer another feature such as the “lead time” or “quantity in stock”

Validation Criteria

What if as part of this story we not only implement the feature to show the picture but we also include analytic measurements on the page load times, and even a measurement for the number of sales of a product or products per day and then evaluate 50% of users with the new feature and 50% without the pictures and compared the results. Or we could conduct focus groups on this feature or usability studies to get more subjective but detailed feedback.

As part of the story we could add an additional layer of Validation Criteria, this would be similar to Acceptance Criteria but would be a way to measure the value actually realized by the user

download (6)

What do we gain?

Would including functionality or activities that enable us to measure that we have delivered the value we are expecting make the stories better? Would that information help shape our product and build a better product? Would it help us prioritize our backlog as we get a better understanding of value actually delivered vs value expected to be delivered?

We could either add stories for these measurements or consider these to be encapsulated in the delivery of this story.

Essentially we are asking whether feedback is valuable and if it is – how valuable is it to us.

download (7)

Return on Investment:

When discussing this with a colleague the first response was that this is putting more upfront work and that is a challenge for the ‘lazy’. In Agile ‘Lazy’ is a virtue so this is important feedback.

Naturally there is an overhead in this but as with all feedback loops the information is valuable, knowledge is power and we just need to tune our efforts – our feedback volume to the right level to get valuable information with the minimum necessary effort. Another example of an Andon Cord where if the effort is too much and producing either too much information or nothing of value then we need to retune our feedback loops to give us enough valuable feedback to act.

Also many of these measurements will be applicable to multiple stories so the investment may end up being very limited but the feedback may be far reaching, and once automated the ongoing feedback can be tweaked to add extra sensors to give us more and more valuable information.

Some examples of value assessing measurements:

  • Website analytics: hit rates, click through rates, hot spots etc.  The cost of these is minimal and is often something that can be applied even after development.
  • We could write stories to build into our application measurement for how our product is used, or the performance of our product,
  • We could add usability testing or focus groups, surveys of users
  • By using feature flags we could set up effective A-B testing to get feedback on structured hypothesis validation.

Please note that not all measurements need to be software driven – increased subscribers say may be measured entirely independently of your application.

Navigation

Vision

But ultimately the biggest change would be in your initial vision creation, do you know your product goals and do you have a way to measure success.

Is your goal increased sales, or time saved, or efficiency improvements, or increased users or cost saving and regardless of your goals do you have a plan for measuring whether your product is achieving your goals.

This may seem like stating the obvious but I cannot count the number of projects I have been on where the stated aims were cost-saving or revenue generating and numbers were stated and yet after the project was authorized no one ever went back and assessed whether the project was a success or achieved any of it’s aims. Having an aim was enough to get the project started.  Claiming a 10% increase in sales or reduction in costs should be something you can measure so measure it.

Ironically being able to map a story to one of your stated goals for the product could be another way to filter unnecessary stories if the expected impact is not one of your product goals.

Summary

This is a very simple change to your story writing process – an extra little consideration that could have significant implications to the success of your product, the addition of a very valuable feedback loop on value delivered (rather than value expected)

As a …

I want to …

So that …

And I can verify this by …

 

Post Script:

I presented this notion to a Product Ownership Meetup and the response was amazing the conversation was full of so many great ideas, and examples of how some of the product owners are already putting this in to practice – not explicitly but have made usability and measuring usage a key part of their Product Ownership methodology. I love the conversations this group has each month, but this month I came away with so many new things to think about.

The highlight of which was Goldratt Users stories – which will be the subject of my next blog.

Tell me how you measure me and I’ll tell you how I will behave – Eli Goldratt

If I were a painter, I…

Recently I have spent a lot of time talking about story writing, but this week the focus has been on Acceptance Criteria and there have been a variety of comments that have caused me to reflect and have caused me some concern.
Some examples:

  • Unless stated otherwise a story is assumed to be happy path only. If it isn’t in the Acceptance Criteria we don’t have to do it.
    • Acceptance Criteria must contain all details including all edge cases that must be tested, anything not stated doesn’t have to be implemented.
  • If all acceptance criteria are met the PO/Customer MUST accept the story even if they are not satisfied.
  • If acceptance criteria change before the story is done, it should be completed as specified and a new story should be created for the changes. after all, late changes to Acceptance Criteria demoralize Developers.
  • If Acceptance Criteria are unclear we can choose what to do.
  • The Acceptance Criteria should never tell us how to do the job.

A metaphor

To respond to these questions I thought I would use a metaphor – A homeowner wanting to redecorate her house.

img_0374

The home owner hires a firm of decorators because they have a reputation for quality.

Her first request is that the painter decorate the sitting room,  She would like the walls painted blue, the ceiling an off-white, the windows also off-white but the paint needs to be wipe-able to avoid finger prints, and the doors and trim should match the window color.

A story is an expression of a need

Acceptance Criteria are the notes from the discussion to help clarify the need

In this case the homeowner has specified a number of acceptance criteria:

  • Walls – blue
  • Ceiling off-white
  • Windows off-white but wipe-able paint
  • Trim same as windows

The acceptance criteria are details provided by the Product owner that would help the workman understand how to complete the job to the satisfaction of the Product Owner – in this case the home owner.

The acceptance criteria are details provided by the Product owner that would help the workman understand how to complete the job to the satisfaction of the Product Owner

The job can and probably should be broken down into smaller tasks (stories). Having just the windows painted has value, just the ceiling does too, just the trim and potentially each wall independently may have value although that may be harder to qualify with the Product Owner, depending on their needs.

Detail in Acceptance Criteria

What you will note though is that the acceptance criteria did not state that the areas to be painted must be professionally prepared and sanded, that a primer coat be applied and two top coats are applied.  They do not state that the carpets should be protected and that spills should be cleaned up and that there should be no paint on the glass and no wall paint on the ceiling, no ceiling paint on the walls or that the paint should fully cover the walls and be an even drip free finish.  In fact the Acceptance criteria make no mention of how the work should be completed at all.

Who is the audience of your Acceptance Criteria?

The Acceptance Criteria come from a conversation with the Product Owner, but you may also want to take technical notes from conversations with those that will do the work. E.g. if the contractor has an apprentice, it may be necessary to spell out to him/her the technical details mentioned above, what seems common sense to you and the Product Owner may not be as obvious to someone less experienced. But notes on implementation are not acceptance criteria in the same sense. The product Owner should not be expected to understand all the technical details, what is acceptable is always determined by the Product Owner.

If quality is not specified in the acceptance criteria

But that means I only need to worry about the happy path right? So I need to do a good job with the specified work but I don’t need to care about drips or mess or if I am painting the walls I don’t need to care about the ceiling after all I am only doing the happy path. I can ignore walls that are hidden behind furniture, I can do the job quicker if I don’t put down dust sheets and we can worry about cleaning up the mess later right?

Put like that it sounds silly doesn’t it, but that  is what you are suggesting if you don’t consider anything other than the happy path, the minimal amount is done but you are leaving a mess to clean up later.  If this is clearly understood and acceptable to the customer or that you have split a story like this to get feedback sooner there may be an exception but if you think that you can declare a story ‘done’ with a mess left to clear up later it is a recipe for a disappointed customer or a misleading impression that you have achieved more than you really have. Giving a sense of artificial progress may lead to disappointment later.

img_0376

Acceptance Criteria are comprehensive and exhaustive

If all acceptance criteria are met the PO/homeowner MUST accept the story even if the homeowner is not satisfied.   There is some crown moulding between the walls and the ceiling, it was not mentioned in the Acceptance Criteria and the decorator has painted it the wall colour, but the Home owner wanted it the same as the ceiling and thought it was obvious so had not mentioned it.  The decorator insists the job is done because the Acceptance Criteria are met.

This is actually a very common issue, Acceptance Criteria are often not comprehensive and there are ambiguities and omissions, so how do you handle them?  Assuming you are paying for the work on a time an materials basis the customer is the one that suffers, they will have to pay for the rework(or extra work), but it is generally the workman that gets frustrated, they believed they did a good job and so it is frustrating to have to repeat good work.   But the customer is not satisfied, and that should be the only thing that matters. Your goal as a quality workman is to satisfy your customer.  The job is not done until they are satisfied (assuming the request is reasonable)

It might be that the customer is willing to accept the work as it stands and agree to the repainting later. But more likely they will want it fixed before you move on to the next piece of work.

What is not acceptable is to refuse to make the changes because they were not explicit in the Acceptance Criteria.  The story card is an invitation to a conversation, the Acceptance Criteria are reminders of what has been discussed. At no point do Acceptance Criteria or the card become a sealed contract.

img_0373

Changes during a story

The painter is half way through painting the window and the Customer comes to check on progress. She sees the paint colour of the windows and decides it is too white, too stark and clinical and would like the colour changed to an off-white.  She understands that the rework will cost and the new materials will cost and she accepts this, but she is unhappy with the colour.

The painter responds: “But the thing is the A/C were clear about the colour, the shade was specified, I have spent a long time getting through the first half of this job and it is only fair that I finish it according to the A/C”  Then we can have a separate job to repaint it a different colour.  Frankly my metrics will look bad if this job takes 50% longer, it will look like I am slow. Do you know how frustrating it is to do a job and have it rejected..”

I have heard frequently how when a PO changes A/C the story should be completed as it is and a new story created for the revisions.  But any continuation on work that is not wanted is waste. The notion of continuing to work on something that is unwanted, and even worse billing the customer for that time, is just wrong.  If metrics are driving bad behaviour then the metrics are failing. If someone is demoralized because a PO has clarified their needs then they need to evaluate their understanding of Agile.   A full half of the Manifesto is:

Customer collaboration over contract negotiation
Responding to change over following a plan

If you are unwilling to accept and adapt to changes in A/C even late into development, then you really need to go back to basics and ask if you genuinely support Agile software development.  I am sorry if that sounds harsh but the notion that keeping devs happy and metrics pretty is more important than delivering the right thing for your customer is very frustrating for me to hear.

 If Acceptance Criteria are unclear we can choose what to do.

The homeowner has been pretty vague in her choice of colours, blue and off-white. So presumably we can choose whatever we want?

Clearly that is unrealistic.  What might be more reasonable would be to talk to the homeowner and say “We prefer this brand and type of paint and these are the available colours within that range can you pick one”   The homeowner may say, No I want a very specific Blue only available in this brand that needs to be imported from New Zealand.  In which case you should explain that such a choice will increase the cost and duration of the job and adds to the risk – being an unfamiliar material, it may need more coats etc.  If the homeowner understands and accepts those risks then it should be done, but if the homeowner can be presented with a less costly and less risky alternative then they may be willing to modify their request.

Ambiguity should trigger conversation it should not be an excuse for making assumptions.

Ambiguity should trigger conversation it should not be an excuse for making assumptions. The goal is a finish that the homeowner is happy with, the specifics are likely open to negotiation. But whilst you may present alternatives it is for the homeowner to make the decision.

Anything not stated doesn’t have to be done

There are two types of things not stated in the Acceptance Criteria that apply here, how we do the job and some of the specifics.

How: The Acceptance Criteria doesn’t explicitly say that we need a primer coat, but we know that for a quality job it needs to be done. The Acceptance Criteria doesn’t tell us which brushes to use, or how to paint to get a good neat coverage. The contractor may need to specify those to an apprentice, but the homeowner should not need to specify the tools to be used or how to do the job (unless that forms a particular part of the need)  You as the contractor may need to spell out those details to your apprentice, consider those sub-tasks of the story.  Prepare and protect work area; Sand surface, apply primer, apply first coat,  apply 2nd coat, clean up area, clean brushes etc.  But these are NOT Acceptance Criteria to be supplied by the home owner. They are about how to do the job not about the job that needs to be done. Sometimes these do overlap but generally we try to keep the how out of the request.

There are also some specifics that may not be mentioned, for example there is a recess around the fireplace but only 4 walls were mentioned.  Can I ignore the other surfaces?  Likely not, details may have been assumed, if there is doubt clarify with the homeowner. There is something fixed to the wall which is too close to be painted behind, but if not painted it would look odd. The Acceptance Criteria shouldn’t need to specify that you need to remove it and paint behind to do a good job. But again if there is any doubt clarify it with the homeowner.  The Acceptance Criteria will rarely be comprehensive and conversation will be necessary.

Specifics: The Acceptance Criteria should not specify how the job must be done

Normally we try not to specify how the job must be done, but there will be exceptions to this. The homeowner may have a need to decorate using traditional materials, and so may specify that particular tools are used – e.g. No power tools, no lead based paint. No synthetic materials in paint or tools.  Sometimes those requirements are not necessarily clear to you. The Customer may insist on 2 primer coats and 3 top coats.  Two more coats than you feel is necessary for a quality finish, but they insist are how the job should be done.  In those instances you may make recommendations or suggestions, and try to understand why they feel this is a requirement. But ultimately it is for them to decide – so long as they understand the additional cost and risk.

But again there are exceptions to this, for example the customer my request that you skip a primer coat and only apply a single top coat, or insists that you use low quality materials to save on cost. If you believe that this will result in a substandard job, then again you should explain your reasons and try to find an alternative and if necessary refuse to accept the job if you feel that the result would reflect badly on you.  Ideally it should be clear from the start what the expectations around quality of service are, but it may be necessary to re-clarify periodically if there is doubt.

Summary

  • In story terms, the story should express the need, what your goal is. It is the start of the conversation(not the end) for assessing how to satisfy that need.
  • The Acceptance criteria are simply notes to help remember that conversation, they do not form a contract.
  • Acceptance Criteria should never be considered absolute, immutable or comprehensive.  Uncertainty should be clarified with the Product Owner early and often.
  • You should respond to changes in Acceptance Criteria quickly, the quicker you respond the less waste there will be.
  • Acceptance Criteria do not normally express how to do the job, although you may note how you intend to do the job to illustrate to the PO to help clarify any ambiguities.

What about Gherkin?

You can’t have a discussion on Acceptance Criteria without mentioning Gherkin, Gherkin is simply a tool for expressing Acceptance Criteria in a standardised and hopefully clearer format. However, Acceptance Criteria Gherkin and Test scenario Gherkin are very often confused, and the Gherkin constraint may result in the focus shifting from the conversation about the user’s need, to how to use Gherkin and the user’s need may get lost.

Story Acceptance Criteria is also not the place for identifying all of the test scenarios and edge cases, it is about understanding the need of the customer.

Another common issue with Gherkin is that the precise nature of the format and terms implies that the Acceptance Criteria are absolute and non-negotiable which is the exact opposite of how they should be considered.  If you start to fall in to the trap of seeing the Gherkin as a contract (rather than notes on a conversation) then stop using it until you get in the habit of regular conversation with the Product Owner.

By all means practice with Gherkin until you get better, but if you remember that it is about capturing notes from a conversation with the PO and most certainly NOT negotiating the terms of a contract then it is a useful tool.  But personally I prefer not to use Gherkin in the initial conversations about Acceptance Criteria simply because it is not a natural way of thinking for many POs so the format constrains thinking and conversation and the frequency in which it is treated as an explicit contract undermines it’s value.   Once you have discussed the Acceptance Criteria converting them to Gherkin may be beneficial.

However, I prefer to see Gherkin used for Acceptance Testing where those writing the tests are familiar with the syntax and where precision is more important.
If you take nothing else from this, please remember that Accceptance Criteria are notes on a conversation, they are not clauses in a contract.

Accceptance Criteria are notes on a conversation, they are not clauses in a contract.