Recently I have spent a lot of time talking about story writing, but this week the focus has been on Acceptance Criteria and there have been a variety of comments that have caused me to reflect and have caused me some concern.
Some examples:

• Unless stated otherwise a story is assumed to be happy path only. If it isn’t in the Acceptance Criteria we don’t have to do it.
  • Acceptance Criteria must contain all details including all edge cases that must be tested, anything not stated doesn’t have to be implemented.
• If all acceptance criteria are met the PO/Customer MUST accept the story even if they are not satisfied.
• If acceptance criteria change before the story is done, it should be completed as specified and a new story should be created for the changes. after all, late changes to Acceptance Criteria demoralize Developers.
• If Acceptance Criteria are unclear we can choose what to do.
• The Acceptance Criteria should never tell us how to do the job.

A metaphor

To respond to these questions I thought I would use a metaphor – A homeowner wanting to redecorate her house.

The home owner hires a firm of decorators because they have a reputation for quality.

Her first request is that the painter decorate the sitting room,  She would like the walls painted blue, the ceiling an off-white, the windows also off-white but the paint needs to be wipe-able to avoid finger prints, and the doors and trim should match the window color.

A story is an expression of a need

Acceptance Criteria are the notes from the discussion to help clarify the need

In this case the homeowner has specified a number of acceptance criteria:

• Walls – blue
• Ceiling off-white
• Windows off-white but wipe-able paint
• Trim same as windows

The acceptance criteria are details provided by the Product owner that would help the workman understand how to complete the job to the satisfaction of the Product Owner – in this case the home owner.

The acceptance criteria are details provided by the Product owner that would help the workman understand how to complete the job to the satisfaction of the Product Owner

The job can and probably should be broken down into smaller tasks (stories). Having just the windows painted has value, just the ceiling does too, just the trim and potentially each wall independently may have value although that may be harder to qualify with the Product Owner, depending on their needs.

Detail in Acceptance Criteria

What you will note though is that the acceptance criteria did not state that the areas to be painted must be professionally prepared and sanded, that a primer coat be applied and two top coats are applied.  They do not state that the carpets should be protected and that spills should be cleaned up and that there should be no paint on the glass and no wall paint on the ceiling, no ceiling paint on the walls or that the paint should fully cover the walls and be an even drip free finish.  In fact the Acceptance criteria make no mention of how the work should be completed at all.

Who is the audience of your Acceptance Criteria?

The Acceptance Criteria come from a conversation with the Product Owner, but you may also want to take technical notes from conversations with those that will do the work. E.g. if the contractor has an apprentice, it may be necessary to spell out to him/her the technical details mentioned above, what seems common sense to you and the Product Owner may not be as obvious to someone less experienced. But notes on implementation are not acceptance criteria in the same sense. The product Owner should not be expected to understand all the technical details, what is acceptable is always determined by the Product Owner.

If quality is not specified in the acceptance criteria

But that means I only need to worry about the happy path right? So I need to do a good job with the specified work but I don’t need to care about drips or mess or if I am painting the walls I don’t need to care about the ceiling after all I am only doing the happy path. I can ignore walls that are hidden behind furniture, I can do the job quicker if I don’t put down dust sheets and we can worry about cleaning up the mess later right?

Put like that it sounds silly doesn’t it, but that  is what you are suggesting if you don’t consider anything other than the happy path, the minimal amount is done but you are leaving a mess to clean up later.  If this is clearly understood and acceptable to the customer or that you have split a story like this to get feedback sooner there may be an exception but if you think that you can declare a story ‘done’ with a mess left to clear up later it is a recipe for a disappointed customer or a misleading impression that you have achieved more than you really have. Giving a sense of artificial progress may lead to disappointment later.

Acceptance Criteria are comprehensive and exhaustive

If all acceptance criteria are met the PO/homeowner MUST accept the story even if the homeowner is not satisfied.   There is some crown moulding between the walls and the ceiling, it was not mentioned in the Acceptance Criteria and the decorator has painted it the wall colour, but the Home owner wanted it the same as the ceiling and thought it was obvious so had not mentioned it.  The decorator insists the job is done because the Acceptance Criteria are met.

This is actually a very common issue, Acceptance Criteria are often not comprehensive and there are ambiguities and omissions, so how do you handle them?  Assuming you are paying for the work on a time an materials basis the customer is the one that suffers, they will have to pay for the rework(or extra work), but it is generally the workman that gets frustrated, they believed they did a good job and so it is frustrating to have to repeat good work.   But the customer is not satisfied, and that should be the only thing that matters. Your goal as a quality workman is to satisfy your customer.  The job is not done until they are satisfied (assuming the request is reasonable)

It might be that the customer is willing to accept the work as it stands and agree to the repainting later. But more likely they will want it fixed before you move on to the next piece of work.

What is not acceptable is to refuse to make the changes because they were not explicit in the Acceptance Criteria.  The story card is an invitation to a conversation, the Acceptance Criteria are reminders of what has been discussed. At no point do Acceptance Criteria or the card become a sealed contract.

Changes during a story

The painter is half way through painting the window and the Customer comes to check on progress. She sees the paint colour of the windows and decides it is too white, too stark and clinical and would like the colour changed to an off-white.  She understands that the rework will cost and the new materials will cost and she accepts this, but she is unhappy with the colour.

The painter responds: “But the thing is the A/C were clear about the colour, the shade was specified, I have spent a long time getting through the first half of this job and it is only fair that I finish it according to the A/C”  Then we can have a separate job to repaint it a different colour.  Frankly my metrics will look bad if this job takes 50% longer, it will look like I am slow. Do you know how frustrating it is to do a job and have it rejected..”

I have heard frequently how when a PO changes A/C the story should be completed as it is and a new story created for the revisions.  But any continuation on work that is not wanted is waste. The notion of continuing to work on something that is unwanted, and even worse billing the customer for that time, is just wrong.  If metrics are driving bad behaviour then the metrics are failing. If someone is demoralized because a PO has clarified their needs then they need to evaluate their understanding of Agile.   A full half of the Manifesto is:

Customer collaboration over contract negotiation
Responding to change over following a plan

If you are unwilling to accept and adapt to changes in A/C even late into development, then you really need to go back to basics and ask if you genuinely support Agile software development.  I am sorry if that sounds harsh but the notion that keeping devs happy and metrics pretty is more important than delivering the right thing for your customer is very frustrating for me to hear.

 If Acceptance Criteria are unclear we can choose what to do.

The homeowner has been pretty vague in her choice of colours, blue and off-white. So presumably we can choose whatever we want?

Clearly that is unrealistic.  What might be more reasonable would be to talk to the homeowner and say “We prefer this brand and type of paint and these are the available colours within that range can you pick one”   The homeowner may say, No I want a very specific Blue only available in this brand that needs to be imported from New Zealand.  In which case you should explain that such a choice will increase the cost and duration of the job and adds to the risk – being an unfamiliar material, it may need more coats etc.  If the homeowner understands and accepts those risks then it should be done, but if the homeowner can be presented with a less costly and less risky alternative then they may be willing to modify their request.

Ambiguity should trigger conversation it should not be an excuse for making assumptions.

Ambiguity should trigger conversation it should not be an excuse for making assumptions. The goal is a finish that the homeowner is happy with, the specifics are likely open to negotiation. But whilst you may present alternatives it is for the homeowner to make the decision.

Anything not stated doesn’t have to be done

There are two types of things not stated in the Acceptance Criteria that apply here, how we do the job and some of the specifics.

How: The Acceptance Criteria doesn’t explicitly say that we need a primer coat, but we know that for a quality job it needs to be done. The Acceptance Criteria doesn’t tell us which brushes to use, or how to paint to get a good neat coverage. The contractor may need to specify those to an apprentice, but the homeowner should not need to specify the tools to be used or how to do the job (unless that forms a particular part of the need)  You as the contractor may need to spell out those details to your apprentice, consider those sub-tasks of the story.  Prepare and protect work area; Sand surface, apply primer, apply first coat,  apply 2nd coat, clean up area, clean brushes etc.  But these are NOT Acceptance Criteria to be supplied by the home owner. They are about how to do the job not about the job that needs to be done. Sometimes these do overlap but generally we try to keep the how out of the request.

There are also some specifics that may not be mentioned, for example there is a recess around the fireplace but only 4 walls were mentioned.  Can I ignore the other surfaces?  Likely not, details may have been assumed, if there is doubt clarify with the homeowner. There is something fixed to the wall which is too close to be painted behind, but if not painted it would look odd. The Acceptance Criteria shouldn’t need to specify that you need to remove it and paint behind to do a good job. But again if there is any doubt clarify it with the homeowner.  The Acceptance Criteria will rarely be comprehensive and conversation will be necessary.

Specifics: The Acceptance Criteria should not specify how the job must be done

Normally we try not to specify how the job must be done, but there will be exceptions to this. The homeowner may have a need to decorate using traditional materials, and so may specify that particular tools are used – e.g. No power tools, no lead based paint. No synthetic materials in paint or tools.  Sometimes those requirements are not necessarily clear to you. The Customer may insist on 2 primer coats and 3 top coats.  Two more coats than you feel is necessary for a quality finish, but they insist are how the job should be done.  In those instances you may make recommendations or suggestions, and try to understand why they feel this is a requirement. But ultimately it is for them to decide – so long as they understand the additional cost and risk.

But again there are exceptions to this, for example the customer my request that you skip a primer coat and only apply a single top coat, or insists that you use low quality materials to save on cost. If you believe that this will result in a substandard job, then again you should explain your reasons and try to find an alternative and if necessary refuse to accept the job if you feel that the result would reflect badly on you.  Ideally it should be clear from the start what the expectations around quality of service are, but it may be necessary to re-clarify periodically if there is doubt.

Summary

• In story terms, the story should express the need, what your goal is. It is the start of the conversation(not the end) for assessing how to satisfy that need.
• The Acceptance criteria are simply notes to help remember that conversation, they do not form a contract.
• Acceptance Criteria should never be considered absolute, immutable or comprehensive.  Uncertainty should be clarified with the Product Owner early and often.
• You should respond to changes in Acceptance Criteria quickly, the quicker you respond the less waste there will be.
• Acceptance Criteria do not normally express how to do the job, although you may note how you intend to do the job to illustrate to the PO to help clarify any ambiguities.

What about Gherkin?

You can’t have a discussion on Acceptance Criteria without mentioning Gherkin, Gherkin is simply a tool for expressing Acceptance Criteria in a standardised and hopefully clearer format. However, Acceptance Criteria Gherkin and Test scenario Gherkin are very often confused, and the Gherkin constraint may result in the focus shifting from the conversation about the user’s need, to how to use Gherkin and the user’s need may get lost.

Story Acceptance Criteria is also not the place for identifying all of the test scenarios and edge cases, it is about understanding the need of the customer.

Another common issue with Gherkin is that the precise nature of the format and terms implies that the Acceptance Criteria are absolute and non-negotiable which is the exact opposite of how they should be considered.  If you start to fall in to the trap of seeing the Gherkin as a contract (rather than notes on a conversation) then stop using it until you get in the habit of regular conversation with the Product Owner.

By all means practice with Gherkin until you get better, but if you remember that it is about capturing notes from a conversation with the PO and most certainly NOT negotiating the terms of a contract then it is a useful tool.  But personally I prefer not to use Gherkin in the initial conversations about Acceptance Criteria simply because it is not a natural way of thinking for many POs so the format constrains thinking and conversation and the frequency in which it is treated as an explicit contract undermines it’s value.   Once you have discussed the Acceptance Criteria converting them to Gherkin may be beneficial.

However, I prefer to see Gherkin used for Acceptance Testing where those writing the tests are familiar with the syntax and where precision is more important.
If you take nothing else from this, please remember that Accceptance Criteria are notes on a conversation, they are not clauses in a contract.

Accceptance Criteria are notes on a conversation, they are not clauses in a contract.