Agile Transition Frustrations

On the back of my good day last week, I have run in to a somewhat frustrating day today.

I have taken it upon myself to challenge the adoption of Agile in the organisation, it has been over 18 months since they adopted Scrum, and there are currently 7 Scrum teams each with a Product owner and there are 3 Scrum Masters and an Agile Coach/Scrum Master. Sounds great! except that not one of those 11 roles have a job title to match.

In every case someone is ‘acting …’ – they are seconded from another role. In most organisations that wouldn’t be a problem, you could be pretty flexible, temporarily at least. But in the Civil Service a job description and responsibilities are a big deal, every 6 months there is an appraisal of your work where you are measured on objectives that are constructed based on your job description and designated roles and responsibilities of that job, so unfortunately those in the roles of PO and SM are struggling to meet any of their objectives, not to mention a lack of support and backup as those more senior in the organisation are not familiar enough to formally provide it. So it is being provided informally by those that are enthusiastic about agile.

So I decided that the time had come and I wanted to press it with HR, I want to create formal roles to reflect the work being done, to allow those interested to be able to apply and those doing the roles to be recognised and rewarded. And then finally I wanted to ensure there was a support network in place. (I could be biting off more than I could chew)

But today I met with two people – one responsible for organising the support network and one from HR, the lady from HR was fantastic, all you could ask for from an HR rep, she seemed genuinely concerned about the people, about ensuring they got the support and had career options and had clarification of their roles and responsibilities. But the support network guy… Now I don’t know if he misunderstood my request, or whether he was irritated by agile or what the situation was, but I felt I was on the back foot, he suggested that ‘some agile advocates’ were so zealous in their agile application that they created a rigid process worse than the ‘traditional’ approach. He seemed to suggest that as agile had no process that formal support was unnecessary, and he then spoke for a while about how the process was not understood, followed or even liked by the organisation, and I must confess at this point I wasn’t sure if he was talking about agile or the professional support network he was responsible for.

I felt very deflated, I really thought I would be pushing on an open door, this was someone (me) wanting to utilise their expertise and help make the system work by making it relevant to those doing the job and I was being criticised for not being ‘agile’ I found myself questioning what I was doing. Was I being un-agile? Was I imposing structure and regulation?

In the end the HR rep concluded very diplomatically that we were not ready for him yet and we would re-convene again without him – phew. But it still made me stop and think, was I doing the right thing?

So I have spent this evening thinking a lot about it, and on reflection I put it down to a belief that agile means no formal process. This is clearly wrong on a number of levels, but in the Civil Service it would be an impossible situation, formal process is so ingrained that you need a very formal process to not have a process.

The agile principle is that we favour “Individuals and interactions” over “processes and tools”. But that is not the same as saying we don’t believe in processes. I believe that the people are crucial and that providing structure, support, training and opportunities is very much the agile way, and if I need a process to do that so be it. What is un-agile is to not provide these vital people support because a process gets in the way – which is the current situation.

Sometimes being challenged in your thinking really helps clarify understanding, and whilst I left the meeting frustrated I am now very pleased I had that conversation, I feel more certain than ever that it is both right and necessary for me to challenge this situation, and keep the focus on the people.

Agile in action

For the most part it was a pretty normal sprint, the team read and refined the stories, asked questions and clarified the requirements. One of the stories referred to a text editor where text would be cut and pasted. There were questions about formatting and content and the PO was sure that maintaining formatting was not required and that the text would be standard text.

The team worked through the story and completed all acceptance criteria and everyone was happy, except one of the development team who felt an urge to work closely with the users and get actual examples of the text they used and how they worked. It turned out that one of the source documents was non-standard in the sense that the text editor selected was not able to cope with the source data, when pasted critical characters were lost. We investigated further, and it also turned out that formatting and layout were also very important to the users in some situations, something not fully understood at analysis.

The team, the PO and the BA worked together, they all got involved and came up with a solution, and we have written a new story to be included in the next sprint, which means a slight delay in functionality being released to the users but as an observer to this incident I was very proud of how agile the team behaved and how smoothly we adjusted to what was in effect a significant blow to the initial design.

That may not sound a big-deal but if you consider that had we not challenged the requirements, had we not had open and effective communication with the Product Owner and the users, had we implemented requirements as written we would have been able to sign-off the work and likely get all the way to release and delivery to the users – weeks if not months later before this problem was identified, by which time the cost to resolve would have been considerable and the disruption to all would have been significant.

This is a great example of where agile can be a great tool. A situation where the team made customer collaboration a priority over contract negotiation, and responded to a change rather than adhering to a plan. Sometimes I worry that the benefits of agile get lost, but on days like today I feel like agile really has made a difference. I feel immensely pleased with the team.

Velocity

I have seen a number of discussions recently about velocity, and generally the harm it can cause when used incorrectly. Most seem to advocate not using it to avoid mis-using it but whilst it is a blunt instrument it is just information like any other metric and for a process that relies on empirical evidence to adapt and evolve it seems odd to ignore a very useful metric, so instead I will try to explain how, when and why velocity could be used to add value.

My car has a trip computer and among its varied metrics it offers an ‘instant fuel economy’ feature. Why? I don’t know, but to my mind it is a pretty useless feature, but it is there and at any time I can take a spot check and it will tell me how much fuel I am using in mpg. When racing up the hill to the Air Balloon roundabout foot down it drops to around 20 mpg, when coasting down with my foot off the pedal I get 99 mpg.

If I were to take a single reading half way up the hill and claim that my car does 20 mpg I would get a totally distorted view of the velocity with which I consume fuel, equally if I took a single reading coming down at 99 mpg. I’m sure most of you would think I was nuts to even consider taking a single reading as an absolute indicator of future velocity. But somehow with ‘sprint velocity’ people think that is okay.

Let me stretch the analogy further. I know that a single measurement is a little crazy to base things on, so let’s take an average over the last 100 miles. That is a pretty solid and reliable metric, most of us would consider that to be a reasonable predictor. But I have spent the last few weeks only driving around town, stop start, traffic jams, short journeys and so on. But now I have a long trip, mostly motorway, will my average fuel consumption velocity be a good indicator of the next 100 miles? No it won’t, common sense tells me before I even look that my velocity will be different under different conditions. A velocity only has meaning if the measured data is representative of the future conditions. Equally I tend to drive with a heavy foot, but I am quite sure if my father were to drive the same journey in the same car the consumption velocity would be quite different. The same is true if I drove a different car, the velocity would be different even if my driving conditions were the same.

Velocity tells you only one thing – what was consumed in this car in these conditions. It offers no view of the future, or of alternative conditions.

But I am free to interpret, I can predict, I can guess, I can extrapolate all I like, but it is not the velocity that comes up with these figures – it is me. If I make a prediction and it is wrong, I cannot blame the velocity, the velocity is a measurement of what was, not of what will be.

So how is it useful then? The most obvious is to say that if I believe that my typical journey over the next 100 miles will be similar to the last 100 miles, similar driver, similar journeys, similar car then it is not unreasonable to expect similar results. It can be a predictor.

Or let’s say that I am a little bit eccentric and I decide that I want to increase my average MPG and I spend the next 100 miles on similar journeys in a similar car, but I try very hard to improve my driving to be more efficient, in that case it can be a blunt measure of improvement in my driving if the average goes up, I can measure improvement. I would caution myself very carefully on this particular measurement because it could easily be a change in the road conditions not the driver that is the cause for a change but even so it is a measure – the interpretation of meaning is mine.

Or if nothing changes that I am aware of and the average goes noticeably down or up, it might be an indication of a problem it might be an indicator there is a problem with the car, or my driving, or even just different fuel.

So now I understand all about the velocity of fuel consumption my current average is 50mpg. How much fuel will I consume on my next journey? There is no way to tell, to even make a reasonable estimate I’d need to know an approximate distance, I’d need to know the terrain and even with all that there are still unknowns, I may hit a delay in traffic, or a diversion, I may get asked to call in and pick up milk, I may even breakdown. And if I could predict all of that and gave you an estimate, would you trust it to any degree of accuracy? I wouldn’t, I might trust the estimate to a degree, but I wouldn’t play to fill up on empty on such an estimate I’d almost certainly plan a significant buffer to ensure I didn’t run out of fuel.

I hope most of you would agree with my views on fuel economy, there are simply too many unknowns to make accurate predictions.

And yet, I reset my trip computer fairly regularly every 1000+ miles or so, and on average over that 1000 miles my economy is in the region of 48 ish, it very rarely fluctuates significantly from this. Over a very long timeframe I can get a very accurate estimate, that is consistent 1000 miles after 1000 miles. That still tells me nothing about the next 20 miles or even 100 but long term I can be consistent.

Another interesting observation is that to get this measurement I am using an extremely accurate tool to measure, I am in a known unvarying state – same car, same driver, similar journeys. And yet with all this certainty I still don’t trust my predictions to any degree of accuracy I buffer I plan contingency. But if I stop talking about cars for a moment and I talk about software, more specifically a team developing complex software with a great many unknowns, where the team members can take leave, be sick, have family troubles, they can leave the team, or new people join the team, they can learn new skills, use new tools. But we are expected to estimate, often to a very specific dates, large quantities of unknown and often very complex work. What is more odd is that those estimates are often taken as commitments.

If a rational person wouldn’t trust a reliable and regular machine like a car to give specific accurate predictions with fairly well known parameters, why do similar rational people expect such accuracy from a domain far more variable, far more complex and far more unpredictable?

My rather long winded discussion on fuel economy is really summed up in one sentence.

There is only one estimate on how long a software project will take that I would trust and that is one given the day after the project is completed.

Otherwise use projection tools like velocity in context, if the future conditions are likely to be unchanged then velocity can be a useful tool, but the longer the average the better. And if your circumstances change: the team, the type of work the domain, the tools your current velocity is meaningless until you have enough information to create a new measurement. And think of your car, what value is an snapshot measurement without context.

Coaching is the easy option?

I have run into a strange dynamic in the agile transition today. As part of the transition to an ‘Agile’ organisation I have been pushing HR to create and formally recognise new roles, specifically Scrum Master and Product Owner, up until now it has been by unofficial secondment. Part of me feels pleased that we are getting traction over the move and formal recognition is a major step forward in the Civil Service.

But now I have hit a rather significant hurdle, how grades are classified, the Civil Service – or at least this particular location – classifies and grades roles based on accountability, responsibility and number of reporting staff. So a servant leader that coaches and supports and has no direct reports comes out classified as a very junior admin assistant, the people doing the role it turns out are two or three grades higher than ‘proposed’ which is clearly a serious mismatch.

The advice has been to reword the job description to add responsibilities and authority and accountability, but that feels wrong to me. I am trying to educate and apply Agile principles to the organisation and this feels like a very unpleasant compromise.

It seems that somehow coaching is classified as an easy skill and the lack of apparent direct accountability and responsibility does not sit well with the pigeon holes that the roles must fall in to.

I have tried explaining that coaching is actually far harder than managing, using coaching skills to help a team discover for themselves what to do next is a very difficult skill, but when done right what they learn gives a far deeper understanding and more likely to be maintained than any amount of compelling, controlling or micro managing task assignment.

Those involved seem to understand and appreciate what I am saying but we are challenging the system, it will be an uphill battle. This is new ground and a transition to Agile is rarely easy and it seems the bigger and more established the processes the harder it is to change.

But I wonder why coaching is seen as an easy option. It rarely is. A coach actually has to be pretty mean, they have to be tough. We have to turn a mirror on people and often show them some unpleasant aspects of themselves, often they are unaware, which can be even tougher to deal with. I sometimes have to sit back and watch someone fail knowing I could have prevented it – not because I want to see them fail, but because failure can teach far more than correcting them.

I sometimes get pressure from management to assign tasks, or to get the expert to do a job because it is quicker, or to challenge certain behaviours directly. I resist and this can and often does reflect badly on me, but I resist because I believe providing an environment where the team learns for themselves is far more valuable than any short-term gain resulting from taking away their independence.

I see my job as protecting the team and creating an environment to grow and learn. I do this by asking a questions to provoke ideas, I challenge where I see potential, sometimes I stray into leading questions if I think it helps but I try very hard not to provide answers, and if I slip into leading rather than coaching I hope I get someone turning a mirror on me and coaching me how to be better.

I love my job, but I don’t coach because it is easy, I coach because someone needs to be tough and push you to do better.

There is a quote in Lyssa Adkins’ book – Coaching Agile teams, which I will paraphrase a little, it says so much.

A friend likes you the way you are, a coach respects you too-much to let you stay that way.

Anyone that thinks a coach or a Scrum Master is the easy option should try walking in those shoes for a while, it may put a whole new perspective on things.

Back to Scrumdamentals

It has been a pretty difficult few weeks for me, busy, busy, busy and frankly more management than coaching. Cue air violins!

In the division I am working in there are three Scrum Teams, up until now I have been responsible for two and the third has tried a number of ways to accommodate the lack of a Scrum Master, one member of the team took on the role for a while on top of their other duties and a Project Manager took it on for a while, again on top of other duties. But these part-time roles have been unsuccessful and I was asked to take on a third team.

I was very reluctant to dilute my other responsibilities and expressed this to the department head, and so we agreed it will be temporary and I’ll be training a new Scrum Master for that team to take over in a few months. Which is great but also means I now have responsibility for three teams and training of a new SM, and we will be losing one extremely valuable member from a team. As I say Cue violins, but my concern is not just workload, it is the teams themselves, one of the teams already feels I don’t spend enough time with them and spreading me thinner wont help with that. So that is problem 1, 2 and 3 but my other and now more serious problem is that the final team is in a state of flux. The Product Owner has moved on to a new role and it has been difficult to find a replacement.

A Product Owner is a difficult role, and seemingly undesirable. The product is essentially a suite of very closely integrated sub-products with what is intended to be a single front end, the product cannot easily be split because of how closely coupled it is, but the system is so complex there are at least three Business Analysts and an Architect who when combined cover the understanding of the system. The chief architect is ultimately the product owner, he makes the call on the roadmap and the priority of content into the system, but doesn’t have visibility of the fine detail at an individual story level, that is down to the BAs who are doing it on top of their other responsibilities. So prioritisation is at Epic level, with the BAs prioritising the smaller individual stories.

And to cap it all we have hit a spate of recruitment and retention issues. We had decided that the volume of work was such that we would expand and have a second team on this product, but over the last few months two contractors have left, primarily for family reasons (and higher rates) and a very experienced member of the team has got a promotion so he too has left, and as I say another team member has volunteered to be trained as a ScrumMaster, all told a major impact on one relatively small team. So I have been very very busy recruiting (30+ interviews over the last 3 weeks). I have spent the last couple of weeks almost exclusively interviewing and screening CVs. Thankfully offers have been made and I’m hopeful we will have a new team up and running in a few weeks.

But it is essentially a brand new team; a new Product Owner Dynamic; and a major new piece of work. A new Product Owner, and 3 new proxy product owners(Business analysts) a new Scrum team comprised of a former tester and 4 new former developers (I say former as once in a Scrum team I hope they will all see themselves as a cross-functional team). All this got me thinking. At first I was anxious about the disruption, I am also anxious that the Product Owner representatives are not dedicated to what is a major product, but after some thought I realise that disruption is the wrong word, the team can’t really be disrupted, there isn’t enough left of the old team to call it a disruption.

What we have is a fantastic opportunity, we can create and build a new team from scratch. It is not often you get to join a team at the beginning and are able to shape it to be the team you want to be a part of, usually it is a case of fitting in. But the new team members are joining a high profile new product together and get to shape it as we collectively see fit.

All of the team have had some exposure to some form of agile development they seem strong developers and eager to use Agile, so there will be a degree of Scrum training and coaching but for me the crucial task is to build a team first, get them working together, learning strengths and weaknesses and growing into a team. I still have a lot of worries, especially about growing a new team whilst also supporting the existing teams, I cannot give the teams as much attention as I would like, but I am actually pretty excited about the prospect of helping build a new agile team.

I have been trying to think of a good place to start, no doubt the level of Scrum training will vary so I thought that I might try a combined Scrum Training and team building exercise. My train of thought led me to http://www.lego4scrum.com/ it is my hope that when they get on board, I will set up a session for them and mix it up with the other existing Scrum Teams. I plan to write up a review after the event which I will post here.