Typical interview styles

The interview itself.

What I have observed is that interviewing is generally composed of one or more of a very small number of techniques:

  1.  Free-form, gut-feel unstructured or semi-structured chat and questions between candidate and hiring manager.

  2.  More formal pre-defined questions structured and consistently asked to all candidates.

  3.  Some variety of standardised test, essentially IQ based.

  4.  Questions posed by a technical expert with a desire to highlight his superiority rather than assess your capability.

  5.  A technical test or series of technical questions intended to be pragmatic and fair.

  6.  Aggressive panel questioning

  7.  Psychometric testing: verbal/numeric/diagrammatic reasoning or personality tests.

  8.  Presentation by interviewer on why the candidate should work for you.

There are studies and statistics that rank the effectiveness of the different techniques for selecting good candidates, I won’t pour them out here – I am not an expert and I’d just embarrass myself. But in my experience 1 is the most common, often combined with 4 or 5. But logic states that 4 is a waste of time and puts off candidates, and studies show that 1 and 5 are actually very, very poor indicators of ability and capability for the role and do not result in long-term success.

So what works? 

2, 3 and 7 are by far a better method of assessing candidates, followed by an independent panel-decision based on the documented evidence taken from the interviews – the interviewer should not be allowed to make the decision independently as they display bias (subconscious or otherwise).

But 2, 3 and 7 are hard, it requires yet more work on what is already a tough and time-consuming process, so very rarely gets done.

The reciprocal nature of an interview (8) is so often overlooked, if you want the best people and you want to get the best out of them, then not only do you need to decide if they are right for you, you need to convince them that your business/team is right for them. You should be putting as much effort in to impressing candidates as they put into impressing you. You are competing for them in a buoyant market.

recruitment2

Doing it right.

As an interviewer, my best experience has been for a company that did a combination of 2, 3, 7 and 5. We put a lot of effort into interviewing, but the majority of people still failed the combination of tests, especially the technical test. The test felt ‘easy’ to us and some candidates did well, but the results didn’t match the apparent skills of candidates, in hindsight I think it confused the process. The problem is that technical tests are not effective at assessing ability during an interview, there are simply so many other factors at play.

Relying on structured questions, IQ tests and psychometric tests may feel clinical and impersonal, but very likely the best way to find the right candidate. But ego plays a role and when hiring for a team, we are only human and so many hiring managers want to rely on their own instinct, even if evidence demonstrates this is unreliable, so it is hardly a surprise that many hiring managers favour their instinct over a structured process.

What makes a good interview?

My experience of recruitment is largely anecdotal, I am not a professional interviewer or HR expert. My limited experience of interviewing has been as a candidate, a hiring manager, or sometimes even as a subject expert. I’d estimate that over my career my experience is approximately 8:1 in favour of being an interviewer over being an interviewee, but I have been on both sides of the table often enough to know how it feels from either perspective. 

Over the last 5 or so years the amount of effort I have put into recruitment has been considerable, especially considering it is not really meant to be a significant part of my role. I’d estimate that I have hired an average of one person ever 2 months or so.   I even like to think I am pretty good at it,  although as you will see I am probably mistaken in that opinion.

What is striking though is how much time this takes, very anecdotally I’d estimate that depending on the quality of the CVs that I receive, on average only 5% will make it through the screening and interviewing to getting an offer(probably less). But the 95% that get rejected still consume a lot of time, and of the 5% selected we don’t always get it right, and I’m pretty sure quite a few good candidates get missed. The process is far from ideal.

recruitment

But what makes for a good interview?

As a candidate I have had interviews lasting less than 15 minutes (where I was offered) and interviews spanning 3 days, for a graduate recruitment programme, and just about everything in between. I have had good experiences and bad.

Some, in fact most interviewers seem to forget that the candidate is also interviewing them, and that the candidate is deciding if they want to work with you in the same way you are assessing them – it is not a one way decision by any stretch. Such is the lack of understanding and sheer arrogance of some that I have been asked extremely obscure technical questions that provide no clear value, I have been challenged(insulted) to gauge my response, I have had panel of interviewers quick-firing questions and had interviewers who were clearly reading my CV for the first time during the interview.  Why would any good candidate have any desire to work for you after an interview like that?

I haven’t kept accurate records, and I appreciate this is anecdotal but I’d estimate I have turned down close to three job offers for every one I have accepted over my career. Now I am well aware that I am fortunate to have in-demand skills at a time when the market is booming, I have also had a lot of rejection.  Similarly, as a hiring manager I’ve had too many candidates turn down offers, or more frustrating get a better offer before we could complete the hiring process.

The hiring process is so expensive and so important I am surprised at how often it is treated in a cavalier manner.   Why oh why would you be so arrogant to treat candidates with anything less than the respect you would expect them to show you?

To be the best, hire the best… for you.

I have recently been reflecting on the impact of our choices when hiring new staff. Over the last few years hiring has been a major activity for me, and so I am particularly interested in what the impact of these hiring decisions has on the team and the organisation.

I have a number of experiences that I feel are relevant and two particular opinions I have recently heard that I would like to discuss.

The first was from a blog post: http://zachholman.com/posts/0x-engineers/   in this the author is critical of everyone chasing after the type ‘A’ engineer, it makes an interesting read and refers to a popular belief that the good software engineers are worth 10x as much as the ‘average’ engineer.  I don’t intend to get into the basis or accuracy of the claims, but I am satisfied that it is a generally well accepted opinion that there is a significant difference in ability, and value of output between those deemed average and the top end of the scale, I have heard figures ranging from 10x as valuable to 1000x as valuable from quite well respected authors.

But ultimately the author compares software engineers to restaurants and concludes that most of the time he would be content with an average restaurant as we can’t always all eat in the best restaurants.  This is the point where I try hard not to be dismissive. The analogy is suggesting that most of us are content with ‘Good’ food and don’t need ‘Great food’ but the reality is that we are not only comparing quality, we are comparing relative quantity AND quantity, if we use his 10x value estimate then what we are comparing is a 1/10th of a ‘Good’ meal to a whole ‘Great meal’  We’d need to eat (and pay for) 10 ‘Good’ meals to get the same quantity/quality (value) of food.

He does have some good points about a good team being better than a star player and I’ll come back to that but I for one am not content with a tenth of a meal.

By contrast I looked at Work Rules! by Laszlo Bock (SVP People Operations at Google)

Lessons from WORK RULES! include:

* Take away managers’ power over employees
* Only hire people who are smarter than you are, no matter how long it takes to find them
* Pay unfairly (it’s more fair!)
* Default to open: be transparent, and welcome feedback
* If you’re comfortable with the amount of freedom you’ve given your employees, you haven’t gone far enough

Laszlo advocates hiring only the best, by hiring the best you get the best output, you certainly have to pay more but you don’t have to pay 10x the salary to get a 10x software engineer, it makes financial and practical sense, a small team of high achievers must be amazing and they clearly are for Google. But…

Here is where the real word kicks in.  Most of us don’t work for Google or Apple, we will not attract the best of the best, most organisations are unable to employ an entire team of 10x engineers, many of us are lucky to hire a few high achievers, when location, salary, benefits, and image are factored in, we can be selective and choose the best of what is available to us, but a good engineer is hard to find.

So I am saying that I don’t want a team where the primary qualification is a pulse, and I am highly unlikely to be able to hire (and retain) an organisation full of 10x engineers.  My world is the grey area somewhere between.

This brings me to what has been an inevitable consequence of the real world – a mixed team.  I didn’t like the restaurant analogy so I will go with cars.

cars

Lets say I have a team of five developers, as a very general rule a team will travel at the pace of the slowest member, and whilst you may be able to improve the speed of the slowest a little, the high performers are stuck at the slowest speed and may well be getting bored and frustrated.  It is no good one or two members zooming-off, knowledge doesn’t get shared resentment grows and very quickly you are at a standstill with a dysfunctional team. An effective team moves at one pace – the pace of the team, and as the original blogger wrote, a team that works well together can actually be far more effective than a single star player.

And this is where things get more messy, as either a Scrum Master or a Department Manager the goal is the long-term development of the team, I value all the team and I’d like to see everyone have the opportunity to develop and get better, a single star-player held back is bad, I don’t want to see that, but equally I am aware that an effective team can be amazing, and then again a dysfunctional team is a disaster. I want to grow and build a team that are greater than the sum of their parts not a group of individuals moving at the pace of the slowest.

So I find myself both agreeing and disagreeing with Laszlo, yes I want a group of the best engineers, but that isn’t going to happen, I don’t live in silicon valley and I don’t have that budget, and even if I did then I’d still prefer a great team of engineers. I believe that a team working well together can achieve far more and far better software than even ’10x’ individuals.

I aim to get that by hiring for the team. I will still aim to hire a team where they are smarter than me if I can but my primary goal is to find someone good that will enhance the existing team, that means I value the ability to communicate over their technical depth, I value their willingness to learn and to question over their certainty of their own knowledge, I value a desire to give customers what they want over a desire for a perfectly engineered solution. I suppose I am saying I value the team over individuals – even star players.

But I have a dark side too, anyone that is holding the team back needs to go, whether they are disruptive or not pulling their weight or simply because they don’t mix well with the others. I think we should be aggressive in hiring the best people for the team but equally aggressive in removing obstacles whatever form they take even if it means changing the make-up of the team.

I don’t want to close before addressing the other items on Laszlo’s list.  I want to voice my agreement over the other points,

* Take away managers’ power over employees

It really makes no sense to hire smart people and then tell them what to do, we hire smart people because they know more than us in their area of expertise – Let them use it!

* Only hire people who are smarter than you are, no matter how long it takes to find them

I agree with the caveat mentioned above, we don’t all have that luxury and for the most part it depends heavily on the next point.

* Pay unfairly (it’s more fair!)

Double what you are willing to pay and be selective. But don’t stop there! Retention is as important as recruitment, don’t lose someone good by being stingy, if you discover you have hired a star, give them a pay raise immediately, I can guarantee that when they go it will be for only 5%-10% more than you currently pay – don’t let that happen and offering to ‘match’ after they have one foot out of the door is far far too late. Let them know you value them now and make it hard for a competitor to lure them away.  Don’t worry if it upsets other people, if they know that reward is based on ability and performance they will respect it even if they don’t like it.

* Default to open: be transparent, and welcome feedback

I am a huge believer in transparency in development, so many projects fail because problems were hidden ignored or deferred or where customers were excluded until it was too late and too costly to change. Be honest and open and learn to see that finding a problem early is a good thing, even if you’d prefer not to hear it.

 * If you’re comfortable with the amount of freedom you’ve given your employees, you haven’t gone far enough

This goes with the first point but let the team lead the way, they likely do know best, and make work a place they want to be, if a team of engineers is happy and enthusiastic you will see an amazing improvement. And trust them.

Trust the team

I shall add that as my own piece of advice for success with software development.  Trust the team, trust their judgement, trust their opinions, trust their concerns and trust that they will do the right thing. In my experience a good team will reward that trust ten-fold.

In my opinion if you cannot Trust your development team it is a failure of management as you have hired the wrong people.