Immersive Interviewing - Building Great Agile Software Teams

Mitch Lacey, Mitch Lacey & Associates, http://www.mitchlacey.com

Building great agile software development teams is challenging using traditional hiring methods. Candidates might be able to answer your questions and prove C++ skills, but what you really want are people who are competent and capable, who will work well with others and fit with your Scrum team. Immersive interviewing is the best way I know to hire for agile software development teams, or for any other position in your organization. And like all good agile practices, it begins and ends with the team.

The Story

Let me start by telling you a story. Scott’s business was growing. His 100-person company had a solid reputation. The trouble was, they were having a hard time keeping up with all the new business they were generating. It was time to hire some new people.

Prior to starting his own shop, Scott had worked at a large software company, where he felt the hiring policies got in the way of finding the right people. He wanted to establish better procedures at his own company while they were still small enough to do so. This proved to be a harder sell than expected.

"Scott, I don’t know why you’d want to change things. We have built a great company by hiring people this way. Why fix what isn’t broken?" asked Janeen, the director of program management.

"I agree that we’ve had some success so far," replied Scott. "As we grow, though, I won’t be able to screen every new hire personally and neither will you. We need to have a scalable model that builds on what we value as a company. Think about it. Even with you and me screening people, we’ve ended up with some bad hires."

Janeen looked down at the pen that she was twirling. Scott continued.

"Remember Victor? That guy was a train wreck when it came to working with people. Sure, he knew his stuff, but he was such an introvert! It was like pulling teeth to get him to work with anyone. We just fed him work and when it didn’t work, it was always someone else’s fault!"

"Now Scott, that’s not fair. Victor was my hire. I think he was a great person. When he was focused, we got a lot out of him," said Janeen.

"I agree, but when was the key. And even when he was focused, his attitude towards people was awful. He was just not a team player. I’d rather have a guy who can work with others and have half the skill of Victor than go through that again," said Scott.

Janeen nodded. "Yeah, that was pretty rough."

"So, I think I have a way to do it better," explained Scott. "You know how we’ve been really focusing on a candidate’s technical capabilities? Well, I think we should take some lessons from how the teams have been pairing and build on that."

"What do you mean? You want candidates to pair for an hour with a team and call that the interview?" asked Janeen.

"In essence, yes, but before we talk about the how, let’s focus on the why. Do me a favor and go write our company values up on the white board," Scott said.

When Janeen hesitated, he added, "Trust me on this one."

Janeen gave Scott a look that said, "You’re crazy," but went up to the white board and wrote the company values:

"Now Janeen, please tell me how in our current hiring process that we are able to screen for these values," said Scott.

"We don’t specifically. But we look for senior-level people; they should have this stuff already," said Janeen.

"I agree that they should. But I think you and I both know that not everyone does. Victor wasn’t the only bad fit we’ve had to deal with. And I think that’s on us," Scott explained.

Janeen looked perplexed.

"You think it’s our fault that Victor didn’t work out? Why?" she asked incredulously.

"It was our fault. We hired someone who didn’t share our company values. We were under pressure to hire a certain set of technical skills and, even though we were concerned about his lack of people skills, we didn’t think they’d be a big problem. Instead it was a battle to get Victor to even understand why we were working the way we were, much less convince him to comply. And bad hires snowball into other losses. Remember Mark, who was on his team? Do you know why the real reason he left the company?" asked Scott.

"Nooooo!" said Janeen in sudden realization.

"Yup," said Scott, "He left because of Victor. He didn’t tell most people that was the reason, but he did tell me. He said life was too short to deal with people like that."

"That was a hard loss. We’re still feeling the impact from that," admitted Janeen.

"Right…" said Scott, "That’s why I’m convinced we need to look at competencies and values as much as we look at technical skills. Technical skills can be learned in months, but the ability to work with someone is, well, something that takes a very long time."

Janeen nodded in agreement. "OK. So what do you propose?"

"First, I think we need to consider an interview as a forecast. We are forecasting if the person will be a good fit, and how long it will take them to become a contributing team member. Next, I think we need to consider the competencies, skills and level of the role we are hiring for…"

Janeen interrupted. "The level?"

"Yes" said Scott, "do we need someone junior-level, senior-level - or is either one acceptable depending on the candidate. The level of maturity and expertise we’ll expect will be different depending on the level we’re hiring for."

Janeen nodded again, "Got it. Go on."

"Next, I think we need to understand why we are hiring. Is the team working in the best possible way, are there organizational issues that are blocking progress, is the skillset we’re lacking one that could be filled by a team consultant [LACEY] or do we truly need a new core team member? The answer to these questions directly affects who we will hire.

We need a way to make these determinations in a repeatable, consistent way across the company. Many people, me included, have picked up bad habits in terms of hiring. We are all inconsistent with our approach and we don’t treat it as a team event, more of an individual burden for an hour or two on any given day. We need to hire as a team," said Scott.

"You’ve sold me," admitted Janeen, "Now what?"

"Now we model it out" said Scott.

Model

Hiring practices come in many different shapes and forms. Some teams do whiteboard team interviews, others do one-on-one interviews in offices, still others start with coffee and a social chat.

However they structure the interview, most organizations fail to ask the most important questions, not of the candidates but of themselves:

1. Why are we hiring? What problems are we trying to solve?
2. Should we even be hiring? Is there potentially an impediment inside the company that is going unresolved and a new hire is only the mask?
3. What are the most important skills and competencies to consider?
4. How are we hiring?

Before we delve into these questions though, let’s delve into Scott’s definition of an interview: a forecast.

Forecast

To forecast is to make a prediction in advance. When we hire people, we make predictions based on written and verbal communication as to whether a given person will be a good fit, both culturally and skillfully, in the company over a period of time. Although, some organizations do this by having a trial period, most companies in the tech industry hire solely based on the interview. That means the forecast made in the interview has to be fairly accurate. Yet this forecast is only as good as the common understanding of why and how they are hiring.

Why Companies Hire and the Problems They Try to Solve

So why are you hiring? To meet current needs? Because you anticipate a new contract? Or are you part of a hierarchical organization and feel the need to build up one particular department or have more direct reports? Some companies simply go on hiring binges, where they start out hiring for the right reasons and then can’t seem to stop. I think of this like an ice-cream or chocolate binge; you overdo because it feels so good at the time that you forget how much pain you’re going to be in later.

Brooks’ Law states that "adding manpower to a late software project makes it later" [BROOKS]. I believe that’s partly because some companies add people merely to mask or cover up some issue. Complaints of not enough testers is really a sign of poor test automation. Reports of not enough developers really points to a lack of collaborative teams. I’ve even see people bring in additional project managers to bring a project in on time when the real problem is lack of commitment, goals and team alignment.

In our story, Scott had organized his company in a flat structure, so that people felt ownership, commitment and independence. He had laid out clear values for the organization, and for the most part the employees shared these ideals. Even so, he wanted their first question to be "Why are we hiring?" The right answers are to address a skill that is missing on the team or in the consultant pool, or to address a core principle in the organization. For example, Scott might want to expand his market share. In that case, he understands why he’s hiring (to expand market) and is able to develop a question set that applies to that scenario.

Cost of a Bad Hire

We’ve all been part of a bad hire – either as the person doing the hiring or as the person getting hired. We all know it feels bad, but what does it cost? Careerbuilder.com, a large online career site, runs frequent surveys to find out just what a bad hire would cost. The results are staggering [CAREERBUILDER].

What’s driving up the cost of a bad hire?	2012	2011
The price of a bad hire adds up in variety ways. According to the survey, the most common money-sucks that result from a bad hire are:
Lost productivity	39%	41%
Lost time to recruit and train another worker	39%	40%
Cost to recruit and train another worker	35%	37%
Negative effects on employee morale	33%	36%
Negative impact on clients	19%	22%
Fewer sales	11%	N/A
Legal issues	9%	N/A
What defines a "bad" hire, anyway?	2012	2011
When classifying what makes someone a bad hire, employers reported several behavioral and performance-related issues:
Quality of work was lackluster	67%	63%
Failure to work well with other employees	60%	63%
Negative attitude	59%	62%
Attendance problems	54%	56%
Complaints from customers	44%	49%
Failure to meet deadlines	44%	48%
Why do bad hires happen to good people?	2012	2011
When asked what accounted for the bad hire, survey participants reported the following reasons that led to their hiring mistakes:
Needed to fill the job quickly	43%	38%
Insufficient talent intelligence	22%	21%
Sourcing techniques need to be adjusted per open position	13%	N/A
Fewer recruiters to help review applications	10%	N/A
Failure to check references	9%	N/A
Lack of strong employment brand	8%	N/A

Remember Victor from our story? Victor was classified as a bad hire because he could not work well with other employees. To make matters worse, his conduct caused one of the good hires to leave the company. It’s no wonder that company culture suffers from bad hires! Let’s look at the factors we should consider when hiring that might help us avoid these costly disasters.

Skills, Competencies, or Both?

Let’s say your reason to hire is that you need more C# programmers. Most companies would screen prospective candidates based on that particular skill: Do they know C# or not? While that makes sense on the surface, it might not be the best way to make a hiring decision. Skills, after all, are easy to learn and can be picked up in a matter of months. A developer who knows Java but has never tried C# will be able to pick up C# after a few months, especially when paired with an experienced C# programmer. This assumes, of course that the developer has an ability and willingness to learn, to ask questions, to share his strengths and weaknesses; to put his ego aside and have the courage to learn a new language in order to best help his team.

Let’s reverse that, then, and imagine we hire a very experienced C# programmer who is uncomfortable working in pairs. He doesn’t want to improve or branch out from C#, so can never be taught new languages. And he doesn’t enjoy mentoring others, so he cannot share his extensive skillset. No matter how senior or experienced this developer is, he would not be a good fit at Scott’s company, or on any agile team, because of the misalignment of values.

Based on those two scenarios, screening candidates should always include competencies - how a person thinks, how that person interacts with others, and what that person values. These will impact how quickly he can assimilate with a team and how rapidly he can learn new skills. After all, you might need that same person to learn yet another new language a few projects down the road. In general, you should hire for long-term fit, not for short-term expertise.

How to Hire

In my time both working for and consulting at companies both large and small, I’ve seen just about every hiring style and just about every interview question. Some were a bit more obscure, like tell me your favorite sports team (I had this one once) to the more basic, "What would you do in case of fire?"

In our story, Scott and Janeen were struggling to find an interview style and question set that they could use to consistently and evenly hire people, while maintaining the company values. What I find of value, and what Scott recommends in the story, is to apply agile techniques to hiring - more specifically pairing.

Pair hiring is an immersion technique that relies on screening rather than a series of questions. When you’re hiring, you’ll have a list of things to validate with the candidate. Is he a good cultural fit? Can he give and accept criticism? Is he comfortable with conflict? These questions can be answered through discussion or they can be answered through immersion.

Let’s go back to forecasting, and the importance of making an accurate one when hiring. Recall that in the agile manifesto, the preference is for Individuals and Interactions, Working Software, and Responding to Change. Why? Because they are more likely to lead to success. Agile hiring should be no different. We want to see how a candidate works, thinks, and behaves in the real world –the metaphorical equivalent of working software. We want to know how a candidate will interact with the other members of the team. And we want to understand whether they are comfortable in an environment of feedback, continuous improvement, and change.

When it comes down to it, traditional approaches to hiring can only show us how well the candidate can answer the questions of the interview. While those answers might be an honest reflection of a candidate’s ability, they are analogous to slideware showing the intended functionality. I don’t know about you, but I’m going to feel much better about my forecast if I see the person in action, with his or her intended team.

That’s why I prefer to immerse the candidate in a typical day (or multiple days) in the life of a project team. I have the candidate pair with team members in their team space. I prepare the team for the items they are looking for in the candidate – both competencies and skills – and have them weave that into the day. I encourage the candidate to write code, test, document - basically act as a team member for the day. I often have them attend a daily standup or come to the demo or retrospective if the timing works out. The point is to fully immerse them in the environment in which they will be expected to perform.

Candidate Screening

In the story Janeen and Scott agreed that their current hiring process was insufficient for screening for competencies. Victor was an example of someone who had slipped through the cracks; he had the right skills but the wrong competencies. Many organizations try to solve this problem by developing a standard question set to use company-wide. I prefer to screen based on the actions and behaviors I observe, rather than how a candidate answers a set of questions.

During an immersive interview, it’s easy to gauge a candidate’s skills and experience. Coding not up to snuff? It’s easy to see. Trouble with TDD? That’s obvious too. There’s no hiding when your hands are on the keyboard and someone is watching how you work. Competencies might seem a little harder to judge, but they really aren’t. It’s just a matter of learning to look for and comment on the competencies a person exhibits while working with you.

For example, let’s say Scott’s company is hiring a new candidate for a five-person team. The candidate will be scheduled to work in a pair with each member of the team throughout the day. During that time, the team members are not going to be asking typical interview questions such as, "So, tell me about a time you demonstrated a willingness to learn?" or "When making mistakes, what do you do to take accountability for them?" That would be a waste of time.

Instead, prior to the interview, each team member will pick (or be assigned) one or two skills to pay special attention to. So one person might choose C# skills while another might choose TDD. Everyone, however, should be looking for competencies: how the candidate fits the company values.

Preparation and Setup

If this is the first time you have done immersion or if the team members are not familiar with what the company values mean, you should also take some time to discuss what each of the competencies mean and specific behaviors to look for, likely with the help of management and leadership, to ensure the screen is the same for all candidates. This might even be something you standardize and write down, so that the team members can refresh their memories prior to an interview.

You should also keep in mind the difference between hiring a senior-level person and a junior-level person. If you’re interviewing a senior person, you would expect a certain level of maturity and expertise. You would expect professionalism, experience, and insight. Your screen, then, would be focused primarily on exposing moments of unprofessionalism, discomfort when working as part of a team or with others, and similar signs of immaturity. You’ll have a chance to see their technical expertise, including the way the approach problems and their openness to feedback, during the pairing sessions. At the end of the day, I’d much rather teach a knowledgeable and mature person a new technical skill then try to change a person’s mindset, no matter how talented they might be.

If on the other hand, you are interviewing a junior person, say someone fresh out of school, you would be more forgiving of lack of expertise. What you would want to see technically is a basic level of competence coupled with a willingness to learn, ability to communicate, and excellent problem-solving skills. While you would generally expect a more immature individual, you will still need to make sure the person is able to accept feedback and work as part of a team.

But beware, where is actually some advantage to the lack of experience you find with junior people. Experience can sometimes blind people and force people into ruts. Of course, mature people with a great deal of experience can also be open-minded and attack problems without experience getting in the way. That’s why it is so important to hire individuals with the willingness to learn, the ability to teach, the openness to share knowledge, and the desire to create a learning organization – no matter their experience level.

Scoring Candidates

Throughout the immersion interview, the candidate will work with multiple people, each of whom is looking specifically at a few chosen skills and at all of the required competencies. At the end of the day, the team will meet to discuss the candidate(s), but it’s critical to capture each interviewer’s impressions in real time as well.

To collect all of the data from the various interviewers, I use an online table like the one shown below and a scale of 1-10, where 1 is the worst and 10 is the best. As the candidate leaves one team member and moves to the next, the interviewer should take a moment to jot down scores that correspond to how the candidate performed. The chart will not be completely filled in at the end of the interview, but you will have enough data to have relevant discussions to determine if the candidate should be hired or not.

15-Jun	Candidate: Steve Jones	Team
	Company Values	Billy	Scott	Janeen	Venkat	Xiao
More Junior	Willingness to learn	9	9	2	8	9
	Doesn't have a sense of entitlement	10	10	10	10	10
	Takes responsibility	7	5	6	7	8
	Ability to stay focused
	Asks for constructive feedback
	Do what they say they will do
	Owns up to mistakes
	The ability to help make others great
	Easily approachable
	Develop and mentor the team and others
	The willingness to share knowledge (ideas, experience, information)
	Can foster candid conversations
	Interested in finding the solution than being right
More Senior	Help grow a strong team
	Technical	Billy	Scott	Janeen	Venkat	Xiao
	C++	9	9	2	8	9
	SQL	7	5	6	7	8
	Architecture and Design	9	10	9	10	10
	…..

Think of this like planning poker but not in real time. Each team member shares his or her own estimate of the candidate’s fit in each of the key areas. Then, once the interview ends, the team gets together to have the all-important conversation. If there are outliers, such as Janeen’s rating of 2 for C++ in Table x.y, the team should talk about the reasons for the extreme difference of opinion and try to reach a consensus. If all the ratings are similar, like the ones for Architecture and Design and Doesn’t have a Sense of Entitlement, the discussion will be more about general observations than reaching consensus.

Regardless of whether the numbers match or are wildly different, the point of the table is to serve as a real-time data collector for a future conversation. Do not, under any circumstances, use the table without also having an in-depth team discussion; you’ll miss something vital.

Hiring Managers & Non-Technical People

It might seem as though this approach only works for technical hires, but this approach also works when selecting non-technical people.

What I do, and what I propose you do, is ensure you have a good set of competencies that everyone screens for. An immersive experience for a non-technical hire, though, would include attending meetings, participating in brainstorm or creative sessions, or maybe even doing a small portion of a presentation that they can learn quickly. Yes, there are risks here, but remember, we are looking for an immersive environment that will help everyone arrive at an accurate forecast of the candidates competencies and skills.

Keys to Success

When Valve Software published their employee handbook [VALVE] in March 2012, it set the Internet a-buzz with how perfect Valve is as a company. They are flat, everyone is accountable and responsible; people own their own fates. In fact, Valve puts so much emphasis on hiring that the handbook even states, "Hiring well is the most important thing in the universe. Nothing else comes close. It’s more important than breathing." [VALVE]. That’s powerful stuff.

Hiring people for any role-manager, ScrumMaster, Product Owner, team member, architect - is challenging. Anomalies can occur in hiring that end up hurting the culture of the company (but keep in mind that your culture can only be hurt if you know what it is and are actively try to maintain it).

Build a Repeatable Hiring Process

Having a consistent hiring process is key. It should be based on company values and principles. You should screen candidates in an immersive environment so you can determine their real behavior versus how well they answer questions. The larger a company gets, the harder this may become, which is why it’s a good idea to hold regular "how to hire at our company" sessions-training to help keep all teams fresh and up to date. This helps prevent bad hires and also refreshes and refocuses current employees on the company culture.

Focus on Competencies, not on Questions

Interview questions are great, and having a good set of them is definitely helpful. However, running an interview by just having the candidate answer questions and solve problems will only tell you how well they can interview. What you really want is an accurate forecast as to their fit as a team member. Focusing on competencies will help increase your confidence in that forecast. Make sure everyone knows and understands the competencies are that you are looking for, both at a team level and company level. You might even want to do mock interview workshops where people from an existing team interview another team (and vice versa) to teach people how to focus on competencies.

Skills are Easy to Learn, Competencies are Not

When I was a child, I was deathly afraid of public speaking, or any sort of presenting. I was good, however, at research and analysis. The papers I wrote for school always received good marks, all the way through my college career. It was not until I was in my late 20s, however, that I became comfortable with public speaking and presenting. I read books on how to overcome my fears, about how to be a better presenter. Overcoming my fear and becoming a better speaker took years. Public speaking is a competency.

Skills, on the other hand, can be taught rather quickly, especially given that most, if not all, of your hires will have some sort of background in the area that you are hiring for. For example, perhaps the candidate knows Oracle systems and you are hiring for a Microsoft stack – the candidate won’t have to overcome a childhood fear or a deeply seated habit to learn Microsoft. Picking up new skills is relatively easy. Of course, that assumes that the person has the competency of willingness to learn. Having the ability and courage to say, "I don’t know how to do that, will you please help me learn?" is a competency that takes years to master, and can only be practiced safely in a culture that supports it.

Find People Stronger than You

Insecurity and politics can sometimes make it difficult to hire a person, even when you know they have great skills and competencies. I’ve seen people throw common sense out the window and ignore a candidate that seems "too strong" or comes from "so-and-so’s department," especially when organizations are structured in a hierarchical fashion. Why? Perhaps the person posed a threat to one of the interviewers, or isn’t specialized enough, or comes from a department where the two department heads do not get along. Don’t fall into this trap. Hiring strong people will allow you, and the rest of the organization, to grow.

Understand the Costs and Invest Heavily

As Valve said, hiring is indeed more important than breathing because a bad hire can suck the air right out of the room and have damaging effects on all things in the environment. Invest in good hiring practices, create immersive environments where you can determine if the candidate is a good fit for the culture of the company, and push accountability. Have a united approach so you can avoid the mistakes that traditional company do. Hiring is hard, but using this approach (one of many out there) will definitely help you increase your chances of a good hire.

References

[BROOKS] Brooks, Jr., F.P. The Mythical Man-Month: Essays on Software Engineering, 20th Anniversary Edition.Addison-Wesley

[CAREERBUILDER] CareerBuilder.com. What Bad Hires Really Cost Companies (Hint: It’s More Than Just Morale). http://thehiringsite.careerbuilder.com/2012/12/13/cost-of-a-bad-hire/, (accessed January 11, 2013).

CareerBuilder.com. More than Two-Thirds of Businesses Affected by a Bad Hire in the Past Year, According to CareerBuilder SurveyHeadline. http://www.pitchengine.com/careerbuilder/more-than-twothirds-of-businesses-affected-by-a-bad-hire-in-the-past-year-according-to-careerbuilder-surveyheadline, (accessed January 11, 2013).

[LACEY] Lacey Mitch, The Scrum Field Guide : Practical Advice for Your First Year, Addison-Wesley

[VALVE] Valve Software. Handbook for New Employees. http://www.valvesoftware.com/company/Valve_Handbook_LowRes.pdf., (accessed January 11, 2013).

Works Consulted

Rothman, Johanna. 2012. Hiring Geeks that Fit. Vancouver, BC, Canada; Leanpub.

Copryright © 2013 Mitch Lacey

---

Agile and Scrum Team and People Articles

Choosing Candidates for Large Company Agile Initiative

Fear of Intervention - How Subordinates Grow to be Entrepreneurs

Agile Coaching Tips

T-shaped Skills and Swarming Make for Flexible Scrum and Agile Teams

Scrum and Project Management Knowledge

Scrum Expert

Agile Videos and Tutorials

Project Management Planet

---

Click here to view the complete list of archived articles

This article was originally published in the Spring 2013 issue of Methods & Tools