How to Choose Quality Candidates/Consultants for Your Large Company Agile Initiative

Daryl Kulak & Anita Shankar, http://www.perficient.com/

We created this set of questions to help corporate managers select Agile-experienced consultants and candidate employees for project work. Assembling a team of qualified Agile people is one thing, but the fact that some Agile practices and principles mean different things to different people makes it even harder to succeed in staffing your initiatives.

A consultant or potential employee who has great Agile experience in small product companies may be exactly the wrong type of person to help you inside a large corporation. An Agile evangelist from a small product company may actually be horrified by some of the compromises that are absolutely necessary in your large company environment. We developed this questionnaire to make those distinctions.

Depending on the role you are interviewing for, you may want to leave certain questions out or perhaps add some of your own. Certainly there will be questions you want to ask about things like the person's outlook towards businesspeople and technical people working closely together, which is important no matter whether the project is waterfall or Agile.

1. How long were the iterations (or sprints) on the projects you worked on?

Agile project methodology moves at a fast pace and you should already have a good idea of the length of the iterations for the pending project. Answers of between 1-week to 3-weeks are ideal. If your candidate has worked on Agile projects which have long iterations (4 weeks or longer), or wildly variable-length iterations, it will make sense to determine if this person is comfortable with the iterations as defined for your project. A steady set of fixed-length iterations that are fairly short is best. The theory that big companies need longer iterations is not based in fact. We've done many big company projects in the 1 to 2 week iteration range.

2. Are you a Certified Scrum Master (CSM)? If so, how do you view the way Scrum projects need to be organized?

Often we use certifications as a golden way to qualify candidates. But be somewhat careful with people who have gone through the Certified Scrum Master (CSM) training. The goal here is to make sure that your project team member is comfortable with structure as each iteration progresses. Is she comfortable working with a project manager, or communicating to a team lead? Working inside a large corporation, you personally might feel that a project manager or team lead position is required on all but the smallest effort. But certain Agile consultants will insist that project managers are no longer necessary under Agile because teams will "self organize." Steer clear of these candidates unless that is clearly the direction you want to go. Does your organization have defined roles and career paths like project manager, tester, business analyst, etc.? Do you have a strategy for muddying those career paths for the employees you bring onto this Agile initiative? If so, great. If not, avoid the candidates who tell you there is no alternative to self-organization.

If the person is not a CSM, you may still encounter these issues, so pursue the line of questioning either way.

NOTE: There is nothing systemically wrong with self-organization, but as a hiring manager, you need to know what your own strategy is for Agile adoption and then hire accordingly. I personally haven't met many corporate managers who are terribly comfortable with self-organizing teams.

3. Did you use automated test tools on your projects? Explain how that worked.

Agile project team members should have experience (or at the very least, the desire) to use automated testing tools for regression and performance testing of each iteration. At the end of each iteration you want to have something that your customer/client can "see." Automated testing allows quick identification and isolation of development defects as well as the ability to test development work completed in previous iterations. Expect the candidate to talk about automated regression testing, continuous integration and testing and even performance testing techniques and tools. Also expect them to discuss the need for manual testing as well as automated testing, due to the ever-changing nature of the code base in Agile.

4. Have you done continuous integration on a project before? Describe.

Here you want to get a pretty detailed explanation of how the candidate has used continuous integration on previous projects. Continuous integration is a set of automated build, integration and test steps that executes against the code base in a configuration management repository. For instance, if you were using Java and CVS, the CVS repository would have a set of triggers that automatically built, integrated and unit tested the code often, perhaps each night, perhaps a few times a week, or even every time someone checked in new code. Each of these is a valid continuous integration story.

5. Did your iterations overlap? For instance, were the testers still testing Iteration 6 while Iteration 5 was being designed/developed?

There are many views of how iterative/incremental projects should run under Agile. Overlapping iterations is certainly one strategy. The danger is to pay attention to the candidate if they say that "iterations should never overlap." This may tell you that the candidate is used to having what is called "multidisciplinary teams." This type of team consists of a set of generic team members who all have the skills and enthusiasm for requirements, design, coding and testing and who each participate in those activities almost equally. If your company has defined roles (business analyst, test analyst, etc.) and career paths then this candidate may have a tough time fitting into your structure. They will want BAs to code and testers to do design. Is that okay? It is your decision. Again, nothing wrong with multidisciplinary teams, but your organization must be able to handle the change if you decide to go that way.

6. Have you used story cards or use cases? Explain how that worked for the team.

Often, Agile projects are associated with the use of story cards. However, our goal is to simply understand if our potential team member is comfortable implementing a project (designing, developing, testing, etc) with business information which has been documented in some way. The requirements (story cards or use cases or a combination of both) should have enough information to provide guidelines for developing and testing and also allow the development team to come up with a creative and effective solution. By asking this question, you want to understand if your potential team member is comfortable working with requirements in a structured development environment (versus brief summaries from which the developers build code). Again, this is a matter of matching the Agile candidate's experience with your organization's needs.

7. How did you manage traceability of the requirements to testing?

The point here is to make sure testing goes all the way back to requirements for validation. Not only is it important to test that the functionality a developer has created during an iteration actually functions, it is also important to determine if it functions the way the business wanted it to function. Does it meet the requirements defined in the story card / use case? Your team members should understand the importance of this concept and if they understand and accept traceability, you should be able to count on this person to help you meet project goals.

8. How comfortable are you with ever-changing requirements?

Many development methodologies specify that requirements are locked down at the beginning of a project. Although that is not the case in Agile, it does not mean that requirements are loosey-goosey! The advantage of Agile and its short iterations is that it is easy to quickly recognize that work is not progressing as desired - that what the customer 'asked for' is not what they 'wanted' - and the requirements must be changed. Team members should be able to handle such changes on an Agile project. It shouldn't be so tied to code, a story card or any other component of work that prevents providing a solution which meets the customer's needs. The general idea is that requirements can change a lot at the beginning of the release, and very little at the end.

9. Did people play multiple roles on your development team?

Here we get back to multidisciplinary teams. What you are really trying to understand from this question is how well an individual would fit into your organization and your style of Agile development. If your organization is one in which individuals select a specialized role (such as Java Developer) as part of their career path, your interview candidate may have difficulty on your team if she is more comfortable working in Agile projects where she has had the ability to play multiple roles (Scrum for example). Conversely, if your candidate's primary experience was developed on projects where roles were clearly defined and individuals did not work in multiple capacities, then he may be uncomfortable in an organization where team members can play any role on a project to achieve its end goal. You must choose the candidate which fits your organization well and is flexible enough to suit the needs of your Agile project implementation. Two particular roles that are more easily interchanged are business analysts and testers, other roles are harder to be "multifunctional."

10. What project management tools were used on your project?

This question is more for Agile project managers. Do you have corporate PM tool standards? If so, this question becomes quite important. Agile has a new breed of PM tools including Rally Software, Version One and XPlanner. These tools bear no resemblance to the waterfall PM tools like MS-Project or Clarity. If your candidate is more comfortable using one of these Agile PM tools, try to identify if they will be able to fit their Agile project plans (and issues, milestones, change requests, etc.) into your company's structure.

11. Can you explain the purpose of a burndown chart?

This is more of a question for candidate project managers, although all Agile people should know the answer. A burndown chart is a graph that shows the progress of the team in terms of work "burned through." The work is usually put in terms of a set of "story points" which represent functionality. Once a piece of functionality is coded and tested and reviewed by the user, it is considered to be "burned" and the graph will reflect this. The graph should show a steady movement down until it is clear that the team will have completed the story point backlog by the release date. There is also a variation called a "burnup chart" that works the same way but can handle scope changes more easily than the burndown variety. Again, you want to see how the candidate will link their burndown/burnup chart into your overall project management structure.

12. What does project velocity mean?

This is a PM question, but most Agile people should know the answer. Project velocity is the rate at which a team is "burning" through story points, so a possible velocity might be "30 story points per iteration." That means that so far, the team has been able to identify, code and test 30 units of functionality (story points) in an average iteration and can expect to do about that much in future iterations, giving a fairly good view towards what can be accomplished by a release date.

Hopefully, these twelve questions can be a good start for your qualification process in bringing in new Agile consultants or candidate employees. Our hope is that you can build a highly-qualified team that will fit in well with your corporate development process, culture and PM standards.

References:

Our work is based on the research paper by Dr. Hong Li. The paper is available here:

http://agileplusrigor.ning.com/forum/topic/show?id=2075814%3ATopic%3A565

Here is some good reading on Agile in the enterprise:

Augustine, S. - Managing Agile Projects (Prentice Hall PTR, 2005), 264p

Highsmith, J. - Agile Project Management (Addison-Wesley, 2004), 312p

Leffingwell, D. - Scaling Software Agility (Addison-Wesley, 2007), 384p

Possible related articles and blogs of interest:

LitheSpeed Blog: lithespeed.blogspot.com

The Agile Executive: trailridgeconsulting.com/blog

Leading Answers: leadinganswers.typepad.com

Scaling Software Agility: scalingsoftwareagility.wordpress.com

---

Related Agile Articles

Adopting an Agile Method

Creating an Agile Environment

Five Symptoms of Mechanical Agile

More Agile Knowledge

Scrum Expert

Agile Tutorials and Videos

---

Click here to view the complete list of archived articles

This article was originally published in the Fall 2008 issue of Methods & Tools