Click here to view the complete list of archived articles

This article was originally published in the Winter 2010 issue of Methods & Tools

---

Testing Dojos

Markus Gaertner, it-agile GmbH, www.it-agile.de, www.testingdojo.org

Programmers use Coding Dojos[1] to exercise their programming skills in a group setup. Deliberately they get together to pair program and get feedback from peers. But how do testers train their skills? Testing Dojos provide a similar setup as Coding Dojos, but help testers to practice their individual skills in a group.

What is a Testing Dojo?

A Testing Dojo is a meeting where testers come together to work on a testing challenge. The challenge can consist of testing a product, generating test ideas for a particular software or even exercising bug reporting. The challenges will use mainly Exploratory Testing, but learning a new test automation tool is also a very nice mission.

A group of testers may decide to get together on a Lunch & Learn meeting each Friday and test an application from the Internet. The team members take turns facilitating the session and coming up with a mission and a product. The facilitator is responsible for the session. She can pick an application from the company or one that she uses regularly during her free time. It could even be one that she came across on the Internet. A mission provides the goal for the testing activities. Setting up a mission for a particular application might be a bit more difficult. That's why we will examine testing missions separately later.

Testing Dojos take testing to a safe environment without schedule pressures and deadlines. Testing Dojos are a way to train testers new to the profession or the company in a collaborative manner.

Equipment

For a Testing Dojo you should have some things in place:

• a meeting room large enough for the group
• access to a computer
• a video projector so everyone can see what's happening
• pen and paper, a flipchart or a whiteboard to take notes

A meeting room with a computer and a projector are available in most companies. If the meeting room lacks a flipchart or a whiteboard, it is easy to get pen and paper for note taking. You will get more benefits if the notes from the session are clearly visible to everyone in the room, especially for a mission to learn about note taking. You can look at tools like Session Tester[2] or Rapid Reporter[3] to take session notes directly on screen.

Roles

During a Testing Dojo, each person in the room always fills one role. There is of course the role of the tester who has the power over the keyboard and who interacts with the software. Then there is the recorder's role, which takes session notes and makes sure to get reproducible steps noted down. The observer watches the performance. He thinks about suggestions for improvements and gives observations about communication in a paired setup. Finally, the facilitator makes sure that the rules of the dojo are followed.

Tester

As the tester has the control over the keyboard, he interacts with the program. It's crucial for the tester to expose his testing ideas about what to try or not to try and describe his mental model to the audience.

Recorder

The recorder takes session notes about the activities. In a single tester setup the tester is simultaneously the recorder and he has to take care about his own session notes. In a paired setup there is a dedicated person for this.

Observers

In a Testing Dojo there are one or more observers, depending on the size of the group. The observers take notes about the process. These notes include the interactions between tester and recorder in a paired setup or spoken thoughts and test activities in a single setup. Observers should monitor the testing activities and the interactions between the pair, rather than watch the screen. Additionally, observers should try to keep track of one or two points, rather than trying to observe everything. If you are neither testing nor recording, you are not taking a break. As an observer you must keep engaged in order to follow what happens.

Specific things to observe might be test heuristics[4], test framing[5], oracles, coverage ideas, management of focus, conversation, monologues, pauses in talk, pauses in testing, eye contact. The facilitator might want to hand out some common oracles [6] or mnemonics[7,8] for the participants.

Facilitator

The facilitator picks the mission for the dojo. She enforces the rules like switching timeframes. At the end she moderates the feedback activity. The facilitator could also take on the role of the tester, recorder or observer, but her main responsibility is to facilitate the dojo. A junior tester can take on the role of the facilitator in order to learn the skills needed to lead a group or meeting.

Mechanics

A Testing Dojo starts with the facilitator introducing the rules. The facilitator provides a mission and clarifies the structure of the dojo. If you run a Testing Dojo for the first time, you could consider providing a first focused test that gets the group going, rather than leaving it too broad.

The session facilitator should change so that everyone gets the opportunity to lead a small group of people.

Any testing can be done by a single tester in front of the computer or in a paired setup.

The missions vary between testing a product, evaluating the usage of the following tools or using a new approach to check if we could incorporate it into our testing process.

When the team has run many Testing Dojos, the introduction tends to get shorter since the facilitator has to explain less of the standard dynamics.

Single tester

In a single tester session, the person with access to the keyboard takes on the role of the tester and the others fulfill the role of observers. On a previously agreed upon time the tester is replaced by another participant from the audience. Five minutes seem to be just enough for this, but you can try different timings based on the size and skills of your group. The new tester continues to follow the mission tackling the product under test. When the individual tester gets stuck, he can ask for support from the observers. Otherwise observers have to remain silent while watching the performance.

For instance the facilitator has picked the latest application from the company and decided to explore it. A tester unfamiliar with the software is asked to begin for five minutes. Right after starting the application, she faces a problem because she does not know what it does. After a few hints from the observers about the documentation, she continues to explore the software, but does not quite understand it. After the five minutes have expired, a tester more familiar with the application starts, coming from the group of observers. He shows some of the features and explores areas of the product. He provides the first tester with some new insights about the application, as well as the approaches used to test it. A third tester then takes charge and explores the product from a completely new perspective.

The tester must explain every step of her thoughts for the observers to follow the individual actions. As a rule of thumb, the more the tester speaks, the better it is for novice testers in the audience. More experienced testers need less explanations of a particular motivation or applied heuristic.

Paired session

In a paired session, two participants sit in front of the computer. The tester is working on the keyboard, while the recorder writes down the test ideas and discovered bugs. After a previously agreed timeframe, between five and ten minutes, the tester goes back to the group of observers. The recorder takes over the role of the tester and one of the observers becomes the new recorder.

During a paired session the tester and the recorder in front of the computer need to clarify their steps so that everyone from the observers understands what they're doing, and more importantly why they are doing it. They should at least talk as much as they test and probably talk more than test. The pair is allowed to involve other participants only when it gets stuck just as in the single tester setting.

The paired setup is similar to a Randori Kata in a Coding Dojo. The people in front of the keyboard need to make their steps clear to the audience. Over the course of the time-box, they explain the model they got from testing the application. This knowledge is then transferred when the pairing partners switch. Since all observers watched and listened to the particular steps performed, the next partners will have similar knowledge about the product. Over the course of the session, each participant takes notes and tests.

Pairs switching can be decided using a round robin style or by picking the next volunteer for a larger group. In round robin switching, each participant should get into the role of the tester and the recorder at least once. The size of the group will influence how much time you schedule for the session. If you have one or two testers who feel uncomfortable to expose their testing steps to others, you can make testing and recording optional.

To make the activity more fun, you can include a ceremony activity for pair switching like a handshake when the tester leaves or a Karate-style bow when a new recorder joins. You can also bring in special things if the team found a bug, like a high five.

Suppose that John and Susi are the first pair. John decides to pick up the keyboard and Susi begins with recording notes. They take the application and discuss which areas to focus on. The session facilitator starts a timer for five minutes, informing the pair when they have two and one minutes left. After getting a brief charter up, John and Susi start to test the application with the first item on their list. They make pretty good progress. The time also passes very quickly. John goes back to the observers after the facilitator announced that the time is up. Susi now takes the keys and Paul joins her. Paul raises a question about one area that he just saw while John and Susi were performing. Susi gets back to this area and Paul asks some questions that eventually result in Susi doing some follow-up testing.

Missions

This section takes a brief look into possible missions for a Testing Dojo. Testing missions provide the goal for any testing activity. Testing missions can vary between "Test this", the evaluation of tools or learning new approaches.

Test this

The classic mission involves testing an application. The variety of applications includes open source programs, commercial software available in your organization or even your company’s latest product. Such a session can end up as a bugfest. The facilitator needs to check if the mission is doable before the session starts. The company’s firewall may block some content on a web page or it might be cumbersome to test a web page with unstable network connections.

This type of missions has many different forms. The facilitator can focus the session on a particular aspect of the application like usability problems. The audience could get to know how to learn and model a new product without thorough documentation.

You could also pick test automation, though this needs more planning and preparation from the facilitator's point of view. The facilitator should provide some initial examples of automated tests and then ask the participants to extend them. Usually during a short period of a single hour or two, automating test cases for a new application seems to be not as productive as an exploration session conducted with mostly manual tests.

Evaluate tools

A mission to evaluate a tool could use mindmaps for test ideas or try out a particular test tool for the whole session. In such a mission, the product under test is usually the tool itself, but you can run it also for a common program that you test at work and compare the results directly with your daily work. Evaluating tools can help to decide if you want to learn more about them.

You could evaluate an open source application for writing test documentation similar to the company standards or assess several free online services such as URL shorteners to be incorporated into the next product. Yet another mission could be to evaluate a new bug tracking system and provide management with information regarding the capabilities of the tool given the company's context and culture.

Learn new approaches

There are many testing approaches to try out. You could focus the mission on some particular mnemonic like FCC CUTS VIDS[9] or use soap operas[10] to generate test ideas. Like tools evaluations, this type of mission aims to try out and learn about new approaches. After these sessions, the whole team will have made some experience and can make a more informed decision about the usefulness of the approach.

Reflection

Take some time after each session to think about it and share observations. To keep the discussion focused on suggestions, you can take turns with stating positive things from what you saw. After that, each participant can provide specific suggestions of what can be improved. Try to fill in the template "I suggest to…", and remind participants to stay positively focused at this. It's easier to accept feedback in this manner.

On a meta-level you should also think about things to change after each dojo. For example you can modify the switching time from five to six minutes, drop a rule or try something new in the next session. The group should make the decisions.

After the reflection, the team votes for the next facilitator, the time and place for the next dojo. The new facilitator becomes responsible to schedule and organize the next session.

Putting it all together

Testing Dojos help your team gain a shared understanding of their approaches to testing. The collaborative setup allows sharing individual approaches quickly. New testers can directly see how a more senior tester would tackle the program, while a more senior tester can get new insights from the fresh perspective of the rookies. A Testing Dojo conducted with project managers and programmers can bring transparency to the magic thing that's called testing. This surely aids in the team building process, since the whole team gets to know how a skilled tester works.

Based on your team and company culture, you can play around with conducting a Testing Dojo each other week, running a Coding Dojo between them. With this alternation you make sure, that you get a balanced approach to your deliberate practice. In the end, programmers attending a Testing Dojo might even find out the value of testing.

You can use Testing Dojos in your local testing user group. Check the Internet if there is already a public Testing Dojo in your neighborhood.

A rich variety of Katas exists for Coding Dojos. There is currently not an equivalent resource to pick testing missions from. One source of inspiration is the catalogue of testing missions from Weekend Testing sessions[11]. Blogs about testing are another source of information. You can also simply ask your favorite search engine on the Internet about a testing challenge. As more teams use Testing Dojos, the collections of possible missions might start to grow.

Happy testing.

Special thanks to Tiina Kiuru and Michael Bolton and all the attendees of the Agile Testing Days 2010 Testing Dojos on their early feedback.

Bibliography

1. Coding Dojos, http://codingdojo.org
2. Session Tester, http://sessiontester.openqa.org/
3. Rapid Reporter, http://testing.gershon.info/reporter/
4. Test Heuristics Cheat Sheet, http://testobsessed.com/2007/02/19/test-heuristics-cheat-sheet/
5. Test Framing, http://www.developsense.com/blog/category/test-framing/
6. Testing Oracles, http://en.wikipedia.org/wiki/Oracle_%28software_testing%29
7. Testing Mnemonics, http://www.qualityperspectives.ca/mnemonics.html
8. The Power of Mnemonics, http://curioustester.blogspot.com/2009/10/power-of-heuristics-and-mneumonics.html
9. FCC CUTS VIDS test heuristics, http://www.michaeldkelly.com/archives/50
10. Soap Opera Testing, http://www.logigear.com/logi_media_dir/Documents/whitepapers/soap_opera_testing.pdf
11. Weekend Testing, http://www.weekendtesting.com

---

Related Resources

Software Testing Magazine

SQAZone.net - Software Quality Assurance Portal

Software Testing Videos and Tutorials Directory

---

Back to the archive list