Model-Based Testing Adds Value

Ewald Roodenrijs, Sogeti Nederland B.V., http://www.sogeti.nl

As a test manager I always hear the same complaints about testers: good testers are hard to come by and those testers take a lot of time working on their tasks. Both are true, there are not enough good resources available to thoroughly test all the applications that are on the market today. And testing can take a lot of time to do it correct and with the right coverage. There are solutions available to help with these complaints, but they only address one of them: they help you get more good testers or they speed up the testing process. How can we address these two complaints at once?

One of the most time and resource consuming tasks of testing is the creation of test scripts for checking if the software product complies with its specifications. Scripted testing can do this. While testers interpret the specifications, test cases and test scripts are created which can be executed to check if the software was created according to these specifications.

Manual scripting is like using a hammer to wrench a screw into the wall; it can be done more efficiently. Reducing time spend on manual scripting and having less testers involved with execution can create an added value for testing. If this task could be automated it would help those testers who are creating theses test cases. Model-Based Testing (MBT) is a method for creating functional test cases.

With MBT it is possible to generate test cases from models that describe the test object or system under test (SuT). According to the Dutch test association TestNet (www.testnet.org), Model-Based Testing is:

"Software testing in which a test model is created that describes some of the expected behavior (usually functional) of the test object. The purpose of creating this test model is to review the requirements thoroughly and/or to derive test cases in whole or in part from the test model. The test models are derived from the requirements or designs."

Some benefits of MBT are:

• MBT provides the opportunity to automate the process for test specification.
• MBT creates acceleration in the specification of test scripts.
• MBT makes the time and resource consuming task of test specification less dependent on the amount and expertise of testers.
• With the use of a model, small changes in the documentation are translated into new test scripts in few seconds.

So MBT seems to be a useful tool for the (future) test market. Testers can use MBT to specify all the functional test cases! MBT is one of the things that make testing applications easier and faster in the future. But how does MBT actually work and what can you do with it?

Model, then generate

Model-Based Testing is an approach based on creating test cases derived from a behavior model of the test object, the (test) model. This model describes the expected behavior of the test object. These test cases are then, where possible, automatically generated from the test object. The challenge with this approach lies creating a formal behavior model in which the operation of (part of) the application is represented.

Functional requirements, for instance derived from use cases, can serve as a starting point for the creation of the model. The model used for developing the application or a separate test model can do be used. A test modeler creates a test model by hand and a tool that will create test cases can read this model. A lot of tools use an UML (Unified Modeling Language) model. UML is a unified accepted language to create. With UML (2.0) standards, the model is interchangeable with multiple applications that are available on the market. The tools can also improve the process of designing the models.

Figure 1 - UML Class Diagram

Using a separate test model has some advantages over taking a development model. The test model is an isolated model, not a model created with a purpose for analysts, designers or developers. The benefits of this approach are:

• The test model consists of other topics in the classes than a development model, for instance the use of test data and the use of booleans;
• Errors of development model will not be incorporated in the test model, allowing testers to find defects created during design.

An advantage of using a development model is that no separate model has to be created for testing. Some tools even allow the use of other models like pseudo code, decision tables, Markov chains or model specially designed for the tool.

After the creation of a model, a MBT tool can use it. This tool can automatically generate multiple test cases from the model, including traceability to and from different requirements (depending on the MBT tool used). The MBT tool generates test cases by using ‘paths’ through the system on which the test cases are based. These test paths are the same that are used in test design techniques. During the generation of the test cases some applications can automatically check for consistency. When the consistency of test cases is not achieved, errors should be fixed in the test model.

Because MBT generates test cases from a model based on the requirements, it is easy to change them when the requirements change. After a requirements change, you adjust the model and generate the new test cases. This creates a dynamic creation of test cases compared to a more static situation when they are created manually.

Not always automated execution

Model-Based Testing gives testers an excellent option to use the automatically generated test cases for automated test execution. Most tools used for MBT have this ultimately in mind when generating test cases from a model.

Tip: When modeling appoints some functional ‘keywords’ serving test automation, which can be developed more quickly.

The use of ‘keywords’ in the test model helps the automation of test case execution. Some tools have a plug-in for various test execution tools to manage and execute the generated test cases automatically.

Automated test execution is not always possible or even wanted. So some MBT tools generate test cases into readable steps. These test cases can therefore the executed manually from the script. In manual execution, I recommend to use keywords for easy reading. Once the keywords are implemented, the test cases can be executed manually. After the creation of the test cases MBT tools often have an option to export it to multiple formats, such as MS Word or Excel.

Attachment	Step Name	Description	Expected Result
	createCase	Press the Create Case (Label: Insert appointment) button on the menu. with inCanMakeModifications = 'false' with inCanReopenAccount = 'false'	'sut.message' has to be 'NONE'
	Body: createSavingsAccount	Click the Open Savings Account (Label: Open savings account) button.	'sut.message' has to be 'NONE'

Table 1 - Manual test case

Test design techniques covered

When starting a test project, a test strategy is formed based on the wishes of the client, defining result, risk, time and costs. This test strategy is then agreed upon with the client to make sure the strategy is what the client wants. With design techniques (TDT), the risks of the defined impact of the testing can be implemented in the test execution when approving the test object. Assigning the test design techniques to the test object, the test strategy is realized for approving the test object. Therefore it is important that test design techniques covered by Model-Based Testing match the generated test cases with the established test strategy.

Using UML models, MBT covers the standardized test design technique Process Cycle Test (PCT). In the PCT test, increasing or lowering the test depth changes intensity. It also shows that a MBT tool can not only cover the functionality quality characteristic, but also suitability. The same can be said for the Use Case Test (UCT). Because the model can not only be derived from requirements, but also from use cases, it is also easy to cover this test design technique. As with PCT, it also covers suitability.

Not only PCT (and UCT) can be covered with MBT. UML models use for MBT can be customized to cover other test design techniques. Other types of models can even cover more test design techniques. Using different types of data in the UML model can create various situations, like the ones needed for boundaries and equivalence classes. This creates the possibility to use other test design techniques like Data Combination Test (DCT), Semantic (SEM) and Syntactic Test (SYN), Decision Table Test (DTT) and Elementary Comparison Test (ECT). However, if one of these techniques is chosen to test the software, this should be thought through before creating of the (UML) models. The logical test cases should be created before creating the model to know how to customize it. With the DCT, SEM and SYN the use of the correct test data may be sufficient.

Test design technique	Quality Attribute	Standard	Possible
Process Cycle Test	Suitability Functionality	X
Use Case Test	Suitability	X
Data Combination Test	Functionality		X
Semantic test	Functionality		X
Syntactic test	Functionality		X
Decision Table Test	Functionality		X
Elementary Comparison Test	Functionality		X

Table 2 - Test design techniques covered with MBT

As an additional advantage, MBT allows the creation of standard test cases. This can be an added value for the ability to outsource the test execution.

Evaluate early

With Model-Based Testing (MBT) it is possible to automatically generated test cases derived from a (test) model. But as you derive the models from the requirements you also evaluate the requirements!

When creating test models questions come up about the quality of the functional requirements. These questions are simple discoveries of defects about the requirements. To solve them they need to be presented to the designer or analyst of those requirements. In a normal, non-MBT situation, these questions could only be asked during the (manual) specification of the test cases. So these defects could already be added to the code or prototype. With MBT however, these discoveries should already be made after delivering the functional requirements during the preparation phase. It pushes the preparation phase even more upward in the development process.

And now?

Testing doesn’t stop with the automated creation of test cases. The test cases should still be executed. When ‘keywords’ are used to help the test execution, it is also possible to automate this test execution. Using formal UML models test data helps to generate automated test cases. Some tools even have plug-ins for various automated test execution tools. Tools that can be used for MBT are (not extensive):

• Smartesting Test Designer
• COVER
• Conformiq Qtronic
• All4Tec MaTeLo
• Microsoft Spec Explorer

However MBT cannot be used to test the whole test object. As stated it only covers the functionality quality attributes and part of suitability. Other quality attributes should be tested otherwise. MBT checks if the functional requirements are covered, but you should use other test methods to cover the whole application.

Using MBT can reduce testing time compared to specifying test cases manually! The challenge lies in learning how to create the models. This is an unexplored territory for most testers and could be a separate specialization within testing. However, if you succeed, automatic test cases generation with automated test case execution will become a realistic possibility.

---

Software Testing Resources

• Software Testing Magazine
• Software Testing Videos and Tutorials

---

Click here to view the complete list of archived articles

This article was originally published in the Spring 2010 issue of Methods & Tools