Test Automation Strategy Out of the Garage - Part 2

Mark Rossmiller, SWQA Software Design Engineer,Hewlett Packard

Radical ideas are not bad ideas.

I found the following "commandments" by Robert Schuller highly insightful in determining our organizational behavior tendency to resist radical ideas. If one can relate to experiencing one of these commandments then we have taken the first step in recognizing where to start improving how we approach testing:

1. Never reject a possibility because you see something wrong with it.
2. Never reject a possibility because you won't get the credit.
3. Never reject an idea because it's impossible.
4. Never reject a possibility because your mind is made up.
5. Never reject an idea because it's illegal.
6. Never reject an idea because you don't have the money, manpower, muscle, or months to achieve it.
7. Never reject an idea because it will create conflict.
8. Never reject an idea because it's not your way of doing things.
9. Never reject an idea because it might fail.
10. Never reject an idea because it's sure to succeed.

Demand change. Once again the approach taken to automating software testing is greatly affected by the environment in which it is expected to perform. Currently, the software quality organization is entrenched in a technician rich paradigm where testing methods are geared toward manual, ad hoc validation of the software driver and associated printer products. This method is often far down-stream from where the eventually discovered defect - actually was introduced in the development design.

(1) The point is that optimization (automation) should be introduced at the most optimal time in the development cycle to prevent defects from being introduced.

(2) To put automation and tools where they are most effective requires the software quality organization to reclassify the roles and responsibilities of its quality engineering staff. Tools engineering should operate closest to our R&D partners in order to provide automation at the most effective time when design defects can be filtered out. This is before driver release candidates arrive in the labs for validation; rather than down stream where canned testing produces limited results.

(3) Software quality must aggressively evaluate the core competencies of its staff and actively anticipate the knowledge base necessary to meet current and future testing requirements for new and emerging technology. The organizations inability to fill requisitions for needed technical staff and FW engineers, for example, is an indicator that internally we have not adequately anticipated demand external to HP; or we have not grown our own competencies internally to meet future need.

(4) Software quality does not employ consistent testing methods that help facilitate automation, practical tools, or overall efficiency improvement. Other side effects to this inconsistency make valuation of cost or time associated with testing nearly impossible. Features may be added, changed, or even removed in development, which tends to ripple through the test plans and relevant test cases. The way in which we characterize and manage our defects is not consistent creating non-value added work activity increasing the cost of testing.

Invent different ways of working.

Here is an analogy to explain the situation in a software quality organization. There are a number of people feverishly bailing water from a leaky boat. The captain brings on more people to help bail but none of them know how to fix the leak in the boat. The weight of the additional people only makes the water rush into the boat faster and now everyone is feverishly bailing out the boat. Meanwhile, the boat has no direction and is going nowhere. The best solution to the captain's problem might be to bring people on board who can fix the leaks in the boat rather than bail water. Now the captain can stop the leaking, find a direction, and get underway making progress toward a goal .

Our organization must develop a new way of working, by not only investigating but implementing new ways of testing our products. Automated testing methods are one of many ways to do that if properly sponsored and approached. Both management and testing personnel have become accustomed to doing things the same way, with relative success, and continually fall back on inefficient testing methods we are familiar with but cannot make real progress in improving. We have become complacent and indifferent to changing the methods of testing with which we are most familiar and comfortable. So when the work starts rushing in and people become stressed, we all start bailing faster and getting nowhere for the future.

Some common risks to the test automation effort include management and team member's support fading after not seeing immediate results, especially when resources are needed to test the current release. Demanding schedules will put pressure on the test team, project management and funding management to do what it takes to get the latest release out. The reality is that the next release usually has the same constraints and you'll wish you had the automated testing in place.

Make a contribution every day.

I recommend our organization take a long look at the core competencies of our staff and aggressively promote the learning and growth perspective of current software testing personnel. Development plans should have well defined learning expectations that both managers and personnel recognize as a necessary objective for the success of the individual in the software quality organization. These learning expectations should carry weight in deciding opportunities for advancement, ranking, compensation, and overall employee job satisfaction. We also must maintain a standard of expectations on new personnel that they have the skills to improve our organization now and in providing new ideas to increase testing efficiency for future technology of our products. Any concessions made toward qualifications must be made up through continuing education opportunities. Otherwise, it is easy to see why people become resist to using sophisticated tools and what seem to be complex processes they have done without just fine.

Go to part 1    Go to part 3   

---

Click here to view the complete list of archived articles

This article was originally published in the Summer 2001 issue of Methods & Tools