Personal Quality Management with the Personal Software Process

Alan S. Koch, ASK Process

Abstract

Software development organizations have a variety of mechanisms at their disposal to help in managing and improving the quality of the products they produce. Quality Assurance organizations, problem reporting systems, software process improvement and peer reviews (to name just a few) are important tools for product quality enhancement. But an often-overlooked piece of the quality puzzle may well provide the most effective means to improve product quality: the individual software engineer.

After a short introduction to what the Personal Software Process (PSP) ^SM ("Personal Software Process" and "PSP" are Service Marks of Carnegie Mellon University) is, we will highlight the ways in which individual engineers (and their organizations) can benefit from adding the PSP's Personal Quality Management techniques to their professional repertoires. We will take a brief look at the benefits that have been achieved by those who have already learned to apply these principles in their work. Then we will examine in more detail the specific activities PSP-trained engineers engage in to manage the quality of the software they produce. We will look at everything from simple defect logging, to personal and peer reviews, to developing a personal quality plan and using it to guide your work.

What is the PSP?

The concepts and activities discussed in this article are the quality management aspects of the Personal Software Process (PSP) developed by Watts S. Humphrey of the Software Engineering Institute (SEI), and described in his book, PSP(sm): A Self-Improvement Process for Software Engineers [Humphrey].

The PSP is more than just training; it is a boot camp consisting of about 40 hours of classroom instruction, 10 programming assignments, and 3 data-analysis exercises, requiring a total of about 150 hours for the average programmer to complete.

The result of the PSP boot camp is that the programmers don't just learn about good processes, they actually improve their own processes,measure the effects of those process changes, quantify the benefits they have experienced, and set goals for further improvements. The PSP achieves these results by leading students through three steps.

In PSP0, they lay a simple foundation for the learning to come:

• Following simple process scripts,
• Collecting three basic measures of their work (time spent, size of products produced, and defects corrected), and
• Performing a simple post-project analysis.

In PSP1, they begin to build the capability to plan and manage their own work, setting the stage for Personal Quality Management:

• Following a defined project planning process,
• Using their own prior data to make increasingly more accurate estimates for each programming assignment, and
• Planning their work at a level of detail that allows them to track and manage their progress.

In PSP2, they focus on achieving significant quality improvements by learning how to engage in Personal Quality Management:

• Using their prior data to plan for incremental improvements in the quality of their programs,
• Removing defects early using personal review techniques guided by their own prior defect performance, and
• Identifying and capitalizing on defect prevention opportunities in their program design and implementation methods.

Those who complete the PSP boot camp emerge with the knowledge and skills to make accurate plans, work to those plans, and produce superior quality products.

The Motivation for Personal Quality Management

Most programmers have never learned to apply personal quality management techniques like those described in this article in their work. This is because our educational system has generally ignored the topic, and in the software business, "quality assurance" often refers to little more than testing. This is unfortunate, because vast opportunities for improving the quality of the software we produce are being missed as we continue with business as usual.

This is surprising, given that these techniques are anything but new. Other engineering disciplines have embraced them for decades and manufacturers and engineering companies have been applying them for nearly fifty years. Methods like these have been researched and described by such luminaries as Walter A. Shewhart and W. Edwards Deming as well as their more modern counterparts.

Watts Humphrey, in the PSP has given programmers the opportunity to understand these methods and apply them to software development. Those who have done so have found the effects on their work to be dramatic, with the improvement in the quality of their software the most obvious and rewarding aspect. These individuals have begun to take control of the task of writing programs, and are taking the first steps toward changing "software engineering" into a true engineering discipline.

Defect Content as a Dimension of Quality</p>

When we think of software quality, we tend to think about defects. This is a natural tendency, since defective software isn't good for much. But defects (or more correctly, lack of defects) is not the only, or even the most important dimension of quality. Other dimensions of quality include Usability, Maintainability, Install-ability, Security, and Reliability. In any product, each of these dimensions is important to one degree or another, and every development project must be careful to pay appropriate attention to them.But from the point of view of Personal Quality Management, defect management is first priority. This is true for two reasons:

• The other dimensions of quality are issues of either product requirements or organizational standards. While the individual software engineer can affect these things, they are not under his or her direct control.
• Defect injection is a purely personal phenomenon. Each person's defect numbers, types and patterns are unique, and require personalized actions to effectively detect and eliminate them.

You may note throughout this article that I will never refer to "bugs". I believe that this term trivializes a very important issue, making it sound minor, or even cute. In fact, defective software wastes millions of dollars for companies and individuals throughout the world. And as we become more and more dependent on software for critical functions, it is becoming important to organizations' and individuals' health and life. In order to focus on defects as a quality management priority, we must understand their nature. This nature can be summed up this way:

• Software that contains one or more defects is "defective".
• Defects are the result of human errors.
• All humans make errors.
• All software engineers are human.

Therefore, defect management must be the first quality management focus for all Software Engineers.

Career Enhancement by Reducing Both Costs and Defects

As professionals, we Software Engineers must assume responsibility for the quality of the products we produce. Our employers pay us for producing not just software, but good software, high-quality software.

The quality assurance organization cannot be thought of as the people who are responsible for quality. Indeed, of all members of the organization, they are in the position that is least able to affect the quality of the product. Testing and other QA activities can detect defects and make sure that they are removed, but quality cannot be tested into the product at the end. Quality must be built in or it will be absent. Therefore, the software engineers are the ones who have the most direct impact on the quality of the product.

It is easy to lose sight of the tremendous costs that defects cause for our employers. Integration and system test can account for 50% of development time (and most of that time is spent in remediation and re-testing to eliminate the scores of defects that are commonly found). Then post-release defect mitigation and removal can cost more than the original development effort (including testing)! Supporting customers, investigating problems and releasing patches consume huge amounts of time and effort. And the cost of defects in loss of customer good will and market image goes beyond even that.

Defect removal costs escalate with each passing lifecycle phase. Different people have estimated different costs at the various phases, but they tend to agree that the costs rise (possibly even exponentially) with each passing phase. (e.g. A defect that might cost one dollar to fix in the requirements phase may cost $10 during design, $100 during coding, $1,000 in test and $10,000 in the field.) Regardless of the specific numbers one might cite, the cost escalation is obvious from the fact that we talk about removing "defects per hour" before Integration, and spending "hours per defect" during and after Integration.

Your value to your employer will be enhanced by your ability to produce salable (or usable) software at a minimum cost. Although the early defect detection and prevention methods we will discuss all have costs associated with them, they are demonstrably less expensive than removing the same defects later in the software lifecycle. So you can reduce costs through defect prevention and early defect removal, while at the same time producing more salable or usable software with fewer delivered defects. You enhance your career and value to your employer by exercising professional responsibility and managing your own quality.

Quality of Work-Life Issues

Practitioners of Personal Quality Management have also found that it has a positive impact on the quality of their work-life.

Most of us went into programming because we enjoy building things and seeing the results of our labor. We like the challenge of figuring out how to attack a problem, of designing the structures and algorithms that will get the job done, and even of writing the code to make it all happen. And we like to see the fruit of our labor integrated into a system that meets people's needs or provides an important service.

But between the coding and use, there is a long and (for most of us) painful time of fighting our way past defect after defect. It starts with the compiler leading us around by our noses, complaining about every little typo and syntax error, and often totally misinterpreting what the code was supposed to say. Then after achieving a clean compile, we try over and over again to get the program to do the right thing. It loops, it hangs, it crashes; and now the debugger is the thing that is dragging us around by the nose. For each problem we fix, there seems always to be another one lurking in the shadows. And even more frustrating is that too often our fix was the cause of the ensuing problem!

But releasing the software doesn't end this frustration. Right when we're in the middle of designing the next interesting program, we get interrupted to fix a latent problem. There is a fuming customer and an unhappy boss, so we must drop everything to make an emergency fix. And to add insult to injury, the interruption of our design work is likely planting the seeds for future problems and their interruptions and frustrations.

Personal Quality Management allows us to gain control over defects. Instead of being ruled by the defects in our software, we can gain the upper hand. We can understand them, find them, remove them earlier and even avoid them altogether. That means that we will spend a much greater proportion of our time on the tasks we enjoy, and less on the problems we missed. That is a significant improvement in our work-life!

Results in the Classroom & Field

Personal Quality Management is a significant component of the PSP. A growing body of evidence shows that the PSP significantly improves the engineers' ability to remove defects early without compromising their productivity.

The SEI published a study of the data from 298 PSP students [Hays] that showed the differences in programmers' performance as they progressed through the class. Among other things, they removed more than five times as many defects before testing by the end of the course than at the beginning,

Companies that have adopted PSP have confirmed that these dramatic improvements can be achieved in business. Teradyne, Advanced Information Services, Hill Airforce Base and Harris Communications have publicly reported dramatic results. (See my web site at www.ASKProcess.com for a relatively complete compilation of published results.)

Boeing reported the most striking among the results at the 2000 SEPG Conference, when they compared their first project by PSP-trained engineers with the prior three projects from the same group. In spite of being significantly more than twice as big as the prior releases, system test was complete in less than a week, as compared with a month or more for the others. Boeing attributes this 94% reduction in test time to the 75% reduction in the number of defects that were found in system test. PSP's early defect removal paid significant dividends.

Although there is significant variation from one organization or individual to the next, the available data suggest that reasonable expectations for software engineers would be to remove 75% of all defects before integration without extending the project schedule, followed by significant time savings during integration and system testing. And all of this is achieved while enhancing the professional satisfaction (and hopefully, recognition) of the software engineers.

Principles and Practice

Personal Quality Management is not a single thing that you can do. It consists of a number of interrelated activities that build upon each other, resulting in the dramatic improvements we just reviewed. The remainder of this article will examine these specific activities.

Personal Reviews

There is a popular belief that a software engineer can not review his or her own code. The PSP proves that this belief is false if the reviews are done correctly.

An effective code review is not just a matter of reading the code and hoping to find the problems that are hidden in it. Like many other activities, personal reviews are most effective when they are based on a well-defined and structured process. An engineer's personal review process must be structured to direct his or her attention toward the types of defects that he or she has historically injected. The PSP teaches engineers to scan their code multiple times, each time looking for a specific type of defect. This method of focused scanning is remarkably effective at finding defects, and is also quite efficient.

I had been doing personal reviews by simply reading my code for several years, but was finding fewer than 30% of the defects I had injected. After learning the PSP methods, my results were similar to those of many other people, with my personal reviews finding more than 75% of my defects! PSP-trained engineers will agree that personal reviews are definitely worthwhile!

Defect Logging

As was mentioned above, personal reviews will only be effective if they are personalized so they direct your attention to the defects you normally inject. This presupposes that you know your own defect history. As with anything else, informal memory is not likely to serve you well in understanding your defect history.

There is only one way to construct an actuate defect profile for your work: Log every defect that escapes the phase in which it was injected. The PSP suggests logging the following information for each defect:

• Type - To manage the large volume of defect data that you will quickly amass, you will need a way to categorize and summarize it. The next section will describe this step.
• Phases injected and removed - By identifying when each defect was removed and estimating when it was injected, you can understand which defect detection activities are working well and which need to be improved. (The phases identified by the PSP are Design, Design Review, Code, Code Review, Compile, Unit Test and After Development.)
• Fix Time - The number of minutes required to remove the defect (including research, debugging, reworking the design, reworking the code, compiling and re-testing) allows you to understand the magnitude of the problem posed by each defect.
• Fix Defect - If the defect was injected by fixing a defect, record a reference to the other defect so you can understand the types of validation that defect fixes require.
• Description - A concise description of what was wrong (not the symptom, but the cause) will allow you to recognize similar defects in the future.

Making a defect log entry takes only a few seconds, so it has little impact on your productivity. However, if you fail to log the information immediately, you are unlikely to be able to remember all of the defects and the data for each one. This information is critical to your ability to perform effective personal reviews, and defect prevention (which we will discuss later).

Defect Categorization and Analysis

Categorizing defects allows you to group them for convenient analysis. Without reasonable categories, your defect data will be a mass of data from which it would be hard to glean patterns. The PSP suggests a preliminary categorization scheme, but it also teaches engineers to modify that scheme based on their actual defect data.

Pareto analysis of categories helps each engineer to identify actions that are warranted based on his or her own data. For example:

• Re-working the defect categories may be indicated if there is:
• An ill-focused category into which many defects fall or
• A number of closely related categories that each has very few defects;
• Changes to your processes may be indicated in order to catch defect types that:
• Occur quite often or
• Require a lot of effort to remove.

The changes to your processes that can target specific defect types fall into two main categories:

• Early defect removal - Changes to your review (or other defect detection) processes can be designed to improve their effectiveness at detecting specific defect types, or
• Defect prevention - Changes to your design or coding processes can eliminate certain defect types. We will discuss this next.

Defect Prevention

Often, when we identify problems with the way we do our jobs, our first reaction is to resolve to try harder or work more carefully. While such resolve may produce some immediate improvement, in the long run just trying harder will not result in lasting significant change. As soon as other issues attract our attention, our resolve will refocus on those issues, and the initial improvements are likely to slip away.

Preventing defects requires that we make changes to the processes we use to do the job. Those processes then provide the continuous and unwavering resolve we need to make the improvements permanent, and prevent the problem we are attempting to eliminate from returning.

Once you have identified a defect type that is a candidate for prevention, you might eliminate it be doing something like this:

1. Focus on one particularly troubling type of defect. Trying to attack several problems at once is likely to doom your effort to failure. By attacking one single important issue at a time, you can maintain the focus that is more likely to result in success.
2. Identify the conditions under which that defect type occurs. Prevention requires that you understand why you inject that particular type of defect. When does it happen? Under what conditions? What normally are the circumstances at the time? Remember that defects are the result of your human errors. So we are searching for the root cause of those errors.
3. Consider how to change the process you use. Your objective is to eliminate the defects that result from your errors. You can accomplish this in one of two ways:
• Eliminate the error. For example, I eliminated begin/end mismatches by changing my coding process: I now write "Begin" and "End" at the same time, then go back and fill in the code between them.
• Change the situation so that your error does not result in defects. For example, you might avoid the consequences of forgetting to declare variables by using an editor that highlights undeclared variables as you type (much like the spelling checker in Word).
4. Test the process change and collect data. Don't just assume that the changes you decide on will be effective. Try them out on at least one pilot project and collect the necessary data to understand if they are effective. If they are not effective, return to step 3 with the new information you collected from the pilot test to try to develop a better solution.
5. If the process change is effective, permanently adopt it. Change your written process description, checklists or other mechanisms you may use to assure that you follow your new process. (If you have never written your process steps down, doing so may be your first step to avoiding errors!)

The sources of problems among different software engineers are as varied as their personalities. But from my experience teaching the PSP, I have discovered that for many software engineers, a highly effective way to prevent a variety of defects is to formalize the design process and/or design notation. Most of us spend insufficient time working through the details of our program designs, and this results in some of our most expensive and frustrating defects. The PSP recommends a set of design templates that will assure that all of the important parts of the design have been examined.

While defect prevention provides the greatest benefit, the reality is that we will not be able to prevent most of our defects. As we said earlier, because we are human we are prone to error, and those errors naturally result in defects. For this reason, the rest of this article will focus on ways to detect defects earlier in the development process, when they are less expensive to remove.

Review Checklists

Focused checklists are the single most important tool you can use to assure the effectiveness of both personal and peer reviews. While the review process itself will assure that you do the requisite things, only a checklist can ensure that you don't miss any important detail. Peer reviews will be discussed in the next section, so the remainder of this section will focus on checklists for personal reviews.

To get the maximum benefit from your personal reviews, your checklists must be focused and personalized:

• Your checklists must check for the defects you normally inject. That means that your checklists must be based on your personal defect history, and not on other people's performance or your organization's norms. Your checklist should check for compliance with organizational standards only to the extent that you violate them.
• Each of your reviews (high-level design, detailed design, code) needs a different checklist. Each checklist should include only the defects that you normally inject in the immediately preceding phase. Do not check for defects that escape prior reviews. (If prior reviews are consistently missing certain defects then fix those reviews.)
• If you use multiple programming languages, then you need different detailed design and code review checklists for each language. Although there is likely to be some overlap or duplication between checklists for different languages, it is important that each checklist fully and exclusively support the reviews for which you use it.
• Your choice of items for each checklist must be based upon Pareto or other analysis of your defect data. It would be infeasible to check for every possible defect. To make the best use of your review time, make sure you are checking for the defects that you are most likely to have injected. It is best to limit each checklist to one page. This means that you will be able to check for no more than 50 or so items in each review.

Just having a checklist doesn't automatically improve yourreview effectiveness. You must learn to use it systematically. For example, mostpeople find that their reviews are most effective when they scan the workproduct for one checklist item at a time, rather than reading through the workproduct once looking for all checklist items. While it may seem that scanningthrough the work product multiple times is wasteful, this turns out not to bethe case, because when you are looking for only one thing, you can scan at arelatively high rate.

The most important thing to keep in mind when doing personalreviews is to avoid rushing. The PSP teaches engineers to review code no fasterthan 200 LOC per hour. Most people's data shows that when review rates arehigher than this, effectiveness becomes spotty, and more defects escape thereviews. Your objective in the reviews is to assure yourself that the defectsfor which you are checking are actually absent from the work products. Thereview must be careful enough so that checking off a checklist item is apersonal certification that the defect type it represents is absent from thework product.

The final step in assuring the effectiveness of your reviewsis to periodically study your defect data to see what it says. Look for types ofdefects that are consistently escaping your reviews, then consider what it maybe about your review process or checklist that is allowing this to happen. Forexample:

• The defect type may not be represented on your checklist, suggesting that it should be added. If your checklist is already a full page long, this would mean deciding which checklist item must be dropped to accommodate a new one. Naturally, you should drop one that never or rarely results in identifying defects.
• If your checklist includes an item that covers this defect type, then you may want to re-word the item or split it into two or more items so that your attention is focused more precisely during reviews.
• Finally, the fault may lie with the way you use the checklist. A change to your review process may provide the improvement you need.

The purpose of your review checklist is to provide the focusthat will assure that the time you spend doing reviews is effective in improvingthe quality of the work products you produce.

Peer Reviews and Inspections

If your organization consistently does peer reviews oftechnical work, then you should make sure that you take full advantage of them.While personal reviews can find up to 80% of the defects in your work products,the time spent in peer reviews is still very much worth while. By combiningeffective peer reviews with effective personal reviews, you can remove more that90% of the defects from your software before the first test!

Peer reviews can take a wide variety of forms. The mostrepresentative of those forms are these:

• Walk-throughs involve the author walking through the work product and explaining what was done and why before a group of peers. The peers interject questions about the work product to understand it better and to identify defects. These reviews require the least time of any of the peer review types. However, they tend to be a rather poor vehicle for identifying defects, being more valuable for educational purposes and assuring that team members understand each other's work.
• Full reviews involve peers examining the work product independently, then presenting questions, concerns and defects to the author in some way. The feedback mechanism can be as informal as e-mail or as formal as a walk-through meeting. The problem with this review method usually lies in the reviewers spending enough time in independent examination. Even when a formal feedback meeting is held, peers will often have spent little or no independent time on the work product. The effectiveness of these reviews can be highly variable, and often depends on the level of stress in the organization.
• Inspections (Often called "Fagan Inspections") are the most formal of the peer review types. Participants receive training, are assigned formal roles and are required to fully participate in the process. Inspections are generally checklist-based, with each person spending time in independent examination. The formal meeting is not held unless all participants have completed their examination of the work product. The inspection meeting includes a reader (not the author) walking the reviewers through the work product, with the reviewers commenting or questioning on each part. Questions that can not be quickly resolved in the meeting are assigned to individuals for resolution, and all identified defects are entered into a defect tracking system to assure they are corrected.

The most effective of the various review forms tends to be formal inspections. Although they are more work to establish and require more of people's time to do, they are also the most effective at finding defects, so they result in lower over-all cost of quality. Even if your organization does not have a formal peer review program, you can still capitalize on this powerful tool by asking your peers for their opinions on an informal basis. Obviously, you can not do formal inspections this way, but you can ask your peers to do independent examination and even attend review meetings. Often these sorts of informal reviews result in the desire within the organization to institute a more regular and formal peer review program.

Personal Quality Planning

"If you don't know where you are going, any road will do.
If you don't know where you are, a map will not help"

Watts S. Humphrey

High quality at low cost does not just happen; You must plan for it! As suggested by Watts Humphrey, there are two parts to planning: knowing where you want to go, and knowing where you are.

"If you don't know where you are going, any road will do."

Most of us never consider Personal Quality Planning. We may be unhappy with the quality of our work. We may be aware of the damage that poor quality does to our reputations or our company's bottom line. We may wish we could spend less time debugging and fixing problems; but we tend to be unsure what we can do about these things. This article provides lots of ideas about things that we can do, but taking advantage of these ideas requires that we take action. Deciding exactly what actions to take can be difficult to do unless you have a plan.

First, you must decide what quality levels you want to achieve. While absolute zero defects springs to mind (and is a laudable long-term goal), it is not a reasonable goal to strive toward. After realizing that you are human and prone to error, you must decide on a goal that is aggressive and challenging but achievable with available time and effort. If you are unsure of what might be a reasonable goal, a good starting point may be some level of incremental improvement over your current defect levels. (See, "If you don't know where you are..." below.)

Merely setting goals doesn't change much. To make material changes in your performance, you must make material changes in your actions. You must determine what steps you must take to achieve your goals, and what day-to-day process changes those steps imply. Do you need to log and track your defects? Do you need to do reviews you currently skip? Do you need to change your approach to certain tasks?

"If you don't know where you are, a map will not help."

Even if you know the quality that you want to achieve, youwill still be lost until you know your current performance. What quality leveldo you normally achieve? ... at what costs? ... using what methods? If you cannot answer these questions, you will not know where you are falling short ofyour goals or what direction you need to move to reach them. You won't evenknow when you have achieved your goals!

So, you must identify important attributes of yourperformance. What do you need to measure, and at what level of detail? Forexample, if you are going to log defects, what pieces of information will beimportant or useful? What will you care about? What will help you to determinehow to improve your process?

Finally, you must actually measure your performance; collectinteresting data over time and periodically examine it to see how you areprogressing against your goals. If you are not progressing, what changes can youmake to your personal processes to improve your performance? If you areprogressing, do you expect the improvement to continue? And when you'vereached your goal, celebrate! Then check your long-term goals to see if you needto set a new goal and continue improving your work.

Improving your work is hard but rewarding work. It requiresgood plans, specific actions, and measurement of your progress. Without thesethings, your best efforts are merely a shot in the dark

Personal Process Improvement

We have been talking about improving our personal processes,but have not yet looked closely at what this means. Every process (personal orotherwise) has a natural capability and a natural range of variation. This iseasiest to see in a control chart like those used in manufacturing. (See thefigure.)

The solid black line indicates the capability of the process.This is the expected result of the process. The dotted black line indicates therange of variability of the process. This is the range around the processcapability within which the actual results generally stay. The red jagged lineshows the actual performance of the process in specific cases, with time runningfrom left to right.

The first step in improving a process is to gain control overit. A process that is out of control does not stay within an expected range ofvariation; it is essentially unpredictable. For most of us, project scheduleestimation is like this. Each estimate we make is wrong by a different andunpredictable amount, sometimes over, mostly under, and sometimes by severalhundred percent.

Gaining control over a process includes things likeunderstanding the steps in the process, assuring that we do them consistentlyand mitigating outside influences on the process. For many things, gainingcontrol will mean actually writing a description of the process steps, andcreating a checklist or other mechanism to help us to remember to do the stepsin the right order each time. Gaining control over the process results in itsvariability being limited to its natural capability.

Only after you have a process under control can you thinkabout improving it. Improving a process involves things like changing the stepsin the process, changing the way the steps are done, and enhancing the skillsand tools that you use in the process. The net result of improving a process isa narrowed range of variability (as indicated by the big arrows, above). Animproved process has a smaller range of variability, and is more predictablethan it had been before.

The important point of this discussion is that you can notimprove a process before it is under control. Your first focus must be to bringthe process under control. Only after it is under control can you improve it.

Using Your Personal Quality Plan

A Quality Plan is a waste of effort unless you use it tomanage quality. While the thought that goes into making the plan has value, thereal value of a plan is in the effect it has on your work. If you make a planand set it aside until the work is done, then you are losing the major value ofthe plan. The best use of your quality plan will be in validating each step ofyour process; that is, using your quality plan to determine if you have done asufficient job at each step in your process before you move on to the next step.

We tend to think of "validation" as being certaintypes of activities, like reviews or tests. In fact, any activity can andshould be validated, even review and test activities. This kind of validationallows you to manage what you do so you can achieve the desired goals.

For example, before moving on to the code review, we shouldtake a few minutes to validate our coding activities by asking questions like:

• Did I build all of the parts I intended to build?
• Is the product as big as I expected it to be?
• Do the pieces fit together as I intended?
• Do I need to take any corrective action?

With our answers to these questions, we know if we are readyto move to the next step and validate the code itself in our code review. Then,after the code review, we should validate the review by asking questions like:

• Did I find fewer (or more) defects than I expected?
• Did I spend as much effort on the review as I should?
• Is the defect rate consistent with my quality target?
• Do I need to take any corrective action?

By using our quality plan in these validations of the steps in our process, we can detect when things begin to go awry, and take corrective action before the whole project is out of control.

Post-Project Data Analysis

Whether your organization does post-project reviews (sometimes called "postmortem" or "retrospective" reviews), managing your personal quality performance will generally require that you spend some time with your personal data at the end of each project. A small amount of time at the end of the project can yield major rewards in the next one.

At the very least, you should take a few minutes at the end of each project to assure that all interesting data about the project has been recorded and stored in a safe place. The more time that has passed since you did the work, the less likely it will be that you will be able to reconstruct the data from memory. While you should always strive to record data as you work, you should use the post-project time to assure that nothing has been missed, and fill in any holes to the best of your memory.

The data you collect about your project can be a gold mine. Make sure you analyze it on a regular basis and understand what it is telling you. With the right data stored in a usable form, you can do many things. For example, use your data to:

• Plan your next project. Your best indicator of how you will perform on your next project is how you did on the last one. The best way to estimate product size, effort required and schedule is to compare the new project to others you have done and use your actual performance as a guide. (Don't expect your performance to be significantly different this time unless the project is very different, or you have made material changes to your process.)
• Set quality goals. In the interest of making incremental improvement in the quality of your software, you will want to use your historical performance as the basis for setting quality goals. But realize that each project will not be better than the prior one was. There will be a certain amount of variability, but you want to aim for improvement over the long run.
• Defend your plans. If your plans are base in historical fact, you will be better able to defend them to your management. Management will always pressure you to promise too much; but if you can show that you understand your capability and will deliver what you promise, they will learn to respect your estimates and plans.
• Identify candidate processes for improvement. When your data shows that a process is either out of control, or has too wide a range of variation, you should use that information to look for ways to change the process to bring it into an acceptable range of variation.
• Evaluate the effectiveness of process changes you have made. Each process change is not guaranteed to actually improve things. After making any change, you should carefully measure the results of the change to assure it had the desired effect and no undesirable side effects.

Personal Quality Management and the PSP

The Personal Software Process (PSP) teaches software engineers how to use a variety of disciplined practices, including the Personal Quality Management techniques described in this article. By learning to apply these disciplined methods, programmers can begin to make the transition from programming as an ill-defined craft toward software as an engineering discipline.

References

[Humphrey] Humphrey, Watts, PSP(sm): A Self-Improvement Process for Software Engineers, Addison Wesley, 2005

[Hays] Hayes, Over "The Personal Software Process (PSP): An Empirical Study of the Impact of PSP on Individual Engineers"CMU/SEI-97-TR-001

© 2000-2007 Copyright ASK Process, Inc.

---

Related Software Process Articles in Methods & Tools

• Don't Write Another Process
• Assessing Readiness for (Software) Process Improvement
• Process Improvement - Is it a Lottery?
• Mature Scrum at Systematic

---

Click here to view the complete list of archived articles

This article was originally published in the Summer 2007 issue of Methods & Tools