Click here to view the complete list of archived articles

This article was originally published in the Spring 2007 issue of Methods & Tools

Process Improvement – Is it a Lottery?

Paul Morgan, Technology Process Group, GTECH Corporation

Abstract

If process improvement is a journey then the experience ofmost organizations is that the path is a rocky one and the places encountered onthe way are seldom the ones originally sought. A major contributing factor tothis problem arises from the difficulties that organizations face when adaptingtheir chosen process improvement model to their ‘real world’ situation.Models such as the Capability Maturity Model (CMM^®) (1993) andCapability Maturity Model Integration (CMMI^®) (2002) are generic bydesign, not prescriptive, and extensive work is required to ensure that duringimplementation they contribute positively towards an organizations businessgoals. Immature organizations are most at risk of allowing the model rather thantheir business needs to dictate the nature of their process improvement initiative.

GTECH is a global technology company operating in over 50 countries, providing software, networks, and professional services that powerhigh-performance, transaction-processing solutions. This article provides anoverview of the approach utilized to implement process improvement across itsglobal organization without losing focus of its business drivers. It provides apractical overview of how over a four year period an organization moved from CMM^®Level 1 to Level 3 and is currently transitioning to CMMI^® Level 4.It will provide a candid insight including lessons learned and approachesadopted to achieve success. It will also provide examples of significant andmeasurable business benefits that can be accrued from adopting a documented and repeatable process improvement framework.

Introduction

"Not everything that counts can be counted; not everything that can be counted counts." Albert Einstein

In 1994 the Standish Group, undoubtedly with tongue in cheek, named their original research paper into the state of IT project management The CHAOS Report (1994). Whilst the dire situation described in that original report has undeniably improved over the intervening years there can be few people for whom the humour in that original title has not turned sour. A follow-up report published by the Standish Group entitled Extreme CHAOS (2001) states that still only 28% of IT projects are completed on time, on budget and with all the features/functions originally specified. Of the projects that were successful 97% had a project manager assigned as opposed to 79% for those that were unsuccessful. Whilst this statistic undoubtedly adds weight to the argument that assigning a project manager enhances the chance of success it does not answer the question as to why so many IT projects, even when run by a dedicated project manager, ultimately fail to meet their original objectives.

This article will describe the steps taken to avoid such failures by institutionalizing project management good practices across its global software development organization. It will explain how a process improvement framework was utilized to translate generic project management practices into specific processes and procedures more relevant to the development of software. In other industries, such as construction, the wellbeing of the project can be assessed using all of ones senses from the moment the first foundation is laid. Due to the intangible nature of a software product, and more specifically its component parts as they are transformed through the development cycle, such faculties are either not available or considerably dulled for those assigned responsibility for managing a software development project. Our experience indicates that this constraint, inherent in all software development, can be successfully mitigated by adopting a software process improvement framework wherein management and engineering activities are integrated to establish process controls that provide the prerequisite visibility. Further, by aligning this approach with the business goals of the company the benefits realized by individual projects can be leveraged across the organization.

Process Definition

The Model

The CMMI^® was developed by the Software Engineering Institute (SEI) based upon best practices observed within successful development organizations. (Exhibit 1) We adopted the staged version of the model with each successive level demonstrating an evolutionary plateau that establishes a new capability for performing business functions. This capability realised in terms of organisational maturity, results in improved productivity and quality, and reduced project risk. Each level is broken down into a number of process areas that identify a cluster of related activities considered important for establishing process capability at that maturity level.

Exhibit 1 – Capability Maturity Model Integration (CMMI^®) (click to see full size)

Level 1 of the model symbolizes those organizations that operate within an unstable environment that lacks sound management practices. Project commitments are typically not controlled and success rides on individual talent and heroic effort. Practitioners argue against engineering discipline under the guise of "individual creativity". Those standards and practices that do exist are often sacrificed to management priorities. Process capability is unpredictable with schedule, cost, and quality targets often missed. A level 1 organization might unconsciously do some of the practices expected at higher levels of maturity but they do not get repeatable results, even if they are perceived to be effective.

Level 2 of the model is referred to as "Managed" wherein policies for managing a software project and procedures to implement those policies are established. Realistic project commitments are made based upon previous project results and the unique requirements of the current project. Management tracks software costs, schedules, and functionality against plans. When problems are identified appropriate corrective actions are taken. Project artefacts are base-lined and their integrity controlled by exercising disciplined configuration management. Strong customer/supplier relationships are developed and managed. From an organizational perspective, projects are allowed to have their own unique processes. Level 2 recognizes that project management practices provide an essential foundation for effective software engineering.

Level 3 of the model is referred to as "Defined". A standard set of processes for developing and maintaining software is documented and used across the organization. This standard process includes both software engineering and management processes and integrates them into a coherent whole. Projects tailor the organisations standard process to develop their own defined software process, which accounts for the unique characteristics of the project. A group within the organization is assigned responsibility for software process activities. An organization-wide training program is implemented to ensure that the staff and managers have the knowledge and skills required to fulfil their assigned roles.

Level 4 of the model is referred to as "Quantitatively Managed". The organization sets quantitative quality goals for both software products and processes. Productivity and quality are measured for important software process activities across all projects as part of an organizational measurement program.

An organization-wide software process database is used to collect and analyse the data available from the projects' defined software processes. Projects achieve control over their products and processes by narrowing the variation in their process performance to fall within acceptable quantitative boundaries.

Level 5 of the model is referred to as "Optimizing". Technical, managerial, and process defects are measured and via causal analysis specific process activities are taken to prevent reoccurrences and improve the overall performance of future projects. Technological improvements are systematically discovered, analyzed and applied and continuous process improvement is formalized.

Documenting the Process

A common mistake made by companies when implementing process improvement is to allow the chosen model to dictate the process design. The CMMI^® is a model that needs to be interpreted based upon the business environment and technical needs of the project; it is not a standard that must be implemented exactly as documented. A failure to recognize this might still result in maturity levels being attained; however, the end product is unlikely to be a process suite which complements your operating needs. The correct approach is to balance the recommendations of the model against the business value that the new processes will add. This value added might be recognized by the individual (i.e. improved engineering methods), by the project team (i.e. more accurate estimation), or by the organization (i.e. metrics and project artefacts available for reuse); however, it should not simply be dictated by what the model suggests is good practice or what in principle is a good practice.

Another consideration when documenting your processes is that the majority of process improvement models are designed as auditor’s tools. As such they provide a useful checklist of practices which should be institutionalized within a mature organization but they are not written to easily support the project team in their daily activities. A common trap is for companies to also use the model as a framework for categorizing their process documentation. Separate processes are designed and documented for each process area resulting in a process library that will impress any auditor but which is of limited use for project practitioners whose responsibilities span across process areas. The result is often ‘shelf-ware’ that is seldom referenced and just gathers dust on the bookshelf; for those of you whose process documentation sit on a central server just do a check on the number of hits per day your library attracts.

In contrast we took a role based approach when designing its processes. This entailed identifying the key project personnel that were required to deliver a quality product and then for each designing and sequencing their activities and orchestrating them overall into a coherent workflow. Compliance with CMMI^® was a key factor in the process design but not at the cost of operational efficiency. The resultant documentation formed a ‘to do’ list that project personnel could follow on a daily basis. The emphasis was also on readability and usability. Great care was taken to ensure that process definitions were not verbose. Help text, guidelines, and training material were kept in separate files that could be referenced as needed. The global nature of the process deployment across six continents and seventeen time zones also required a process suite that was intuitive to use with corporate-wide visibility. The method chosen was an intranet based GUI solution that provides automated workflows to guide and control project personnel in their activities. The application also acts as a repository for project artefacts with full configuration management, risk management, and process history.

Process Deployment

The Process Improvement Project

Organizational change forms the basis of any process improvement initiative. Consequently alongside the normal project challenges of designing and deploying a new ‘product’, in this case new processes, there is the added human dimension of changing existing work practices. The sum of these two factors means that to ensure success the process improvement project must be run using the most rigorous of project management procedures. In CMMI^® terms, an organization cannot expect its project teams to function at high levels of maturity if the rest of the organization is still operating at level 1. Management at all levels must practice what they preach; this typically requires a radical cultural change.

To this end a dedicated process management team exists to manage the development, deployment, and ongoing maintenance of its processes. This Technology Process Group (TPG) was staffed by recognized discipline experts and reported into a steering group of senior managers who provided sponsorship, direction, and visible endorsement of the TPG activities. Quarterly progress reviews were held to provide the necessary oversight.

On an annual basis the TPG develops an organizational process improvement plan based upon the business goals of the company. This plan outlines the process improvement objectives for the following year, the approach that will be taken to achieve those objectives and the resources required. Once approved by the steering group this plan is then broken down into manageable tasks and activities to form a Work Breakdown Structure / Schedule that is used to track progress. As with any other project changes to the original requirements do occur and these are controlled by maintaining the developed processes under full configuration management. An independent Standards Compliance function was also established to ensure that the TPG was abiding by its own rules.

Overcoming Resistance to Change

There are a number of factors to consider when implementing a process improvement program not least of which is the resistance to change that you will undoubtedly encounter. Force field analysis is a management technique developed by Kurt Lewin for diagnosing such situations (1946). Lewin assumes that in any situation there are both driving and restraining forces that influence the success of any proposed change. The TPG conducted just such an analysis at the commencement of its process improvement program. (Exhibit 2)

The driving forces for process improvement were provided by the close mapping to business goals, senior management sponsorship, and the consensus by all stakeholders for improved customer satisfaction, productivity, and reduced costs. The restraining forces were less easy to quantify and consisted of a spectrum of people issues from overcoming a natural inertia for change, through issues of self-interest, to a lack of understanding. In a situation such as this just increasing the pressure of the driving forces, for example through management dictates, seldom works. At best a reluctant compliance is achieved and at worst no change occurs as individuals and teams dig their heels in and passively resist. The solution is to understand the reasons for opposition and apply management techniques to address concerns and thus weaken resistance.

At the start of the process improvement initiative, stakeholders were identified and process improvement plans and objectives were negotiated and agreed. Individual performance goals were then established for senior management and these were cascaded down through lower levels of management to the engineering level. Initially simple measures of success were used such as the percentage of personnel trained on the new processes and the adherence to process compliance. Later more sophisticated measures relating to project performance were introduced.

The next technique was to involve the practitioners, those who would use the processes on a daily basis, in the process design. This participation and involvement not only ensured that the processes deployed would meet operational needs it also provided them with a feeling of process ownership. It therefore precluded the feeling that the new processes were being imposed upon them and avoided a potential source of resistance. This involvement is an ongoing theme of the process improvement initiative. A formal mechanism remains in place for anyone on a project to propose a justification for tailoring and customizing the standard processes.

To facilitate and support the institutionalisation of the new processes dedicated TPG resources are located in each development centre. These individuals provided mentoring and coaching support to assist the adoption of the new processes into the working environment. Specialist discipline support in areas such as estimation techniques and configuration management are also provided as needed. A standards compliance function was also added to every project to ensure that processes were being followed. To address any potential resistance such as "I’ve not got a budget for that" all TPG resources were provided at no direct cost to the projects.

A key factor when rolling out process level improvements is to ensure that the people you expect to follow them have the necessary underlying skills. An extensive organizational training program was rolled out concurrently with the process improvement program. This education initiative included training such as; process and CMMI^®, project management (i.e. PMBOK), discipline specific, tools usage, and soft skills (i.e. team building). A variety of delivery methods was utilized including; instructor led, web based, and ‘Train the Trainer’. The latter of these consisted of empowering the project teams themselves to take on responsibility for their own training.

Vital to the success of any project is the ability to maintain effective two-way communication channels. The TPG utilises periodic newsletters and its own website to distribute general information on both project issues and successes. Affinity groups were established to facilitate discussion on specific process areas. As processes were modified and updated formal release notes and delta training were utilised to ensure a smooth transition. Meetings with all levels of management were effectively used for the purposes of both progress reporting as well as to address in a timely manner any issues that had arisen.

Finally, despite every effort to facilitate change, some people and/or teams may refuse to conform. In such circumstances, and as a last resort, management intervention might be the only option. The reassignment of staff can be one alternative so as to prevent the process improvement initiative from stalling.

Process Deployment Costs and Benefits

Process Deployment Costs

Process improvement does not come free of charge although the investment made does reap major rewards. Expenditure is firstly required for the recruitment and training of personnel to manage the process improvement program. Processes need to be designed and documentation created. Deployment requires the production of training materials and the allocation of instructors to conduct classes. Support personnel are required to assist project staff climb the learning curve associated with the implementation of new working practices. External consultants are needed to help fill knowledge gaps and conduct assessment services against the model to judge progress. Opportunities might also exist to purchase tools to help with automation. However commercial off the shelf (COTS) products will, almost as a rule, require customisation to your own needs and thereafter be maintained.

A study by the Software Engineering Institute (SEI) Benefits of CMM-based Software Process Improvements: Initial Results (1995) suggests that the yearly average cost for companies undertaking software process improvement was US$ 1,375 per software engineer. A more recent survey CMMI Market Trends (2005) indicates that companies who invest between 5% and 7% of their development budget have the greatest chance of sustaining a successful process improvement program.

Process Deployment Benefits

The Software Engineering Institute (SEI) Benefits of CMM-based Software Process Improvements: Initial Results (1995) also indicated the rewards that can accrue from a process improvement program. (Exhibit 3) It should however be noted that such benefits usually lag investment by months or years. Greater returns are realized as the experience of operating at a particular maturity level increase.

Category	Range	Median
Productivity Gain per Year	9% - 67%	35%
Early Defect Detection Gain per Year	6% - 25%	22%
Yearly Reduction in Time to Market	15% - 23%	19%
Yearly Reduction in Post-Release Defect Reports	10% - 94%	39%
Business Value (savings/cost of SPI)	4.0 - 8.8:1	5.0:1

Exhibit 2 – Benefits of Process Improvement

Less quantifiable, but equally valuable, business advantages are also achieved. Firstly deploying a standard project process is recognitionthat an organizations development practices are valuable intellectual propertythat must be defined, documented, and secured. If an individual leaves anorganization and takes with them undocumented knowledge disruption and the needfor reinvestment can result. Standardizing project processes irrespective ofwhere development work takes place also provides for; more effective resourceutilization, faster project start-up and less re-training, improved teamwork andemployee morale, and increased customer confidence as project successes becomerepeatable. The availability of a historic project information repository alsoprovides a knowledge base that facilitates not only the reuse of work productsbut also enhances project management capabilities by rendering estimations basedupon accurately identified historical evidence. Quantitative managementtechniques can also be utilized to enhance risk management via exceptionreporting by benchmarking current project performance against establishedhistoric norms (i.e. defect detection profiles). Finally, the visibilityafforded by an intranet based process improvement implementation facilitates real-time management decision making.

Final Words

Here are some final words of advice for anyone that is considering travelling along the process improvement highway. Whilst they maynot guarantee that you reach your desired destination they will prevent you fromfalling into some of inevitable potholes that you will encounter along the way.Firstly, the time to achieve a maturity level depends entirely upon the extentof senior management sponsorship. The commitment required must be more than justproviding the necessary budget. Senior management must be seen to proactivelysupport the process improvement program. When something goes wrong, asinevitably it will, they must support an adherence to the deployed process and not resort to crisis management techniques.

The TPG must be staffed with recognized leaders and discipline experts. Reassigning specialists off projects might in the short termappear to be folly but will over time earn considerable dividends in terms ofthe quality and uptake of the processes deployed. The process improvementprogram must be treated as a high priority project with enforced accountabilityand high-visibility status reporting. Treat the deployment of processes as if it was a product delivery to your most demanding and important customer.

Do not adopt the maturity model as your process. All process improvement models need to be interpreted based upon the specific needs of yourorganisation. It is your responsibility not that of the model to tell you how your business should be run.

Use the best technology available to develop and deploy your processes. However, do not be seduced by the allure of a ‘silver bullet’solution. Just as you cannot loose weight by buying a set of bathroom scales anorganization cannot become more efficient by simply purchasing a tool. Anindividual has to change their lifestyle to see results and an organisation mustchange its work practices to recognise benefits in terms of improved productivity and quality.

Ensure accountability via standards compliance and periodic assessments (internal and external) to check progress. Then report results toall stakeholders. The use of external consultants helps avoid blind spotshowever do not abdicate responsibility to them for the process improvementprogram. You know your business needs better than they do and at the end of the day accountability for success or failure rests with you not them.

References

• Capability Maturity Model^® Integration (CMMI^SM), Version 1.1 (2002). Software Engineering Institute, Carnegie Mellon University, Pittsburgh, Pennsylvania 15213
• Key Practices of the Capability Maturity Model^SM, Version 1.1. Software Engineering Institute, Carnegie Mellon University, Pittsburgh, Pennsylvania 15213
• The CHAOS Report (1994). The Standish Group International, Inc.
• Extreme CHAOS (2001). The Standish Group International, Inc.
• Frontiers in Group Dynamics (1946). Kurt Lewin
• Benefits of CMM-Based Software Process Improvement: Executive Summary of Initial Results (1995). Software Engineering Institute, Carnegie Mellon University, Pittsburgh, Pennsylvania 15213
• CMMI^SM Market Trends (2005). Andrew Griffiths, Lamri, CMMI^SM Made Practical Conference.

Back to the archive list