Quantified Objectives: A Fundamental Advance in Managing Software Development Projects - Page 3

Stuart James Woodward, DoubleHelix Software & Services Limited, http://www.doublehelix.co.uk/

A Shift of Focus to Performance Requirements

Functional Requirements

Just as on OMAR, and in common with nearly all requirements specifications and statements today, the CRM project requirements were concerned with what are called Functional Requirements.

The difference was that, in common with most cases of implementing software systems that aid business functions in commercial organisations, we were not building something totally unique or even new within the organisation. We were implementing business functions within software systems in order to improve those business functions, and the functioning, or performance, of the organisation as a whole. This alone provided me with the reason why it was not only the functionality that should have been the project driver; it was the desired performance levels of the existing functionality that should have determined the justification of the project, its planning and how it should have been monitored.

Occasionally on projects, (though not on CRM), there are specifications that make mention of what are often called Non-Functional Requirements. Some development methodologies even use this exact term although they rarely, if ever, tell you how to specify them in a way that can later be verified.

The term Non-Functional Requirements is somewhat perplexing and, frankly, I propose that we banish the term. We must seek to specify requirements in terms of ‘what they are’, not ‘what they are not’, else how can we ever test if the intended requirement has been met? Non-Functional suggests that these ‘other’ requirements are somewhat less important than or secondary to the functional requirements, perhaps because they are usually described in long, textual documents.

However, these Performance Requirements (to use a more accurate term), are rarely specified well enough, if at all, and rarely tested against when the systems are built. Most importantly, they are usually the real reasons why software development projects come into being in the first place: they are the key objectives.

Performance Requirements

Performance requirements can always be specified precisely as "desired future end states: testable and measurable" [GILB97]. It is, therefore, these performance requirements which can determine the success or otherwise of our projects.

However, they are not necessarily easy to identify. To find them, it is worth considering the primary objectives of the organisation and then how these objectives can be reached. If you can answer this question then you may be able to identify your key project objectives.

The Identification of Performance Requirements

One possible way is to think of the general characteristics of these objectives. For example, you may wish to make better X, save Y, or do more of Z. Put more formally, performance requirements might be thought of as broken down into the three following types: Quality, Resource Saving, Workload Capacity.

A useful sub-division of Performance Requirements (picture adapted from [GILB04])

This categorisation of specific requirements could depend on the perspective of the stakeholders. It is a first level decomposition, a convenient sub-division, and ultimately will not matter when the performance requirements have been fully specified. Therefore, we might consider that the gist of the underlying requirements from CRM’s Credit Approval Process might have included:

• Credit Information Response - the assembling of all related credit information (Quality).
• Credit Request Cycle – the prompt processing of each individual request (Resource Saving), and
• Credit Request Capacity – the handling of all the expected requests (Workload Capacity),

Again, these are deliberately defined in imprecise terms here (but will be specified clearly later in this paper).

The question for the CRM project then, should have been whether it could have improved on the existing, manual system by meeting these underlying requirements. If it could not have done then, in the absence of other requirements, there could have been no economic reason for having proceeded with the project; it may have been nothing more than a "nice to have".

But how could we have known this before the project proceeded and how can we know likewise for future projects?

Key Stakeholder Objectives

We must learn to focus on key stakeholder objectives. These include the system’s functionality, of course, but it is the performance requirements that are usually the key project drivers.

We need methods that:

• Concentrate on identifying the underlying objectives and requirements without assuming specific functional designs.
• Specify objectives in unambiguous and measurable terms of performance levels and costs.
• Provide a cost-effective basis for assessing what to do at each stage in a project and, importantly, when to stop.
• Enable a comparison of the performance to cost ratios of solutions that are competing for resources.
• Enable the management of multiple requirements simultaneously.
• Enable Project Managers to objectively monitor and report progress towards the key objectives, which are agreed by all stakeholders.

Using Planguage to Manage Quantified Objectives

Requirements Specification

The aim of any project is to meet its requirements. To do so we need to use the resources that are available in order to meet the functional and performance requirements. This is an obvious statement but project managers often feel the pressure from senior management in organisations demanding more functions and higher performance levels from the same limited resources.

We therefore specify performance and resource requirements as well as functional requirements. The project manager can then prioritise the delivery steps based on their relative Return on Investment.

Prioritisation is evaluated by calculating the performance to cost ratios for each design idea proposed that contributes towards meeting the requirements. To do this, an estimate must be made of the extent to which each design idea will meet each requirement i.e. its impact. The performance to cost ratios can then be compared in order to ascertain which design provides the best value for money or Return on Investment.

Once the functional, performance and resource requirements are specified and the relative performance to cost ratios calculated, they can be used to plan and monitor the project. This explains the inextricable link between requirements, and project planning and monitoring.

At any point in the project, a reassessment of the requirements and the design impacts will allow the project manager to re-prioritise the delivery steps. This is a key advantage of Evolutionary Project Management.

The Specification of Functional Requirements

Functions that represent the functional requirements are either present or not present in a system. By function we mean something that is an essential part of the system. In this respect they are binaryi.e. there can be no degrees to which a function has been implemented or not. For the purposes of this paper we are not interested further in the specification of functional requirements, merely that functions are to be implemented or enhanced in the system. We are specifically interested in the performance attributes of the system.

The Specification of Performance Requirements

In contrast to functions, performance levels are scalar in nature i.e. there are degrees of performance. To illustrate this in a simple form, we shall specify those identified underlying requirements of the Credit Approval Process (CAP) within the CRM system.

When considering performance levels we consider the key requirements that we wish to address and then provide an unambiguous and testable specification for each one. For CAP, we shall consider that we wish to implement a system that will improve the following:

• Credit Information Response e.g. the system must be able to assemble all related credit information as quickly as possible.
• Credit Request Cycle e.g. the system must reduce the average time it takes to process each credit request
• Credit Request Capacity e.g. the system must be able to handle the expected number of credit requests

Note that we have still not provided specifications at this point.

There may well have been other key requirements, of course, but for the sake of this discussion we shall assume that these were the only requirements of interest. Even if others were deemed more important than these then it does not matter; the principle remains the same. For each requirement we then provide a measurement specification so that we can state the system’s current level, specify its desired level and test its level in the future.

This way we can determine the degree to which the requirement has been met by any solution that is implemented as part of the system. At the very least, by appropriate tests we can publish the current state of the system’s key performance attributes.

Here are the proposed performance requirement specifications:

Credit Information Response Ambition: At least an order of magnitude reduction in the average time taken to assemble all related Credit Information in order for a Credit Request to be processed at each state in the Credit Request Life-Cycle. Scale: The average time taken in minutes to assemble all related Credit Information in order for a Credit Request to be processed at each state in the Credit Request Life-Cycle. Meter: Timings will be obtained automatically from the CRM system upon invocation of each function that assembles all related Credit Information for each Credit Request. Past [Manual System, December 2002]: 60 ß Guess based on hearsay. Goal [December 2003]: 2 ß CRM Project Proposal [May 2001]. Goal [December 2004]: 0.5 ß CRM Project Proposal [May 2001]. Credit Request Life-Cycle: Defined As: The full collection of possible states in which a Credit Request can be. Credit Information: Defined As: For example, for the Counterparty specified on the Credit Request, its associated set of limits and exposure information, its rating information, rationale for the Credit Request, similar information for related Counterparties <and other data to be specified>.

 

Credit Request Cycle Ambition: To sharply reduce the average time taken to handle a Credit Request. Scale: The average time in hours it takes for a Credit Request to change state from an Initial State to a Terminal State after being processed by Trained Users. Meter: The timing data will be available from the audit trail of all Credit Requests in the CRM system. Past [Manual System, December 2002]: 60 ß Data from paper Credit Request documents [January-December 2002]. Goal [December 2003]: 48 ß CRM Project Proposal [May 2001]. Goal [December 2004]: 24 ß CRM Project Proposal [May 2001]. Initial State: Defined As: The state of "Open" of a Credit Request. Trained User: Defined As: A user of the system that has received the necessary instruction in CRM system use in order to perform his/her role correctly.

 

Credit Request Capacity Ambition: A large increase in the number of Open Credit Requests that can be handled by the system simultaneously. Scale: Number of Open Credit Requests that can be handled simultaneously. Meter: The count of Open Credit Requests will be provided as a system function, and the result will be available within 60 seconds of request. Past [Manual System, December 2002]: 5 ß A count of dated paper documents. Goal [December 2003]: 100 ß CRM Project Proposal [May 2001]. Goal [December 2004]: 500 ß CRM Project Proposal [May 2001]. Open Credit Request: Defined As: A Credit Request that has been entered into the system and is not in a Terminal State. Credit Request: Defined As: A formal document requesting authority to trade with a specified counterparty up to a maximum specified monetary amount. Terminal State: Defined As: One state of {"Withdrawn", "Rejected", "Processed to Completion"} of a Credit Request.

These specifications are written in Gilb’s Planguage. There are six essential parts to each specification as follows:

• Tag: A unique identifier for a Planguage or user-defined term, which includes performance requirements e.g. Credit Request Capacity
• Ambition: An informal description of the intended performance level requirement
• Scale: The units of measure in which we intend to specify the level of performance
• Meter: The method by which the level of performance will be measured on the specified scale
• Benchmarks e.g. Past: An actual measure of existing or past performance levels on the specified scale
• Performance Targets e.g. Goal: An estimate or desired target level of performance on the specified scale.

There are many other possible elements and subtleties in Planguage specifications, too, but their explanations are not needed here.

The specifications can include definitions of terms that require explanation in order to fully understand the performance requirements. Defined terms, including the performance requirements that have been specified, are underlined in this paper for emphasis.

Qualifiers in square brackets ‘[ ]’ add detail in terms of places, times, or events. An example of this from a Past parameter in the above specifications is [Manual System, December 2002].

So, for example, the above specification for Credit Request Capacity states that 5 dated paper documents from the Manual System were counted in December 2002. The requirement is for the new system to handle 100 Open Credit Requests simultaneously by December 2003 and 500 by December 2004. These unambiguous statements specify the performance requirements for Credit Request Capacity.

Note that it is legitimate for the specified performance levels to be best guesses, if no more objective information is available from any other source. The left-arrow symbol ‘ß ’ provides sources for any information stated in the specification. Sources can be useful to verify anything stated. ‘Real’ or more objective numbers might be found by testing. Targets can be set by agreement between the stakeholders on their desired requirements.

From these specifications it can be seen that to test and report the degree to which the performance requirements are met is straightforward. For example, if solution A really lowers the measurement of Credit Request Cycle from 60 to 54 then it has met the planned requirement for December 2003 by 50%; if it improves from 60 to 51 then it has met this same requirement by 75%.

By using these specifications and measurements, multiple requirements can be managed simultaneously. This will be shown below.

Go to page 2    Go to page 4    Back to the archive list