Forecasting: Asking Why and Discovering What is Behind the When

John Yorke, WWT Asynchrony Labs, http://www.asynchrony.com/

A product owner or development team will often be asked for an estimate or a forecast for a product, a feature, an iteration, or a story. But when you go to a team member and ask them it is not uncommon for the color to drain from their face, twitching to start, and their pulse to race.

Past experience says that the next words they speak will haunt them for the foreseeable future, maybe even for the rest of their career. It is only a rough estimate you say to reassure them, but they just know you have other plans and the fate of mankind lies in the balance. Or so it seems.

Forecasting vs. Estimating – The Differences

For most practical purposes there are only subtle differences between a forecast and an estimate. The main difference is that a forecast deals exclusively with the future. When I think of the two I tend to think of an estimate as information and a forecast as an expectation. It is likely that I will use estimates to create my forecast and I may use multiple estimates and even apply other factors to derive my subjective forecast. Whereas an estimate is generally an isolated objective assessment. Both have huge degrees of variation; accuracy, precision, and risk factors, but in some instances it may very well be that my estimate and my forecast are the same.

For example, if I was asked to estimate a journey, I may say that it is between 15 and 45 minutes and on average it is 30 minutes. If I am asked to forecast when I will arrive, that requires knowing not only the estimate but the time the journey starts and if there are other factors such as a need to stop to get fuel, money, or food. I may also add a certain weighting to the journey based on the time of day. Thus, my forecast, while based on an estimate, may not be the same as the “estimate.” Other times though I may simply take the estimate and it will become my forecast.

In practice the terms are used interchangeably and often makes little difference, except when it comes to expectations. If I ask “How long will this take?” I am asking for an estimate, but if I ask “When will this be done?” I am asking for a forecast. Both are fine so long as everyone knows what is meant by the question.

Forecasting is Misunderstood

Forecasting is guesswork, it may be scientific guesswork and it may be based on past experience, metrics, and clever projection tools, but it is a guess. You will be wrong far more often than you are right. The more professional, clever and precise the forecast looks the more confidence you may instill in that guess. But it is still a guess, and when your audience gains undue confidence in your guess they have a tendency to believe it as fact and not forecast. It might be that a guess is all that is needed but dressing up a guess and delivering it with confidence may create a perception of commitment.

A forecast is commitment (in the eyes of the one asking)

Generally, when giving a forecast it can be perceived as an implied commitment, even if you give a caveat. If at any, point where you even hint at a commitment to delivery to a fixed scope, fixed cost AND fixed date, you are setting yourself up for disappointment. And sadly that is what a forecast is seen as by most people. By all means, work to a budget, a schedule or even a scope (although that is very likely to vary) but ideally ONLY one and never all three.

Why is a Forecast Even Needed?

There are many reasons why people ask for forecasts. Understanding why is the first step to providing the right information, and hopefully changing the conversation. Forecasting for stories and features is both more reliable and more accurate, but the project/product level forecasting is where there is the most confusion and least understanding.

If possible, try to change the question of “When?” to an understanding of “Why?”

Generally speaking, we gather information to help us make decisions. If the information will have no bearing on our decisions, then the gathering of that information is wasteful. Forecasting all too often falls into this category. We take time and effort to produce a forecast and yet the forecast has no bearing on any decision, that effort was wasted, or the level of detail in the forecast was unnecessary for the purpose for which it was used.

Another issue of importance is when someone makes predictions of unknown work, based on incomplete information, and a variety of assumptions, can lead to poor decisions. This can occur when the questions being asked are not directly reflective of the decisions being made.

In my experience the Why? generally falls into three broad categories

1. How long will this take?
2. How much will this cost?
3. I am simply curiousy

But most people don’t ask why, they spend time creating a forecast and present it without knowing how it will be used. Some people asking for a forecast/estimate may not even know why they are asking, they just always get a forecast. But let us look at this a little deeper and explores routine questions asked and answers typically given.

1. The question asked: How long will this take?

When you ask back: Why does it matter? The typical responses are:

1. I need to plan
2. I have dependencies
3. I need to prioritize
4. I have a deadline

2. How much will this cost?

When you ask back: Why does it matter? The typical responses are:

1. I want to know if I will get a good Return on Investment
2. I need to budget
3. I need to prioritize
4. I have limited available funds

3. Curiosity/Routine

When you ask back: Why does it matter? The typical responses are

1. We always ask for a forecast; I need to put something in my report
2. I want reassurance that you know what you are doing
3. I want to know if my project is on track

What you will notice about these questions is that when you ask why, the first request for a forecast suddenly doesn’t make sense anymore. They are not really interested in the forecast itself, but in some other factor that they can infer from the forecast. If the intent is clear, then the question can be tailored to get the required information in a better way

Example One: “I need a forecast – Why? … I need to know when I can allocate staff to the next product/project.

In this case, would a simple high level guess be sufficient? I feel confident that staff won’t be available for the next 3-6 months, in 3 months let’s review, I’ll have a better idea then…I could put a lot of effort into a detailed forecast but an instinctive response may give all the information needed, saving us a lot of time and trouble.

Example Two: I need a forecast – Why? … I want to ship this to maximize the Christmas shopping period – or I want to time the launch for a trade show etc. This isn’t a request for a forecast, this is a request for an assurance that there will be something suitable available for a particular event/date. I can give you an assurance and confidence level without a detailed forecast, I may even change the priority of some features to ensure that those needs are factored into the product earlier. Or can de-scope some features to meet a certain date.

Example Three: I need a forecast – Why? … I have limited funds available, and I want to know when I can start getting a return on this investment. This isn’t a request for a forecast, it is a request to plan the product delivery so that revenue can be realized sooner and for the least investment. It may be possible to organize delivery so that future development is funded from delivering a reduced functionality product early. Or that development is spread over a longer period to meet your budget.

Example Four: I need a forecast – Why? … I need to budget, the way this company works I must get approval for my project expenses and staffing in advance so I need to present forecasts of costs and timelines. This answer is twofold, first – can you challenge the process? It might be better to have a fixed staffing pool and prioritize products/projects such that the most important ones are done first and then move on to the next, in which case the forecast for this is irrelevant, it is a question of prioritization. Or if the issue is ensuring staffing for the forthcoming year could I simply say whether this product will not be completed in the next budget year.

Example Five: I need a forecast – Why? … I want confidence that you know what you are doing. This is not a request for a forecast it is an assessment of trust in the team. There are many more reliable ways to ascertain confidence and trust in a team than asking for a forecast.

Example Six: I need a forecast – Why? … I have a dependency on an aspect of this project. This may not be a request for a forecast of this project, but more a request to prioritize a dependency higher so it is completed sooner to enable other work to start.

Example Seven: I need a forecast – Why? … I want to know if my project is on track. Essentially what you are saying is that I want to track actual progress of work done, against a guess made in advance that was based on incomplete information and unclear expectations. And I will declare this project to be ahead or behind based on this. I am sure those of you reading this will know that what you are measuring here is the accuracy of the original guess, not the health of the project. But we have been doing that for decades so why stop now.

Finally, this last example is the closest to a genuine need for a forecast

Example Eight: I need a forecast – Why? … I need to prioritize or I want to know if I will get a good Return on Investment.

All of these are very similar questions, but are really requests for estimates and not forecasts. A rough estimate helps me gauge the cost and when I evaluate that against my determination of the value expected to be gained from the project, it may help me decide whether the project is worth doing at all or if there are other projects that are more important. For example, if it is a short project it may impact my decision on priorities. But even here it not the estimate that has value it is just information that helps me evaluate and prioritize. If I already know that this project has huge value and will be my top priority does forecasting aid with that decision.

When Does it Make Sense to Forecast?

Listing those questions above it seems like I am suggesting that a request for a forecast is always the wrong question and is never really needed. And it is true that I did struggle to come up with a good example of when it makes sense to do a detailed project level forecast that included dates or any type of scheduling expectations.

It may be necessary for a sales contract, to have a common set of expectations. Although I would very much hope that sales contract for agile projects are for time and materials and are flexible in scope and dates, if not cost too. So for the purposes of setting expectations and in negotiations I can see that there is value in an estimate, although I do still wonder if a detailed forecast adds anything here that a reasonable cost estimate doesn’t cover. If possible I would rather work to an agreed date or an agreed budget than suggest a forecast that may lead to false expectations.

But the reality is that sometimes your customer does want one and won’t tell you why, or doesn’t know why. Some customers (and managers) are willing to accept that a forecast will cost time and money and the more detailed it is the more it costs, and of course being more detailed may not make it any more accurate.

More detailed investigation for your forecasting is likely to build greater confidence but may not be more accurate and you should ask the question “will more detail have a material impact on your decisions?” and if the extra effort wouldn’t affect your decision then it is just waste.

I would caution that if there is no other alternative and a forecast is made, that it is revised regularly and transparently, the sooner the forecast is seen as variable the more useful it is. There is so much assumption tied to a forecast that it becomes a ball and chain if there is not an expectation of it changing and so it must be refreshed regularly to prevent the early assumptions being seen as certainty, or they will lead to disappointment later.

Short-term Forecasting

The real value in forecasts is when the forecast is for a short time frame. Over the short term, we can have much more confidence in our forecasts, especially if we have been working on this project for a while and have historic information we can base our forecast on. There are fewer assumptions and less variables.

• Can you forecast which features are likely to be included in the next release
• If I add a new feature now can you estimate a lead time for this
• Can you give me an estimate of how much this feature would cost

By limiting the scope of the forecast to an area where we have more confidence in our expectations, the forecasts become more meaningful; however, there is still a danger they are seen as commitments, although the risk is mitigated by the shorter time frame.

Alternatives to Forecasting

Many of the questions above could have been resolved with much less effort than a detailed project level forecast. In most cases we could achieve sufficient accuracy for decision making with a high level estimate. For example, “the product is similar to ‘X,’ therefore I estimate 4-8 months for a small team.” Given that, we could also deliver a rough estimate of 12-18 months for two teams and calculate costs accordingly.

These estimates are certainly broad, but if you have confidence in your teams, believe they will use Agile principles to get value early, and are able to communicate progress and transparency with issues, then I see no issue with broad estimates. It is sufficient to allocate staff and resources, prioritize, schedule and determine Return on investment decisions.

For the other questions, you may achieve far better answers through the use of product/feature burndown charts, user story mapping, or even simple high level road maps. These tools provide useful information which can be used for managing expectations, identifying dependencies and visualizing the progress of a product. And crucially – they can aid in setting priorities and keeping the progress transparent.

---

Related Project Estimating Articles

#NoEstimates - Alternative to Estimate-Driven Software Development

Estimating With Use Case Points

Fingers in the air: a Gentle Introduction to Software Estimation

---

Click here to view the complete list of archived articles

Article published in February 2017