As we have said before, every forecast you’ll ever build is wrong. The truth may be out there, but due to a lack of perfect information about the future, you won’t be able to reveal it before your project review meeting.
There is a paradox here: The more you insist on the truth, the more likely you are to deny the uncertainties obscuring it. Alas, denying the existence of those uncertainties distances you still further from the truth. So, take a deep breath, and let go of your desire to know the truth. Now you are ready to manage (dare I say embrace?) the uncertainty standing between you and the truth. Managing this uncertainty is the key to good forecasting. By effectively managing uncertainty, you’ll be able to honestly assess the value of your forecast and, in turn, make an honest statement about the value of your project.
A tornado diagram can help you find your project’s pot of gold.
As a forecaster, the key question you need to answer is: “Is my forecast precise enough to make a confident decision?” If it is, hooray! You are ready to make an investment decision. If the answer is no, then you need to gather more information. One of the most useful and easiest-to-understand methods we have to assess our confidence in a forecast is the tornado diagram. Using a tornado diagram, we can assess how much our forecast might change if things go better or worse than anticipated. We can also assess which uncertainties have the greatest impact on our forecast—those are the very inputs we should research further if we want to tighten up forecast precision.
Step 1. List the Inputs to Your Quantitative Model
Building a tornado diagram begins with a listing of all the inputs that inform the forecast. Your forecast model may have five inputs, or it may have 50. Either way, you’ll need to gather them all and make sure you have the current values of each input as well.
Step 2. Start Your (Scenario) Engines
Now comes the trickiest part: Enlist your project team in a scenario building exercise. Your mission is to consider what might cause each input to be much lower, or much higher, than your base case estimate. In some cases, it will be easier to think about a group of inputs, such as those that build up revenue or manufacturing costs, rather than thinking about each input individually. From these scenarios, you’ll estimate a reasonable range of values for each input such that there is only a 1-in-10 chance the true value is higher, and a 1-in-10 chance that the true value is lower. Stated differently, you’ll be estimating the 10% fractile and the 90% fractile for each input.
Don’t make the mistake of simply reducing/increasing each input’s value by a set percentage (such as 10%) in this exercise. It is really important that your team goes through the exercise of building plausible scenarios. Humans are optimistic by nature, and we also tend to be overconfident in our own assessments. The scenarios will help stretch the boundaries of what we realize is possible and help us derive more realistic estimates for each inputs. Some things to consider here:
- Your high and low estimates for an input may not be equidistant from the base case. Sometimes the 1-in-10 downside is much worse than the 1-in-10 upside!
- Estimating +/- 10% from the base case is not only overconfident, but also avoids any meaningful discussion of the uncertainty around each input, and doesn’t allow useful comparison across the bars of the tornado. Don’t fall into the false economy of using this kind of knee-jerk assessment.
- Every input will have a different range, because your team will have varying amounts of knowledge about each input, which will result in different high and low values. This will be really important when we interpret the tornado.
- Make sure the entire project team is available for the scenario building exercises. This isn’t just about estimating numbers; as the scenarios are crafted, the team will realize there are mitigation steps that can be taken, and opportunities that should be seized, in the base case. Every scenario building activity I have ever attended has resulted in a stronger project plan and a more valuable product.
- Write up your scenarios and accompanying rationale for each input’s range that you estimate. You won’t remember either in a month, and anyone reviewing your tornado will want this information as they work with you to determine next steps for the project.
If the team is having trouble thinking about a plausible lower bound for each input, consider Clayton Christensen’s question meant to spur active thinking: “What has to prove true for this to work?” Other times, you might employ a series of questions used to discover latent biases in the base case, thereby stretching the conversation about high and low estimates. When we have used these methods with project teams, we have sometimes discovered that what was thought to be the base case is actually the optimistic case—an important detail to reveal before spending begins in earnest.
Step 3. Calculate!
With your high and low estimates in hand, you can now compute the coordinates of the tornado. Select the output metric you want to examine: Net present value (NPV) is the old standby, but you might be more interested in breakeven month or year, or five year revenue, or return on investment (ROI). Within your forecast, set the first input to the low estimate and compute your output metric. Now set that same input to the high estimate and compute the output metric again. Set that input back to its base case value, and repeat the procedure for every other input.
Step 4. Sort and Graph
They’re called tornado diagrams because they look like tornadoes; you’ll need to sort your inputs in descending order of the difference between high and low values of the output metric. This will put the inputs whose uncertainty impacts the output metric the most at the top of the tornado, and those that have the least impact at the bottom. An example is shown below.
If you google ‘tornado diagram’ and look at the image results you’ll see a never-ending variety of styles. We prefer an understated esthetic that focuses the eye on the largest bars without clutter. It is useful to note the base case value for the metric of merit with a vertical line.
For clarity, we have only included the top five inputs on the tornado above. Culling like this isn’t always a good idea: If you are trying to make a point about a particular input having no impact on project value, despite being the constant focus of the project team’s attention, then by all means keep all inputs on the tornado so that they can see where that input falls in relation to all others.
Step 5. Draw Actionable Conclusions
Here is where the rubber meets the road. We often say that tornado diagrams help us separate what is countable from what counts. The items at the top of the tornado ‘count’: Across a credible range of values they have the greatest effect on the metric of merit. The items at the bottom are ‘countable’: They are understood to such a degree that if the plausible low or high value of this input comes to pass, its impact on the metric of merit is negligible. Items like costs of goods sold, packaging costs, and manufacturing yield often end up at the bottom. These are quantities that are easy (or easier) to define, and have less of an impact on most measures of value than the inputs that drive revenue directly.
The relative importance of input uncertainty to your project’s value is given by the rank of each input in the tornado. All other things being equal, your project team should spend its valuable time refining the inputs at the top of the tornado, because those inputs contribute the lion’s share of uncertainty to your project. If you can interview customers or build a prototype and learn something important about the path forward, you can reduce the plausible range for one of the highest ranked inputs to the tornado, and reduce that input’s ranking in the tornado.
The absolute importance of each input’s uncertainty will help you decide if the uncertainties related to that input must be resolved before a decision can be made. For example, if your policy is only to fund projects with an NPV of $30M, and the tornado bar for market share extends from $5M to $45M, then you would be hard pressed to defend the project until the market share estimate is refined or until other business case changes shift the range of NPVs based on market share.
Step 6. Act!
Based on the results of the tornado, teams pursue one of two paths:
- If the value of the project is unambiguous, decide to fund or abandon the project.
- If the value of the project is not yet clear, use the tornado diagram to decide which inputs must be resolved with a narrower range, and perform the research necessary to narrow those inputs’ estimates. In some cases, this means decomposing an input into two or more constituent inputs and explicitly including these inputs in the tornado.
Several of our life sciences clients regularly use a tornado diagram across the epidemiological inputs of a sales forecast model to assess which precursors of revenue require additional primary or secondary research in order to make a confident funding decision. Another client uses a tornado analysis at the portfolio level to identify the biggest drivers of uncertainty for an entire product line, across all products. Yet another client uses a tornado diagram in every finance presentation to remind stakeholders that additional detail on tax schemes, asset capitalization, and discount rates will not improve confidence in a project’s value. These tornadoes are generated automatically using the Enrich Analytics Platform, but there are many templates available online for creating tornadoes using Excel, or you can make your own.
What Comes Next?
Mathematically speaking, it is more accurate to perform a Monte Carlo simulation that considers all uncertainties simultaneously, since some uncertainties cancel each other out, while other uncertainties amplify each other. We see the tornado diagram as a stepping stone to Monte Carlo methods because it is much more transparent than a full-blown simulation, and is therefore easier to explain to all stakeholders, while still spurring conversation about the underpinnings of the forecast, input by input. When you do begin using simulation methods, the input ranges generated in the process of creating a tornado diagram can be used in Monte Carlo simulations. If a client only has appetite for one method of assessing uncertainty, it should be the tornado. With a well-oiled tornado methodology under your belt, you can then augment (but not replace) it with Monte Carlo simulation.
See Enrich Analytics in action here.
