A review of Itanium server sales forecasts made over seven years reveals the same bias year after year after year.
When the economist Kenneth Arrow was working as an air force weather forecaster during the Second World War, he and his colleagues found that their long-range weather predictions were no better than random. They informed the boss but were told, “The commanding general is well aware that the forecasts are no good. However, he needs them for planning purposes.”
David Orrell, The Science of Prediction, 2006
The first step in R&D portfolio management is to generate the list of projects that are under consideration and value them. Unfortunately, this is where many companies spend 99% of their effort, and too often this leads to countless wasted hours and benign neglect of the portfolio as a whole.
Let’s start out with a basic fact: Every project’s net present value (NPV) in every portfolio of every company in the world is wrong. Absolutely, dead wrong. The forecasts, be they cost or revenue, will not hit their projections. They may be higher or lower by one dollar or a billion dollars, but they will not match their projections. The forecasters be lucky if the sales forecasts used to generate the NPVs are within +/- 50% of what will actually happen, yet they’re spending an inordinate amount of time on non-value-adding activities such as:
- Assessing forecasts to the second decimal place
- Explaining why a project’s NPV changed by 2%
- Trying to approximate the uncertainty in a forecast by picking the “right” point estimate instead of simply incorporating uncertainty into the forecast