Forecasting
A junior high school student can project the exact temperature at which water will freeze given the variables of atmospheric pressure and salinity. Conversely, a municipal budget department run by an MBA with a B.S. in mathematics was unable to project the cities tax revenue within 10% in ten of the last twelve quarters ‐‐ often even missing the direction of the trend. Yet he has not been terminated!
The errors made by the budget director were numerous, and we will attempt to summarize all that we saw here. By the way ‐‐ this is not an isolated incident ‐‐ it isa common failure to understanding the anatomy of a forecast.
1. The same model and many of the same assumptions were used in every forecast. There was obviously no penalty for failure so there was no incentives to even try to get the model right.
2. The assumptions used for the forecast were wrong. Seasonal adjustments for the revenue stream were ignored. For example, property taxes are paid twice annually, in January and July ‐‐ while sales taxes increase substantially during the holiday season in November and December, and slow down during January and August. These mandated collection dates and historical trends were simply ignored.
3. They failed to address known events that affect sales, such as rain. The more it rains, the less people shop. The higher the price of gas, the less people shop. Construction will impact the number of shoppers going into a mall. Major stimulus programs (such as cash for clunkers) have dates for initiation and termination. When the dwelling occupancy rate rises, the rental tax will go down. While this is a demonstrably crude model, what was so stunning to this observer is that no adjustment was made when a major sports team moved its home stadium to a new town!
4. Revenue was projected in straight lines, trending either up or down.
5. Applying Bayesian reasoning and formulas to socio‐economic data. Human behavior generates different types of data sets than human attributes.
Admittedly, forecasting the future can be a bit like pondering the imponderable or attempting to create that which cannot be created; but that’s not to say that the effort isn’t worthwhile. On the contrary, authentic forecasting that can identify significant trends and provide answers within a couple percentage points is invaluable. Forecasts are an important tool in every manager’s arsenal. Following are some common traits we have observed in our clients who regularly produce accurate forecasts.
A. If you get it wrong, it’s OK. Business conditions can be volatile, with changes lasting from days to forever. Think of the changes wrought by an Icelandic volcano or a technology change that rendering a product or technology you sell obsolete. If you are wrong, admit it ‐‐ own it ‐‐ and eat it.
B. Test the assumptions on historical data and see how well the forecast correlates to historical trends that are known. For example use data from 2008 to predict what will happen in 2009. Have reasonable explanations for any forecast that deviates from historical behavior or transactions. Do both short and long‐term forecasts, continuing to identify emerging trends that can be used to narrow the error rate.
C. Many variables cannot be fixed as a constant, and the models should project values over a range. As an example, you can’t use a fixed price or an average for gasoline, like $3.10 per gallon – you need to choose a reasonable range, such as $2.80 to $3.40 for the 3rd quarter. If you model both the upper and lower range, you’ll identify whether or not this is an expense that you need to hedge. Also you can get an idea of the sensitivity of your budget if, for example all of the actual data all fall toward one or the others limits of the proposed ranges.
D. Automate the process as much as you can. The more automated the process the more you can work on identifying and weighting your inputs to improve accuracy.
E. Do not trust information past a range where historically you have had error go beyond one standard deviation. At this point there are dynamic inputs that are not allowing the model to be even remotely accurate ‐‐ acknowledge that. Don’t limit your thinking to calendar days. Consider N iterations of the company’s sales cycle, X iteration of inventory turnover, Y iterations of _______, etc… you get the idea.
F. Purge single line projections from your practice. A swimmer would avoid a pool filled with crocodiles, not build a model to determine if they are hungry. At a minimum use a forecasting cone of probability showing ranges of outcomes over time. If you use different models (and we highly recommend this) show each of the straight line projections overlaid on the cone of probability built on all of the models. The best examples I’ve seen are the presentations used to forecast the path of a hurricane with what is referred to as spaghetti model with a cone of probability. http://www.snowbirdnationals.com/weather.htm
G. Look at commercial software to help you. Determine which variable are dependent and which are independent, and learn to use Monte Carlo simulations.
On the other hand, you can use a rainbow of colors, circumnavigating arrows, implied correlations, and issue congruency married to polysylabical descriptors (preferably in Latin) so you will always appear to be a rocket scientist in charge of designing a wind up toy.
