How agents form expectations and, in particular, whether they are rational and efficiently incorporate all available information into their forecasts, is a question of fundamental importance in macroeconomic analysis. Ultimately this question can best be resolved through empirical analysis of expectations data. It is therefore not surprising that a large literature has been devoted to empirically testing forecast rationality based on survey data such as the Livingston data or the Survey of Professional Forecasters (SPF). Unfortunately, the empirical evidence is inconclusive and seems to depend on the level of aggregation of data, sample period, forecast horizon and type of variable under consideration.
The vast majority of studies have tested forecast rationality in conjunction with an assumption of quadratic loss. This loss function has largely been maintained out of convenience: under quadratic loss and rationality, the observed forecast errors should have zero mean and be uncorrelated with all variables in the current information set.
A reading of the literature reveals little discussion of why forecast errors of different signs should lead to the same loss. On economic grounds one would, if anything, typically expect asymmetric losses. For example, over predictions of sales lead to inventory holding costs while under predictions lead to stock out costs, loss of reputation and revenues when demand cannot be met. Most often there is no reason why these costs should be identical. Concerns such as these led Granger and New bold (1986, page 125) to write: “An assumption of symmetry for the cost function is much less acceptable [than an assumption of a symmetric forecast error density]”. Relaxing the assumption of symmetric loss has important consequences. If the loss function under which predictions were derived is not restricted to be symmetric, rationality no longer requires that the forecast errors are unbiased, as demonstrated by Zellner (1986) and Christoffersen and Diebold (1997).
While many studies on forecast rationality testing are aware of the limitations of symmetric loss and indicate that rejections of rationality may be driven by asymmetries, little is known about the magnitude of the problem i.e. how much this really matters in practice. In this paper we therefore examine the theoretical and practical importance of the joint nature of tests for forecast rationality. We show that the coefficients in standard forecast efficiency tests are biased if the loss function is not symmetric and characterize this bias. Under asymmetric loss, standard rationality tests thus do not control size and may lead to false rejections of rationality. Conversely, we also find that even very large inefficiencies in forecasters’ use of information may not be detectable by standard tests when the true loss is asymmetric.
Download
Biases in Macroeconomic Forecasts: Irrationality or Asymmetric Loss?
