Ebook Using Dynamic Programming with Adaptive Grid Scheme for Optimal Control Problems in Economics
In recent times the lack of closed form solutions of dynamic models with optimizing agents has led to the use of computational methods to solve those models. Closed form solutions are available for special cases such as the linear quadratic control problem, see Ljungqvist and Sargent (2001, ch. 4) and growth models with log-utility. While those special models suffice for the analytical study of a variety of economic problems in more general cases they are not sufficient for a robust analysis of interesting economic problems. For the more general cases numerical methods have been proposed in the literature. A detailed discussion of a variety of numerical methods and a comparison with our proposed method is provided in Section 3 below.
Our paper is concerned with a family of continuous and discrete time dynamic models with optimizing agents whose solutions are most conveniently approached by the method of dynamic programming. Dynamic programming provides the value function and the control variable in feedback form. Yet, the application of numerical methods such as dynamic programming to find the global dynamics in interesting regions of the state space were restricted by the use of fixed grid size techniques. Following Grüne (1997) in this paper an adaptive grid scheme is used for finding global solutions of models with dynamic optimization. As Santos and Vigo-Aguiar (1998), we use numerical value function iteration but we employ local error estimates based on a flexible grid scheme. Since those numerical methods provide us with approximate solutions only, it is essential to have accuracy estimates for the numerical methods employed.
We consider discounted continuous time and discrete time optimal control problems and flexible grid scheme based on local error estimates. The advantage of the use of flexible grid scheme is demonstrated by computing the value function and the control variable in feedback form of one and two dimensional economic models. In order to study the accuracy of our numerical methods when applied to economic models we first want to test our algorithm by studying a basic growth model of Brock and Mirman (1972) type for which the exact solution is known. This allows us to judge the accuracy of our numerical method and to explore whether the flexible grid scheme is accurately capturing important solution properties, for example, steep curvature of the value function. Our algorithm will be applied to a deterministic as well as to a stochastic version of the Brock and Mirman (1972) growth model.
Moreover in the economic literature there exist more complicated dynamic models with optimizing agents which have been a challenge to commonly used numerical techniques. Those models exhibit more complicated dynamics due to the existence of multiple steady state equilibria, thresholds (Skiba-sets) separating domains of attraction and periodic solutions as attractors. Examples of such models can be found in the literature on economic growth and development. Multiple steady states and thresholds can also arise in the dynamic decision problem of the firm, for example due to relative adjustment costs of investment, in resource economics and in ecological management problems. Our paper studies a prototype model from each of those areas and applies the proposed dynamic programming algorithm with adaptive grid scheme to find the global dynamics.
The remainder of the paper is organized as follows. Section 2 describes the basic discretization procedure and its strategy of adaptive grid refinement, while Section 3 compares our approach to other advanced dynamic programming schemes found in the literature. Section 4 solves one dimensional control problems. Here we study the basic growth model for which the exact solution is known and compare the accuracy of our numerical methods with previously used methods. We then study one dimensional dynamic optimization models with multiple domains of attraction and thresholds. Section 5 then presents results for two dimensional examples which also can give rise to thresholds and periodic solutions as attractors. Section 6 concludes the paper.
Posted in :
