# The Economics of Inaction: Stochastic Control Models with Fixed Costs

Nancy L. Stokey
Pages: 320
https://www.jstor.org/stable/j.ctt7sfgq

1. Front Matter
(pp. i-iv)
(pp. v-viii)
3. Preface
(pp. ix-xii)
4. 1 Introduction
(pp. 1-14)

In situations where action entails a fixed cost, optimal policies involve doing nothing most of the time and exercising control only occasionally. Interest in economic models that exhibit this type of behavior has exploded in recent years, spurred by growing evidence that “lumpy” adjustment is typical in a number of important economic settings.

For example, the short-run effects of monetary policy are connected with the degree of price stickiness. Data from the U.S. Bureau of Labor Statistics on price changes at retail establishments for the period 1988–2003 suggest that price adjustment is sluggish, at least for some products. The...

5. I Mathematical Preliminaries
• 2 Stochastic Processes, Brownian Motions, and Diffusions
(pp. 17-29)

This chapter contains background material on stochastic processes in general, and on Brownian motions and other diffusions in particular. Appendix A provides more detail on some of the topics treated here.

To define a random variable, one starts with aprobability space$(\Omega ,\;\mathfrak{F},\;P)$, where$\Omega$is a set,$\mathfrak{F}$is a$\sigma {\rm{ - algebra}}$of its subsets, andPis a probability measure on$\mathfrak{F}$. Each$\omega \; \in \;\Omega$is anoutcome, and each set$E\; \in \;\mathfrak{F}$is anevent. Given a probability space$(\Omega ,\;\mathfrak{F},\;P)$, arandom variableis a measurable function$x:\Omega \; \to \;R$. For each$\omega \; \in \;\Omega$, the real number$x(\omega )$is therealizationof the...

• 3 Stochastic Integrals and Ito’s Lemma
(pp. 30-52)

Consider the present discounted value of a stream of returns over an infinite horizon,

$\begin{array}{c} v({x_0}) \equiv \;\int_0^\infty {{e^{ - \rho t}}\pi \,(x(t))dt} \\ {\rm{with}}\;\dot x(t)\; = \;g(x(t)),\quad t \ge \;0, \\x(0)\; = \;{x_0}, \\\end{array}$(3.1)

where$\rho \; > \;0$is a discount rate,$\pi (x)$is a return function,$x(t)$is a state variable that evolves according to the law of motion${\rm{g(x)}}$, and${x_0}$is the initial state. For example,$v({x_0})$might be the value of a firm, where$\rho$is the interest rate,${\rm{x(t)}}$describes the size of the market for the firm’s product,π(x)is the profit flow as a function of market size, andg(x)describes the evolution of market size. Since the horizon is infinite,ρis constant,...

• 4 Martingales
(pp. 53-74)

Martingales are an example of mathematics at its best, sublimely elegant and at the same time enormously useful. Initially developed to study questions that arise in gambling, the theory of martingales has subsequently been used to study a wide array of questions.

The treatment here is only an introduction, covering the key concepts and major results. Section 4.1 provides a formal definition and illustrates the idea with some examples. Section 4.2 shows how martingales can be constructed from the (stationary) transition function for a discrete-time Markov process, using an eigenvector of the transition matrix if the state space is discrete...

• 5 Useful Formulas for Brownian Motions
(pp. 75-106)

In situations where action involves a fixed cost, optimal policies have the property that control is exercised only occasionally. Specifically, optimal policies involve taking action when a state variable reaches an appropriately chosen threshold value. In this chapter methods are developed for analyzing models of this type.

To fix ideas, consider the following example. Suppose the profit flow${\rm{g(X)}}$of a firm depends on its relative price$X\; = \;p\; - \;\bar p$, wherepis the firm’s own price and$\bar p$is an aggregate price index, both in log form. Assume that$\bar p$evolves as a Brownian motion. Then in the absence of action...

6. II Impulse Control Models
• 6 Exercising an Option
(pp. 109-128)

In the presence of a fixed cost of adjustment, optimally exercising control has two aspects: deciding at what point(s) action should be taken and choosing what the action(s) should be. In this chapter the simplest example of this sort is studied, the problem of exercising a one-time option of infinite duration. The option problem is simple because action is taken only once and the action itself is fixed, so the only issue is timing—deciding when, if ever, to exercise the option. Its simplicity makes this problem a useful introduction to methods that are applicable more broadly.

Two approaches are...

• 7 Models with Fixed Costs
(pp. 129-152)

In the option problem of Chapter 6 the fact that the option can be exercised only once acts like a fixed cost. In other settings action can be taken many times, but there is an explicit fixed cost of adjustment. The methods used to analyze the option problem can readily be extended to this class. The problem of optimally exercising control is then more complicated in two respects, however. First, the size of the adjustment must be chosen. In addition, because action is taken repeatedly, decisions must be forward looking in a more complex sense, anticipating future actions by the...

• 8 Models with Fixed and Variable Costs
(pp. 153-175)

In many contexts exercising control involves variable costs of adjustment as well as fixed costs. The presence of a variable cost means that the decision about how much control to exercise is slightly more complicated than in the menu cost model, but the overall character of the solution is similar nevertheless: an optimal policy still involves doing nothing most of the time and exercising control only occasionally.

A standard inventory model of the type studied by Scarf (1960), Harrison, Selke, and Taylor (1983), and others provides an example. Suppose there is a plant that produces output, and there are customers...

• 9 Models with Continuous Control Variables
(pp. 176-196)

In the menu cost and inventory models of Chapters 7 and 8 the only actions taken by the decision maker are discrete adjustments of the state variable. But the methods used in those problems can be extended to allow an additional feature as well, continuously chosen control variables that affect the evolution of the state between adjustments and also enter the return function directly. Because the state variable no longer evolves as a Brownian motion between adjustments, the functions$\hat L,\;\psi$, and$\Psi$are not useful in these settings. The optimal policy and value function can still be characterized with the...

7. III Instantaneous Control Models
• 10 Regulated Brownian Motion
(pp. 199-224)

Consider the following purely mechanical description of a stock. In the absence of any regulation, the flows into and out of the stock are exogenous and stochastic, and the cumulative difference between the inflows and outflows is described by a Brownian motionX. There are fixed valuesb,B, with$b < B$. The stock is not allowed to rise aboveBor to fall belowb. An automatic regulator exercises control when the stock reaches either threshold, increasing or decreasing the stock by just enough to keep it inside the interval [b, B].

For example, the inflow can be interpreted as...

• 11 Investment: Linear and Convex Adjustment Costs
(pp. 225-250)

A variety of investment problems can be analyzed using models similar to the inventory model in Section 10.4. Several examples are studied below to illustrate some of the features that can be incorporated and the types of solutions that result. All of the examples have a similar structure for the revenue function, but they make different assumptions about investment costs.

Consider a firm whose revenue flow at any date, net of wages, materials, and other variable costs, depends on its capital stock and a random variable describing demand. Demand is a geometric Brownian motion or a more general diffusion. The...

8. IV Aggregation
• 12 An Aggregate Model with Fixed Costs
(pp. 253-282)

Aggregate models in which individual agents face fixed adjustment costs fall into two broad categories. In the first agents are subject to idiosyncratic shocks, and the shocks can be modeled as i.i.d. across agents. The law of large numbers then implies that once a stationary cross-sectional distribution of shocks and endogenous states has been reached, economic aggregates are constant over time. If the initial distribution is the stationary one aggregates are constant from the outset. Thus in settings with a large number of agents and idiosyncratic shocks, constructing tractable aggregate models is relatively straightforward.

In the second type of model...

• A Continuous Stochastic Processes
(pp. 283-289)

This appendix contains background material on continuous stochastic processes in general and Wiener processes in particular.

Let$(\Omega ,\mathfrak{F},P)$be a probability space and$\left\{ {{X_n}} \right\}_{n = 1}^\infty$a sequence of random variables on that space. There are several distinct notions of convergence for such a sequence. In addition there are several distinct names for each notion.

(a) Convergence with probability one

The sequence$\left\{ {{X_n}} \right\}$converges to the random variableX with probability oneif

${\rm{Pr}}\left\{ {\mathop {{\rm{lim}}}\limits_{n \to \infty } {X_n} = X} \right\} = 1.$

This type of convergence is also calledconvergence almost surely (a.s.)orconvergence almost everywhere (a.e.).

(b) Convergence in probability

The sequence$\left\{ {{X_n}} \right\}$converges to the randam variable...

• B Optional Stopping Theorem
(pp. 290-294)

This appendix contains proofs of the optional stopping results, Theorems 4.3 and 4.4. The proofs are for discrete time only, although the results hold for continuous-time processes as well. HereZdenotes a stochastic process that may have either a continuous or discrete time index$(t \in {R_ + }or t = 0,1,2,...)$and${Z_k}$denotes a process for whichk= 0, 1, 2, . . . , must be discrete.

Continuous-time processes must have the following two properties, however: every sample path, at every datet, must be continuous from the right and must have a limit from the left. Since Brownian motions have sample paths...

9. References
(pp. 295-302)
10. Index
(pp. 303-308)