Optimal Control in Agent-based Economics: A Survey

“The laws of history are as absolute at the laws of physics, and if the probabilities of error are greater, it is only because history does not deal with as many humans as physics does atoms, so that individual variations count for more.”
Isaac Asimov, Foundation and Empire (1952)

Optimal Control in Agent-based Economic Models: A Survey
James Matthew B. Miraflor | CS 296 - Seminar

“This is what I mean by the ‘mechanics’ of economic development - the construction of a mechanical, artificial world, populated by the interacting robots that economics typically studies, that is capable of exhibiting behavior the gross features of which resemble those of the actual world.”
Robert Lucas, Jr., “On the Mechanics of Economic Development”. Journal of Monetary Economics. 1988.

The “Robots” = “Representative Agents”
•Firms. Production function
•Consumers. Utility function
•Government
–Usually is a constraint rather than an objective function
–Behavior – Efficient or Inefficient? Benevolent or Corrupt?
•Optimization
•Equilibrium
–When does an economy stop growing? At what point do firms stop producing, consumers stop consuming, government stop spending?

PRODUCTION
Firms, Technology, Capital, and Labor

The Neoclassical Production Function
•Production function: 푌=퐹(퐾,퐿,퐴)
–where 푌 is the output, 퐴 is the total factor productivity/technology, 퐿 is the labor input, 퐾 is the capital input
–satisfies conditions below:
1.Constant returns to scale:
퐹휆퐾,휆퐿,휆퐴=휆퐹(퐾,퐿,퐴),∀휆>0
–Replication argument
2.Positive and diminishing returns
o 휕퐹 휕퐾 >0, 휕2퐹 휕퐾2<0, 휕퐹 휕퐿 >0, 휕2퐹 휕퐿2<0

The Neoclassical Production Function
3.Inada (1963) Conditions:
olim 퐾→0 휕퐹 휕퐾 =lim 퐿→0 휕퐹 휕퐿 =∞,lim 퐾→∞ 휕퐹 휕퐾 =lim 퐿→∞ 휕퐹 휕퐿 =0
4.Essentiality:
o퐹0,퐿=퐹퐾,0=0
•Per Capita values
푌=퐹퐾,퐿,퐴→ 푌 퐿 =퐹 퐾 퐿 ,1, 퐴 퐿 =퐿푓푘,
푦=푓(푘) 휕퐹 휕퐾 =푓′푘, 휕퐹 휕퐿 =푓푘−푘푓′(푘)

Per Capita values
•퐾(푡) =퐼푡−훿퐾푡=푠퐹(퐾,퐿,퐴)−훿퐾푡
• 퐾푡 L= 푠퐹퐾,퐿,푇 퐿 − 훿퐾푡 퐿 =푠푓푘−훿푘
•Take the time derivative of 푘 to get:
•푘 = 푑 퐾 퐿 푑푡 = 퐿퐾 −퐾퐿 퐿2= 퐾 퐿 − 퐿 퐿 퐾 퐿 = 퐾 퐿 −푛푘
o 퐿 퐿 =푛 (population growth)
o푘 = 퐾 퐿 −푛푘→ 퐾 퐿 =푘 +푛푘
•푘 +푛푘=푠푓푘−훿푘
•푘 =푠푓푘−훿푘−푛푘

Capital Dynamics
•Change in capital is produced good not consumed.
•푘 =푓푘−푐−(푛+훿)푘, where:
–푘 is the per capita capital (퐾/퐿), 푘 = 푑푘 푑푡 is the change in capital, or the investment, due to savings.
–훿 is the depreciation rate of capital
–푛 is the population rate
•Notice that 푛 behaves like a depreciation rate since it represents the fraction of resources to be given to the next generation.

Savings/Investment and Consumption
Capital (k)
Output (y)
Gross
Product
f(k)
Some level of Capital (k)
Actual GDP (y)
Actual
Savings
(s*y)
Gross Savings s*f(k)
consumption per worker
investment per worker
Borrowed/derived from DE201 lecture slides of Prof. Emmanuel de Dios

Gross
Product
f(k)
k*
Population Growth
k1
k2
Investment is greater than population growth; capital per person increases
Population growth is greater than investment; capital per person decreases.
Savings/Investment and Consumption
Capital
(k)
Output (y)

Savings and Economic Growth
Gross Product f(k)
High economic growth
Zero growth
Negative economic growth
Population Growth
Capital
(k)
Output (y)

Cobb-Douglas Production Function
푌=퐹퐾,퐿=퐴퐿훼퐾훽
•훼 and 훽 are the output elasticities of capital & labor
–measures the responsiveness of output to a change in either labor or capital, ceteris paribus.
•If we want production per capita, we divide the function buy 퐿 to get 푌 퐿 =퐴퐿훼−1퐾훽=퐴퐿훼−훽−1퐾 퐿 훽 →푦=퐴퐿훼−훽−1푘훽
•where 푦 and 푘 are per capita production and per capita capital respectively.
•Conventionally, we set 훼+훽=1 so that 훽=1−훼

CONSUMPTION
The Utility Function and Discounting of the Future

The Utility Function
•푐푡=푐푡 represents the consumption at time 푡
•푢푡= 푢푐푡=푢(푐푡) represents the utility of consumers from consuming 푐푡.
•푈0 is the total, accumulated utility over infinite time of the consumer, i.e. 푈0= 푒−휌푡 푢푐푡 푑푡 ∞ 0
oIf 퐿푡 is the population level at time 푡, we have: 푈0= 푒−휌푡 푢푐푡퐿푡 푑푡 ∞ 0

Constant Elasticity of Substitution
•If production is Cobb-Douglas, then the necessary and sufficient conditions for optimal savings (Kurz, 1968) :
1.A 푢(푐푡) must be Constant Elasticity of Intertemporal Substitution (CEIS)
푢푐푡= 푐푡 1−휎−11−휎
–where 휎= 1 푠 and 푠 is the savings rate
–Constant aversion to fluctuations in consumption
–One doesn’t get more or less risk averse as one gets richer (or poorer).
2.Discount rate 휌 must be related to the parameters of 휌=훽−푠, where 훽 is share of capital to production.

Optimal Cumulative Consumption
•A consumer agent will want to maximize consumption over time, i.e. max푈0= 푒−(휌−푛)푡 푐푡 1−휎−11−휎 푑푡 ∞ 0
푠.푡. 푘 =푓푘−푐−푛푘−훿푘, 푘0>0 푔푖푣푒푛
•The solution to this optimal control problem will then govern the dynamics of savings across time.
•How do we solve?

BASICS OF OPTIMAL CONTROL
The Hamiltonian and the Maximum Principle

The Lagrangean (Static Optimization)
m푎푥 {푓(푥)|푔푖 =0,푖퐾} •Given the function 푦=푓(푥1,…,푥푛) subject to a constraint 푔푖(푥1,…,푥푛)=0, 푖=1,…,푘 the Lagrangean is
퐿푥;휆=푓푥+ 휆푖푔푖(푥) 푘 푖=1
•where the  is the vector of Lagrangean multipliers.
•Let 푦=푓푥1,푥2, 푔푥1,푥2=0
퐿푥1,푥2,휆=푓푥1,푥2+휆푔(푥1,푥2)
•To determine 푥1∗,푥2∗,휆∗: 휕퐿 휕푥1= 휕푓푥1,푥2 휕푥1+ 휕푔푥1,푥2 휕푥1=0 휕퐿 휕푥2= 휕푓푥1,푥2 휕푥2+ 휕푔푥1,푥2 휕푥2=0

The Hamiltonian (Dynamic Optimization)
•Definition. Given the problem
푀푎푥 퐽= 푓푥,푢,푡푑푡 푇 0
푆.푡. 푥′푡=푔(푥,푢,푡)
푥0=푥0,푥푇 free,푇 fixed
푢푡휖 ℝ
•The Hamiltonian of the problem is the function
퐻푥,푢,휆,푡=푓푥,푢,푡+휆푔(푥,푢,푡)
•휆(푡) is called the costate variable.□
Borrowed/derived from ECON207 lecture slides of Prof. Rolando Danao

Problem of Dynamic Optimization
•From among functions x 휖C1[0,T] starting at (0,x0) and ending at (T,xT), choose an x* such that J(x*) ≥ J(x).
•Note that in Optimal Control terms, we are actually selecting a u (governing x) that optimizes x
0 T
x
x0
xT
x*(t)
Figure 1
□

The Pontryagin Maximum Principle in Terms of the Hamiltonian
•Theorem. Given the problem
•푀푎푥 퐽= 푓푥,푢,푡푑푡 푇 0
•푆.푡. 푥′푡=푔(푥,푢,푡)
• 푥0=푥0,푥푇 free,푇 fixed
• u(t) 휖 ℝ
•퐻푥,푢,휆,푡=푓푥,푢,푡+휆푔(푥,푢,푡)
•If the pair (u*(t), x*(t)) is optimal, then there is a continuously differentiable function 휆(푡) such that:
a. 휕퐻 휕휆 = x*′ (Hλ = x*′)
b. 휕퐻 휕푥 =−휆′ (Hx = −λ′)
c. 휕퐻 휕푢 =0 (Hu = 0)
d.x*(0) = x0
e.λ(T) = 0

Infinite Horizon Problems
•The Infinite Horizon Optimal Control Problem
•푀푎푥 푓푥,푢,푡푒−푟푡푑푡 ∞ 0
•푆.푡. 푥′푡=푔(푥,푢,푡)
• 푥0=푥0,푢(푡)∈푈
퐻푥,푢,휆,푡=푓푥,푢,푡푒−푟푡+휆푔(푥,푢,푡)
•The objective function is sensible only if, for all admissible pairs (푥,푢,푡), the integral converges.

Infinite Horizon Problems
Transversality Conditions. 푎 푥∞ 푓푟푒푒
lim 푡→∞ 휆푡=0 푏 푥∞≥푥푚푖푛
lim 푡→∞ 휆(푡)≥0 lim 푡→∞ 휆푡=0 푖푓 푥∞>푥푚푖푛 푐 푥∞ 푓푖푥푒푑
lim 푡→∞ 퐻=0
TVC is a description of how the optimal path crosses a terminal line in variable endpoint problems.

DYNAMIC OPTIMIZATION
The Ramsey-Cass-Koopmans Model

The Ramsey-Cass-Koopmans Model
•Question: How much should a nation save?
•max푈0= 푒−(휌−푛)푡 푐푡 1−휎−11−휎 푑푡 ∞ 0
•푠.푡. 푘 =푓푘−푐−푛푘−훿푘, 푘0>0 푔푖푣푒푛
•The solution to this optimal control problem will then govern the dynamics of savings across time.
•Note that lim 푡→∞ 푒−(휌−푛)푡 푐푡 1−휎−11−휎 =0
oOne can verify that for this to happen, 휌>푛

Solving the Model
•Setting up the Hamiltonian
퐻=푒−(휌−푛)푡 푐푡 1−휎−11−휎 +휐(푓푘−푐−푛푘−훿푘)
•Where 휐 is the dynamic Lagrange multiplier
o휐 can also be interpreted as the shadow price of investment
•The First Order Conditions (FOCs) are:
o퐻푐=0
o퐻푘=−휐
oTransversality condition (TVC): lim 푡→∞ 푘푡푣푡=0

퐻=푒−(휌−푛)푡 푐푡 1−휎−11−휎 +휐(푓푘−푐−푛푘−훿푘)
•퐻푐=0→푒−(휌−푛)푡푐푡 −휎−휐=0 (1)
•퐻푘=−휐 →휐푓′푘−푛−훿=−휐
o− 휐 휐 =푓′푘−푛−훿 (2)
•Take logs and time derivative of (1):
o−휌−푛푡−휎ln푐푡=ln휐
o−휌−푛−휎 푐 푐 = 휐 휐 → 푐 푐 =휎−1(−휌+푛− 휐 휐 ) (3)
•Plug (3) to (2) to get:
o훾푐= 푐 푐 =휎−1푓′(푘)−휌−훿
oIn the Cobb-Douglas case: 훾푐=휎−1훽푘−(1−훽)−휌−훿

Equilibrium in Consumption
훾푐= 푐 푐 =휎−1푓′(푘)−휌−훿 =휎−1훽푘−(1−훽)−휌−훿
•In equilibrium, i.e. 푐 푐 =0, 푓′푘=휌+훿
oIf consumption is to remain at its current level, marginal return to capital must at least reach the level of the combined future discounting and capital depreciation
oAt this level, an individual is indifferent between consuming and spending.

Transitional Dynamics
From our constraint: 푘 =푓푘−푐−푛푘−훿푘
• 푘 푘 = 푓푘−푐 푘 −푛−훿
• 푐 푐 =휎−1푓′(푘)−휌−훿
•In equilibrium
o 푘 푘 =0→푐=푓푘−푛+훿푘
o 푐 푐 =0→푓′(푘)=휌+훿

푘 =0 curve: 푐=푓(푘)−(푛+훿)푘 푐 =0 curve: 푓′(푘)=휌+훿
(푛+훿)푘
푓(푘)
푘
푘
푐
푘∗
푐 =0
푘 =0
퐸
퐸 = steady state
푘0

푘∗
푐∗
퐸
푐 =0
푘 =0
푐
푘
+
−
+
−
The Ramsey Model
Stable branch
Unstable branch

The Ramsey Model
•The steady state E = (k*,c*) is a saddle-point equilibrium. It is unique.
•At equilibrium, k* is a constant; hence, y* = f(k*) is a constant. Since k ≡ K/L and y ≡ Y/L, then at E, the variables Y, K, and L all grow at the same rate.
•The only way for the economy to move toward the steady state is to hitch onto a stable branch.
•Given an initial capital-labor ratio k0, it must choose an initial per capita consumption level c0 such that the pair (k0,c0) lies on the stable branch.

푘∗
푐∗
퐸
푐 =0
푘 =0
푐
푘
+
−
+
−
The Ramsey Model
Stable branch
Unstable branch
If the economy is not on a stable branch, ever-increasing k accompanied by ever-decreasing c leads per capita consumption towards starvation.

푘∗
푐∗
퐸
푐 =0
푘 =0
푐
푘
+
−
+
−
The Ramsey Model
Stable branch
Unstable branch
If economy is not on a stable branch: Ever-increasing c accompanied by ever-decreasing k leads to capital exhaustion.

The Ramsey Model
•If the economy is not on a stable branch, the dynamic forces lead to:
–Ever-increasing 푘 accompanied by ever- decreasing 푐 (streamlines pointing to the southeast) leading per capita consumption towards starvation.
–Ever-increasing c accompanied by ever- decreasing 푘 (streamlines pointing to the northwest) implying overindulgence leading to capital exhaustion.
•The only viable long-run alternative for the economy is the steady state at 퐸.

The Ramsey Model
•At steady state, per-capita consumption becomes constant and its level cannot be further raised over time.
•That’s because the production function does not include technological progress.
•To make a possible rising per-capita consumption, technological progress must be introduced.

ROLE OF THE GOVERNMENT
The Barro Model of Public Spending
Leyte Vice Gov. Carlo Loreto
Tacloban City Vice Mayor Jerry Yaokasin
Gov. Dom Petilla
Coca-Cola FEMSA CEO Juan Ramon Felix
Coca-Cola Corporate Affairs director Jose Dominguez
Reopening of Plant at Tacloban City

Barro Model of Public Spending
•Economist Robert J. Barro (1990) builds on the Ramsey-Cass-Koopmans model to propose a “public spending” model:
푦=푓푘,푔=퐴푘(1−훼)푔훼
o푔 is the government’s input to production (infrastructures, highways, public works, etc.)
o훼 is the elasticity of government’s share to production
o푔 is financed entirely from tax revenues.
oportion of the economy is taxed so the government can spend.

Effect of Taxes
oDefining 휏 as the constant average and marginal income tax rate, we then have
푔= 휏푦=휏퐴푘(1−훼)푔훼.
•Note also, that disposable savings is less than that earlier, since aside from consumption, a portion (휏) of production is taxed. In this case, our constraint is transformed into: 푘 =1−휏퐴푘1−훼푔훼−푐, 푘0>0 푔푖푣푒푛

Barro’s Optimal Control Problem
•The optimal control problem in Barro (1990) then becomes:
•max푈0= 푒−휌푡 푐푡 1−휎−11−휎 푑푡 ∞ 0
•푠.푡. 푘 =1−휏퐴푘1−훼푔훼−푐, 푘0> 0 푔푖푣푒푛
•where 푔= 휏푦=휏퐴푘(1−훼)푔훼
•This is the “competitive case”, wherein consumer agents take government spending as a given and then optimize.

Competitive Case
•Setup the Hamiltonian
퐻=푒−휌푡 푐푡 1−휎−11−휎 +휐1−휏퐴푘1−훼푔훼−푐
•The First Order Conditions (FOCs) are:
o(1) 퐻푐=0→푒−휌푡푐−휎=휐
o(2) 퐻휐=−휐 →휐 =−휐1−휏퐴푘−훼푔훼= −푣1−휏퐴1−훼 푔 푘 훼
o(3) TVC

Competitive Case
•For (1), taking log of both sides and differentiating, we get:
o 푒−휌푡푐−휎=휐→−휌푡−휎ln푐=ln휐
o 푑 푑푡 −휌푡−휎ln푐= 푑 푑푡 ln휐→−휌−휎 푐 푐 = 휐 휐 (3)
oSubstitute (3) into (2) to get:
o 휐 휐 =−1−휏퐴1−훼 푔 푘 훼
o−휌−휎 푐 푐 =−1−휏퐴1−훼 푔 푘 훼
o 푐 푐 =휎−11−휏퐴1−훼 푔 푘 훼 −휌 (4)

Size of the Government
푔= 휏푦=휏퐴푘(1−훼)푔훼
•We can get the size of the government 휏 by
o휏= 푔 푦 = 푔 퐴푘1−훼푔훼= 푔 푘 1−훼 퐴−1→
o 푔 푘 =(휏퐴)1/(1−훼)
•Plug it into (4) to get:
o 푐 푐 =휎−11−휏퐴1−훼 푔 푘 훼 −휌
•훾1= 푐 푐 =휎−1퐴1∗−휌
owhere 퐴1∗=1−훼퐴 11−훼1−휏휏 훼 1−훼

Command Economy
•In a command economy, the government will take into account that private output affects public income and (through the production function) other people’s marginal product of capital.
•To solve, subsitute 퐴1∗=1−훼퐴 11−훼1−휏휏 훼 1−훼 in the A part of the Hamiltonian
•퐻= 푒−휌푡 푐푡 1−휎−11−휎 +휐1−휏퐴푘1−훼푔훼−푐

Command Economy
•The FOCs are:
o푒−휌푡푐−휎=휐
o 휐 휐 =−1−휏퐴 11−훼휏 훼 1−훼−휌
•Substituting the usual way we obtain:
•훾2= 푐 푐 =휎−1퐴2∗−휌
•Where 퐴2∗=1−휏퐴 11−훼휏 훼 1−훼

Efficiency of the Command Economy
•훾1= 푐 푐 =휎−1퐴1∗−휌
owhere 퐴1∗=1−훼퐴 11−훼1−휏휏 훼 1−훼
•훾2= 푐 푐 =휎−1퐴2∗−휌
–Where 퐴2∗=1−휏퐴 11−훼휏 훼 1−훼
•Note that since for all values of 휌, 퐴1∗<퐴2∗→훾1<훾2
•Government is forced to provide one more unit of public input for every unit of savings by individuals.
•The assumption is that the government is a “benevolent dictator”.
•What if it is not?

CORRUPTION IN GOVERNMENT
The Ellis-Fender Model

Ellis & Fender (2006) Model
•Takes off from a Ramsey type model of economic growth in which the “engine of growth” is public capital accumulation.
•Public capital financed by taxes on private output.
•Government can either use taxes to fund public capital accumulation or engage in corruption.
•Ellis & Fender defines output as:
푦푡=푓푙푡,푝푡=푙푡훼푝(푡)훽,0<훼,훽<1
•where 푙(푡) is the effective labor and 푝(푡) is the public capital.

Ellis & Fender (2006) Model
•Public capital then moves according to: 푝 =휏푡−휔−푏푡−휔−휎푝(푡)
•where:
o휏푡−휔 represents the taxes paid at interval of length 휔 in the past,
o푏푡−휔 is portion of past tax payments that were corrupted by the government
o휎 is capital depreciation.
•Interval 휔 is production lag
–But their subsequent results demonstrate that it can also be seen as “transparency”

Optimal Consumption
•Goal of consumer-citizens - maximize accumulated consumption
•Let 푐푡 be the consumption and −푙푡 be the decision to pursue leisure.
푢푡=푢푐푡,푙푡=[푐푡+푙푡]휃
•where 0<휃<1 is the intertemporal substitution parameter (similar to 1−휎 earlier), and r is the discount rate (similar to 휌 earlier).
–instantaneous budget constraint 푐푡=푦푡−휏푡 must be satisfied.
•Optimization problem max [푐푡+푙푡]휃푒−푟푡 푑푡 ∞ 0

Optimal Corruption
•Goal of the government - maximize its accumulated corruption.
•Having defined 푏 earlier: max 푏푡푒−푟푡 푑푡 ∞ 0
•We set up the second goal as a constraint together with the equation of motion 푝
•Reduces to an isoperimetric (due to the integral term in the constraint) Ramsey-type optimization problem

The “Robots” = Economic Agents
•Firms. 푌=퐴퐿훼퐾훽→푓(푘)
•Consumers. 푢푐푡= 푐푡 1−휎−11−휎
•Government. 푔= 휏푦=휏퐴푘(1−훼)푔훼
•Optimization
omax푈0= 푒−(휌−푛)푡푢푐푡푑푡 ∞ 0
o푠.푡. 푘 =1−휏퐴푘1−훼푔훼−푐, 푘0>0
•Equilibrium. 푘 푘 = 푐 푐 =훾=0

“The relevant question to ask about the ‘assumptions’ of a theory is not whether they are descriptively “realistic,” for they never are, but whether they are sufficiently good approximations for the purpose in hand. And this question can be answered only by seeing whether the theory works, which means whether it yields sufficiently accurate predictions.”
Milton Friedman, “The Methodology of Positive Economics” (1966)
Thank you for listening!

Sources
•Ellis, Christopher James & John Fender (2006, May). “Corruption and Transparency in a Growth Model”. International Tax and Public Finance. Volume 13, Issue 2-3: 115-149.
•Sala-i-Martin, Xavier (1990a, December). “Lecture Notes on Economic Growth(I): Introduction to the Literature and Neoclassical Models”. NBER Working Paper No. 3563.
•Sala-i-Martin, Xavier (1990b, December). “Lecture Notes on Economic Growth(II): Five Prototype Models of Endogenous Growth”. NBER Working Paper No. 3564.

Optimal Control in Agent-based Economics: A Survey

More Related Content

What's hot (6)

Viewers also liked (20)

Similar to Optimal Control in Agent-based Economics: A Survey (20)

More from James Matthew Miraflor (13)

Recently uploaded (20)

Optimal Control in Agent-based Economics: A Survey