Date post: | 20-Dec-2015 |
Category: |
Documents |
View: | 222 times |
Download: | 3 times |
MIT and James Orlin © 2003
1
Dynamic Programming 2
– Review–More examples
MIT and James Orlin © 2003
2
Match game example
Suppose that there are 20 matches on a table, and the person who picks up the last match wins. At each alternating turn, my opponent or I can pick up 1, 2 or 6 matches. Assuming that I go first, how can I be sure of winning the game?
MIT and James Orlin © 2003
3
Determining the strategy using DP n = number of matches left (n is the state/stage) g(n) = 1 if you can force a win at n matches.
g(n) = 0 otherwise g(n) = optimal value function.
At each state/stage you can make one of three decisions: take 1, 2 or 6 matches.
g(1) = g(2) = g(6) = 1 (boundary conditions) g(3) = 0; g(4) = g(5) = 1.
The recursion: g(n) = 1 if g(n-1) = 0 or g(n-2) = 0 or g(n-6) = 0;
g(n) = 0 otherwise. Equivalently, g(n) = 1 – min (g(n-1), g(n-2), g(n-6)).
MIT and James Orlin © 2003
4
1 2 3 4 5 6 7 8 9 10
11 12 13 14 15 16 17 18 19 20
1 2 3 4 5 6 7 8 9 10
11 12 13 14 15 16 17 18 19 20
Winning and Losing Positions
MIT and James Orlin © 2003
5
Principle of Optimality
Any optimal policy has the property that whatever the current state and decision, the remaining decisions must constitute an optimal policy with regard to the state resulting from the current decision.
Whatever node j is selected, the remaining path from j to the end is the shortest path starting at j.
MIT and James Orlin © 2003
6
Capital Budgeting, again
Investment 1 2 3 4 5 6
Cash Required (1000s)
$5
$7
$4
$3
$4
$6
NPV added (1000s)
$16
$22
$12
$8
$11
$19
Investment budget = $14,000
MIT and James Orlin © 2003
7
Stages, and the DP recursion
Solve the knapsack problem as a sequence of problems
At stage k we consider whether to add item k or not to the knapsack.
Let f(k,B) be the best NPV limited to stocks 1, 2, …, k only and using a budget of at most B.
Knapsack problem via DP
MIT and James Orlin © 2003
8
Solving the Stockco Problem
At stage 1 compute for each budget from 0 to $14,000 compute the best NPV while limited to stock 1.
At stage 2 compute for each budget from 0 to $14,000 compute the best NPV while limited to stocks 1 and 2.
At stage 3 compute for each budget from 0 to $14,000 compute the best NPV while limited to stocks 1, 2 and 3.
MIT and James Orlin © 2003
9
Capital budgeting in general
Max j=1..n cj xj
s.t j=1..n aj xj b
x binary
Let f(k, B) = Max j=1..k cj xj
s.t j=1..k aj xj B
x binary
MIT and James Orlin © 2003
10
The recursion
f(0,0) = 0; f(0,k) is undefined for k > 0
f(k, B) = min ( f(k-1, B), f(k-1, B-ak) + ck) either item k is included, or it is not
The optimum solution to the original problem is max { f(n, B) : 0 B b }.
Note: we solve the capital budgeting problem for all right hand sides less than b.
MIT and James Orlin © 2003
11
The Knapsack ProblemIs there a feasible solution to
31 x + 35 y + 37 z = 422
x, y, z are non-negative integers.
let f(1, j) = T if there is a solution to 31 x = j, f(1, j) = F if there is no solution to 31 x = j. where x is a non-negative integer
let f(2, j) = T if there is a solution to
31 x + 35 y = j, x and y are non-negative integer
MIT and James Orlin © 2003
12
The Knapsack ProblemIs there a feasible solution to
31 x + 35 y + 37 z = 422
x, y, z are non-negative integers.
let f(3 ,j) = T if there is a solution to
31 x + 35 y + 37 z = j, x, y, and z are non-negative integer
Note: there are three stages to this problem.
MIT and James Orlin © 2003
13
The recursion for the knapsack problem
Is there a feasible solution to
31 x = 422
x non-negative integer?
f(0,0) = T, f(0,k) = F for k > 0
f(1,k) = T if f(0,k) = T or f(1, k-31) = T for k = 31 to 422
f(1,k) = F otherwise.
Does this really work?
MIT and James Orlin © 2003
14
The recursion for the knapsack problem
Is there a feasible solution to
31 x + 35 y = 422
x, y are non-negative integers.
f(2,k) = T if f(1,k) = T or f(2, k-35) = T for k = 0 to 422
f(2,k) = F otherwise
MIT and James Orlin © 2003
15
The recursion for the knapsack problem
Is there a feasible solution to
31 x + 35 y + 37 z = 422
x, y, z are non-negative integers?
f(3,k) = T if f(2,k) = T or f(3, k-37) = T
f(3,k) = F otherwise
MIT and James Orlin © 2003
16
The DP again, without stagesIs there a feasible solution to
31 x + 35 y + 37 z = 422
x, y, z are non-negative integers.
let g(j) = T if there is a solution to
31 x + 35 y + 37 z = j, x, y, and z are non-negative integer
Then g(0) = T.
For j > 0, g(j) is true if i. g(j-31) is true orii. g(j-35) is true oriii. g(j-37) is true
MIT and James Orlin © 2003
17
Optimal Capacity Expansion: What is the least cost way of building plants?
Year Cum. Demand Cost per plant in $millions
2002 1 54
2003 2 56
2004 4 58
2005 6 57
2006 7 55
2007 8 52
Cost of $15 million in any year in which a plant is built. At most 3 plants a year can be built
MIT and James Orlin © 2003
18
Cost of Building plantscost of building in each year
3 177 183 189 186 180 1712 123 127 131 129 125 1191 69 71 73 72 70 670 0 0 0 0 0 0
years to go 6 5 4 3 2 1 0
Minimum number of plants needed
# of plants 1 2 4 6 7 8years to go 6 5 4 3 2 1 0
MIT and James Orlin © 2003
19
Step 1. For each year, identify those table entries that are feasible, taking into account the lower bounds and the fact that at most 3 plants can be built per year.
876543210
6 5 4 3 2 1 0Years to go
(Y, k) is a state denoting having k plants with Y years to go.
MIT and James Orlin © 2003
20
876543210
6 5 4 3 2 1 0Years to go
Step 2. f(Y, k) = remaining cost of capacity expansion starting with k plants with Y years to go.
Fill in the entries for 0 years to go. Then fill in the entries for 1 year to go. Then fill in entries for 2 years to go, and continue.
MIT and James Orlin © 2003
21
Determining state and stage
Stage: Y = years to go State: number of plants f(j, Y) = optimum cost of capacity
expansion starting with j plants with Y years to go
We want to compute f(0, 6).
What is f(j, 0) for different j?
MIT and James Orlin © 2003
22
The recursion Let g(Y, k) be the cost of building k plants with Y
years to go. Let l(Y) be the minimum number of plants needed
with Y years to go. Let M(Y) be the maximum number of plants that can
be built with Y years to go.
Let f(Y,k) be the remaining cost of capacity expansion starting with k plants with Y years to go.
What is f(0,k)? Can you compute f(Y,k) assuming that f(Y-1, j) is already computed for each j?
MIT and James Orlin © 2003
23
Summary for Dynamic Programming
Recursion Principle of optimality states and stages and decisions useful in a wide range of situations– games – shortest paths– capacity expansion–more will be presented in next lecture.
MIT and James Orlin © 2003
24
Dynamic Programming Review
Next lecture: DP under uncertainty.
Applications –meeting a financial goal
(a very simplified version)– getting a plane through enemy territory