Post on 02-Jun-2018
transcript
8/10/2019 Rsh Qam11 Module02
1/24
M2
1. Understand the overall approach of dynamic
programming.
2. Use dynamic programming to solve the shortest-
route problem.
3. Develop dynamic programming stages.
After completing this module, students will be able to:
2
LEARNING OBJECTIVES
Dynamic Programming
MODULE
Summary Glossary Key Equations Solved Problem Self-Test Discussion Questions and Problems Case
Study: United Trucking Internet Case Study Bibliography
4. Describe important dynamic programming
terminology.
5. Describe the use of dynamic programming in solvin
knapsack problems.
M2.1 Introduction
M2.2 Shortest-Route Problem Solved Using Dynamic
Programming
M2.3 Dynamic Programming Terminology
M2.4 Dynamic Programming Notation
M2.5 Knapsack Problem
MODULE OUTLINE
8/10/2019 Rsh Qam11 Module02
2/24
M2-2 MODULE 2 DYNAMIC PROGRAMMING
M2.1 Introduction
Dynamic programming is a quantitative analysis technique that has been applied to large, com-
plex problems that have sequences of decisions to be made. Dynamic programming divides
problems into a number of decision stages; the outcome of a decision at one stage affects the de-
cision at each of the next stages. The technique is useful in a large number of multiperiod busi-
ness problems, such as smoothing production employment, allocating capital funds, allocating
salespeople to marketing areas, and evaluating investment opportunities.
Dynamic programming differs from linear programming in two ways. First, there is no al-gorithm (as in the simplex method) that can be programmed to solve all problems. Dynamic pro-
gramming is, instead, a technique that allows us to break up difficult problems into a sequence
of easier subproblems, which are then evaluated by stages. Second, linear programming is a
method that gives single-stage (one time period) solutions. Dynamic programming has the
power to determine the optimal solution over a one-year time horizon by breaking the problem
into 12 smaller one-month horizon problems and to solve each of these optimally. Hence, it uses
a multistage approach.
Solving problems with dynamic programming involves four steps:
Four Steps of Dynamic Programming
1. Divide the original problem into subproblems called stages.2. Solve the last stage of the problem for all possible conditions or states.
3. Working backward from the last stage, solve each intermediate stage. This is done by
determining optimal policies from that stage to the end of the problem (last stage).
4. Obtain the optimal solution for the original problem by solving all stages sequentially.
In this module we show how to solve two types of dynamic programming problems: net-
work and nonnetwork. The shortest-route problem is a network problem that can be solved by
dynamic programming. The knapsack problem is an example of a nonnetwork problem that can
be solved using dynamic programming.
The second step is to solve the
last stagestage 1.
M2.2 Shortest-Route Problem Solved Using Dynamic Programming
George Yates is about to make a trip from Rice, Georgia (1), to Dixieville, Georgia (7). George
would like to find the shortest route. Unfortunately, there are a number of small towns between
Rice and Dixieville. His road map is shown in Figure M2.1.
The circles on the map, called nodes, represent cities such as Rice, Dixieville, Brown, and
so on. The arrows, called arcs, represent highways between the cities. The distance in miles is
indicated along each arc. This problem can, of course, be solved by inspection. But seeing how
dynamic programming can be used on this simple problem will teach you how to solve larger
and more complex problems.
Step 1: The first step is to divide the problem into subproblems or stages. Figure M2.2 reveals
the stages of this problem. In dynamic programming, we usually start with the last part of the
problem, stage 1, and work backward to the beginning of the problem or network, which is stage
3 in this problem. Table M2.1 summarizes the arcs and arc distances for each stage.
Step 2: We next solve stage 1, the last part of the network. Usually, this is trivial. We find the
shortest path to the end of the network, node 7 in this problem. At stage 1, the shortest paths,
The first step is to divide the
problem into subproblems or
stages.
Dynamic programming
breaks a difficult problem
into subproblems.
8/10/2019 Rsh Qam11 Module02
3/24
M2.2 SHORTEST-ROUTE PROBLEM SOLVED USING DYNAMIC PROGRAMMING M2
1 3
2
4
6
5
7
Stage 3
A Branch
Stage 2 Stage 1
A Node
45
2
10
12
6
4 10
2
14
FIGURE M2.2
Three Stages for theGeorge Yates Problem
10 Miles
10 Miles
12Mile
s
4Mile
sBrown
2Mile
s
14Miles
5 Miles
4Mile
s
2Miles
Lake City
Hope
Athens
Georgetown
Dixieville
4
1 3
2
5
6
7
6Miles
Rice
FIGURE M2.1
Highway Map betweenRice and Dixieville
STAGE ARC ARC DISTANCE
1 57 14
67 2
2 45 10
35 12
36 6
25 4
26 10
3 14 4
13 5
12 2
TABLE M2.1
Distance Along Each Arc
8/10/2019 Rsh Qam11 Module02
4/24
M2-4 MODULE 2 DYNAMIC PROGRAMMING
45
2
10
12
6
410
2
14
Minimum Distanceto Node 7 fromNode 6
Minimum Distanceto Node 7 fromNode 5
1 3
2
4
6
5
7
2
14
FIGURE M2.3
Solution for the One-Stage Problem
45
2
10Minimum Distanceto Node 7 fromNode 3
12
6
4 10
2
14
Minimum Distanceto Node 7 fromNode 4
Minimum Distanceto Node 7 fromNode 2
1 3
2
4
6
5
7
8
12
14
2
24
FIGURE M2.4Solution for the Two-Stage Problem
Step 3: Moving backward, we now solve for stages 2 and 3. At stage 2 we will use Figure M2.4.Step 3 involves moving backwardsolving intermediate stages.
from node 5 to node 6, to node 7 are the only paths. Also note in Figure M2.3 that the minimum
distances are enclosed in boxes by the entering nodes to stage 1, node 5 and node 6. The objec-
tive is to find the shortest distance to node 7. The following table summarizes this procedure for
stage 1. As mentioned previously, the shortest distance is the only distance at stage 1.
STAGE 1
BEGINNING SHORTEST DISTANCE ARCS ALONGNODE TO NODE 7 THIS PATH
5 14 57
6 2 67
8/10/2019 Rsh Qam11 Module02
5/24
M2.2 SHORTEST-ROUTE PROBLEM SOLVED USING DYNAMIC PROGRAMMING M2
1 3
2
4
6
5
7
45
2
10
8
12
6
4 10
2
14
24
Minimum Distanceto Node 7 fromNode 1
12
14
2
13
FIGURE M2.5
Solution for the Three-Stage Problem
If we are at node 4, the shortest and only route to node 7 is arcs 45 and 57. At node 3, t
shortest route is arcs 36 and 67 with a total minimum distance of 8 miles. If we are at node
the shortest route is arcs 26 and 67 with a minimum total distance of 12 miles. This inform
tion is summarized in the stage 2 table:
STAGE 2
BEGINNING SHORTEST DISTANCE ARCS ALONG
NODE TO NODE 7 THIS PATH
4 24 45
57
3 8 36
67
2 12 26
67
The solution to stage 3 can be completed using the following table and the network
Figure M2.5:
STAGE 3
BEGINNING SHORTEST DISTANCE ARCS ALONGNODE TO NODE 7 THIS PATH
1 13 13
36
67
8/10/2019 Rsh Qam11 Module02
6/24
M2-6 MODULE 2 DYNAMIC PROGRAMMING
M2.3 Dynamic Programming Terminology
Regardless of the type or size of a dynamic programming problem, there are some important
terms and concepts that are inherent in every problem. Some of the more important of them
follow:
1. Stage: a period or a logical subproblem.
2. State variables: possible beginning situations or conditions of a stage. These have also
been called the input variables.
3. Decision variables: alternatives or possible decisions that exist at each stage.
4. Decision criterion: a statement concerning the objective of the problem.
5. Optimal policy: a set of decision rules, developed as a result of the decision criteria, that
gives optimal decisions for any entering condition at any stage.
6. Transformation: normally, an algebraic statement that reveals the relationship betweenstages.
In the shortest-route problem, the following transformation can be given:
This relationship shows how we are able to go from one stage to the next in solving for the
optimal solution to the shortest-route problem. In more complex problems, we can use symbols
to show the relationship between stages.
State variables, decision variables, the decision criterion, and the optimal policy can be de-
termined for any stage of a dynamic programming problem. This is done here for stage 2 of the
George Yates shortest-route problem.
Distancefromthe
beginningofagiven
stagetothelastnode
=
Distancefromthebeginning
ofthepreviousstage
tothelastnode
+
Distancefromthe
givenstagetothe
previousstage
The fourth and final step is to
find the optimal solution after all
stages have been solved.
Step 4: To obtain the optimal solution at any stage, all we consider are the arcs to the next stage
and the optimal solution at the next stage. For stage 3, we only have to consider the three arcs to
stage 2 (12, 13, and 14) and the optimal policies at stage 2, given in a previous table. This is
how we arrived at the preceding solution. When the procedure is understood, we can perform all
the calculations on one network. You may want to study the relationship between the networks
and the tables because more complex problems are usually solved by using tables only.
Dynamic Programming in Nursery
Production Decisions
Managing a nursery that produces ornamental plants is diffi-cult. In most cases, ornamental plants increase in value with in-
creased growth. This value-added growth makes it difficult to
determine when to harvest the plants and place them on the
market. When plants are marketed earlier, revenues are gener-
ated earlier and the costs associated with plant growth are mini-
mized. On the other hand, delaying the harvesting of the
ornamental plants usually results in higher prices. But are the ad-
ditional months of growth and costs worth the delay?
In this case, dynamic programming was used to determine
the optimal growth stages for ornamental plants. Each stage was
associated with a possible growth level. The state variables
included acres of production of ornamental plants and carryover
plants from previous growing seasons. The objective of the
dynamic programming problem was to maximize the after-tax
cash flow. The taxes included self-employment, federal income,
earned income credit, and state income taxes.
The solution was to produce one- and three-gallon containers
of ornamental plants. The one-gallon containers are sold in the fall
and carried over for spring sales. Any one-gallon containers not
sold in the spring are combined into three-gallon container prod-
ucts for sale during the next season. Using dynamic programming
helps to determine when to harvest to increase after-tax cash flow.
Source: Jeffrey Stokes, et al. Optimal Marketing of Nursery Crops from
Container-Based Production Systems, American Journal of Agricultural
Economics (February 1997): 235246.
IN ACTION
8/10/2019 Rsh Qam11 Module02
7/24
M2.3 DYNAMIC PROGRAMMING TERMINOLOGY M2
1. State variables for stage 2 are the entering nodes, which are
(a) node 2
(b) node 3
(c) node 4
2. Decision variables for stage 2 are the following arcs or routes:
(a) 45
(b) 35
(c) 36
(d) 25(e) 26
3. The decision criterion is the minimization of the total distances traveled.
4. The optimal policy for any beginning condition is shown in Figure M2.6 and the followin
table:
1 3
2
4
6
5
7
10
8
12
6
4 10
24
12
14
2
Statevariablesare theenteringnodes.
The optimal policy is the arc,for any entering node, thatwill minimize total distanceto the destination at thisstage.
Decisionvariablesare allthe arcs.
Stage 2
FIGURE M2.6
Stage 2 from theShortest-Route Problem
GIVEN THIS ENTERING THIS ARC WILL MINIMIZECONDITION TOTAL DISTANCE TO NODE 7
2 26
3 36
4 45
8/10/2019 Rsh Qam11 Module02
8/24
M2-8 MODULE 2 DYNAMIC PROGRAMMING
An input, decision, output, and
return are specified for each
stage.
The input to one stage is the
output from another stage.
M2.4 Dynamic Programming Notation
In addition to dynamic programming terminology, we can also use mathematical notation to de-
scribe any dynamic programming problem. This helps us to set up and solve the problem. Con-
sider stage 2 in the George Yates dynamic programming problem first discussed in Section
M2.2. This stage can be represented by the diagram shown in Figure M2.7 (as could any given
stage of a given dynamic programming problem).
As you can see, for every stage, we have an input, decision, output, and return. Look again
at stage 2 for the George Yates problem in Figure M2.6. The input to this stage is which con-
sists of nodes 2, 3, and 4. The decision at stage 2, or choosing which arc will lead to stage 1, is
represented by The possible arcs or decisions are 45, 35, 36, 25, and 26. The output to
stage 2 becomes the input to stage 1. The output from stage 2 is The possible outputs from
stage 2 are the exiting nodes, nodes 5 and 6. Finally, each stage has a return. For stage 2, the re-
turn is represented by In our shortest-route problem, the return is the distance along the arcs
in stage 2. These distances are 10 miles for arc 45, 12 miles for arc 35, 6 miles for arc 36, 4
miles for arc 25, and 10 miles for arc 26. The same notation applies for the other stages and
can be used at any stage. In general, we use the following notation for these important concepts:
(M2-1)
(M2-2)
(M2-3)
You should also note that the input to one stage is also the output from another stage. For
example, the input to stage 2, is also the output from stage 3 (see Figure M2.7). This leads us
to the following equation:
(M2-4)sn-1 = Outputfromstagen
s2,
rn = Returnatstagen
dn = Decisionatstagen
sn = Inputtostagen
r2.
s1.
d2.
s2,
Figure M2.6 may also be helpful in understanding some of the terminology used in the dis-
cussion of dynamic programming.
Stage 2
Decisiond2
Output
s1
Return
r2
Input
s2
FIGURE M2.7
Input, Decision, Output,and Return for Stage 2 inGeorge Yatess Problem
8/10/2019 Rsh Qam11 Module02
9/24
M2.5 KNAPSACK PROBLEM M2
A transformation function
allows us to go from one stage
to another.
The total return function
allows us to keep track of
profits and costs.
The objective of this problem is
to maximize profits.
M2.5 Knapsack Problem
The knapsack problem involves the maximization or the minimization of a value, such as pro
its or costs. As in a linear programming problem, there are restrictions in a knapsack proble
Imagine a knapsack or pouch that can hold only a certain weight or volume. We can place d
ferent types of items in the knapsack. Our objective is to place items in the knapsack to ma
mize total value without breaking the knapsack because of too much weight or a simil
restriction.
Types of Knapsack ProblemsMany kinds of problems can be classified as knapsack problems. Choosing items to place in t
cargo compartment of an airplane and selecting which payloads to put on the next NASA Spa
Shuttle are examples. The restriction can be volume, weight, or both. Some scheduling pro
lems are also knapsack problems. For example, we may want to determine which jobs to com
plete in the next two weeks. The two-week period is the knapsack, and we want to load it wi
jobs in such a way as to maximize profits or minimize costs. The restriction is the number
days or hours during the two-week period.
Rollers Air Transport Service Problem
Rob Roller owns and operates Rollers Air Transport Service, which ships cargo by plane
most large cities in the United States and Canada. The remaining capacity for one of t
flights from Seattle to Vancouver is 10 tons. There are four different items that Rob can shbetween Seattle and Vancouver. Each item has a weight in tons, a net profit in thousands
dollars, and a total number of that item that is available for shipping. This information is pr
sented in Table M2.2.
PROBLEM SETUP Rollers Air Transport problem is an ideal problem to solve using dynam
programming. Stage 4 will be item 1, stage 3 will be item 2, stage 2 will be item 3, and sta
The final concept is transformation. The transformation function allows us to go from o
stage to another. The transformation function for stage 2, converts the input to stage 2, a
the decision made at stage 2, to the output from stage 2, Because the transformation fun
tion depends on the input and decision at any stage, it can be represented as In ge
eral, the transformation function can be represented as follows:
(M2
The following general formula allows us to go from one stage to another using the transform
tion function:
(M2
Although this equation may seem complex, it is really just a mathematical statement of the f
that the output from a stage is a function of the input to the stage and any decisions made at th
stage. In the George Yates shortest-route problem, the transformation function consists o
number of tables. These tables show how we could progress from one stage to another in ord
to solve the problem. For more complex problems, we need to use dynamic programming no
tion instead of tables.
Another useful quantity is the total return at any stage. The total return allows us to ke
track of the total profit or costs at each stage as we solve the dynamic programming problem
can be given as follows:
(M2fn = Totalreturnatstagen
sn-1 = tn1sn, dn2
tn = Transformationfunctionatstagen
t2(s2,d2).
s1.d2,
s2,t2,
8/10/2019 Rsh Qam11 Module02
10/24
M2-10 MODULE 2 DYNAMIC PROGRAMMING
TABLE M2.2
Items to Be Shipped
Defining the ProblemThe Southern Company, with service areas in Georgia, Alabama, Mississippi, and Florida, is a major
provider of electric service, with about 240 generating units. In recent years, fuel costs have increased
faster than other costs. Annual fuel costs are about $2.5 billion, representing about one-third of total ex-
penses for the Southern Company. The problem for the Southern Company is to reduce total fuel costs.
Developing a Model
To deal with this fuel cost problem, the company developed a state-of-the-art dynamic programming
model. The dynamic programming model is embedded in the Wescouger optimization program, which is
a computer program used to control electric generating units and reduce fuel costs through better utiliza-
tion of existing equipment.
Acquiring Input DataData were collected on past and projected electric usage. In addition, daily load/generation data were an-
alyzed. Load/generation charts were used to investigate the fuel requirements for coal, nuclear, hydroelec-
tric, and gas/oil.
Developing a SolutionThe solution of the dynamic programming model provides both short-term planning guidelines and long-
term fuel usage for the various generating units. Optimal maintenance schedules for generating units are
obtained using Wescouger.
Testing the Solution
To test the accuracy of the Wescouger optimization program, Southern used a real-time economic dis-
patch program. The results were a very close match. In addition, the company put the solution through an
acid test, in which seasoned operators compared the results against their intuitive judgment. Again, the
results were consistent.
Analyzing the Results
The results were analyzed in terms of their impact on the use of various fuels, the usage of various gener-
ating units, and maintenance schedules for generating units. Analyzing the results also revealed other
needs. This resulted in a full-color screen editing routine, auxiliary programs to automate data input, and
software to generate postanalysis reports.
Implementing the ResultsThe Southern Company implemented the dynamic programming solution. Over a seven-year period, the
results saved the company over $140 million.
Source: S. Erwin, et al. Using an Optimization Software to Lower Overall Electric Production Costs for Southern Company,Interfaces 21, 1 (JanuaryFebruary 1991): 2741.
MODELING IN THE REAL WORLDReducing Electric Production Costs
Using Dynamic Programming
Defining
the Problem
Developing
a Model
Acquiring
Input Data
Developing
a Solution
Testing the
Solution
Analyzing
the Results
Implementing
the Results
ITEM WEIGHT PROFIT/UNIT NUMBER AVAILABLE
1 1 $3 6
2 4 9 1
3 3 8 2
4 2 5 2
8/10/2019 Rsh Qam11 Module02
11/24
M2.5 KNAPSACK PROBLEM M2-
s1Stage 2(Item 3)
Stage 3(Item 2)
d4
s3
r4
s4 Stage 4(Item 1)
s0Stage 1(Item 4)
d3
r3
d2
r2
d1
r1
Decisions
Returns
s2
FIGURE M2.8
Rollers Air TransportService Problem
1 will be item 4. This is shown in Table M2.3. During the solution, we will be using sta
numbers.
Rollers Air Transport problem can be represented graphically (see Figure M2.8). As y
can see, each item is represented by a stage. Look at stage 4, which is item 1. The total weig
that can be used is represented by This amount is 10 because we havent assigned any un
to be shipped at this time. The decision at this stage is (the number of units of item 1 we w
ship). If is 1, for example, we will be shipping 1 unit of item 1. Also note that is the retu
or profit at stage 4 (item 1). If we ship 1 unit of item 1, the profit will be $3 (see Table M2.2).
As mentioned previously, the decision variable represents the number of units of ea
item (stage) that can be shipped. Looking back at the original problem, we see that the proble
is constrained by the number of units. This is summarized in the following table:
dn,
r4d4
d4
s4.
TABLE M2.3
Relationship BetweenItems and Stages
STAGE MAXIMUM VALUE OF DECISION
4 6
3 1
2 2
1 2
THE TRANSFORMATION FUNCTIONS Next, we need to look at the transformation function. T
general transformation function for knapsack problems follows:
Note that and are coefficients for the problem and that represents the decision
stage n. This is the number of units to ship at stage sn.
dncnan,bn,
sn-1 = (an * sn) + (bn * dn) + cn
ITEM STAGE
1 4
2 3
3 2
4 1
8/10/2019 Rsh Qam11 Module02
12/24
M2-12 MODULE 2 DYNAMIC PROGRAMMING
The following chart shows the transformation coefficients for Rob Rollers transport problem:
COEFFICIENTS OFTRANSITION FUNCTION
STAGE an
bn
cn
4 1 1 0
3 1 4 02 1 3 0
1 1 2 0-
-
-
-
First, note that is 10, the total weight that can be shipped. Because represents the first item,
all 10 tons can be utilized. The transformation equations for the four stages are as follows:
(a)
(b)
(c)
(d)
Consider stage 3. Equation b reveals that the number of tons still available after stage 3, is
equal to the number of tons available before stage 3, minus the number of tons shipped at
stage In this equation, means that each item at stage 3 weighs 4 tons.
THE RETURN FUNCTIONS Next, we will look at the return function for each stage. This is the
general form for the return function:
Note that and are the coefficients for the return function. Using this general form of
the return function, we can put the return function values in the following table:
cnan,bn,
rn = 1an * sn2 + 1bn * dn2 + cn
4d33,4d3.
s3,
s2,
s0 = s1 - 2d1 stage1
s1 = s2 - 3d2 stage2
s2 = s3 - 4d3 stage3
s3 = s4 - 1d4 stage4
s4s4
COEFFICIENTS OF
DECISIONS RETURN FUNCTION
STAGE LOWER UPPER an
bn
cn
4 0 dn
6 0 3 0
3 0 dn
1 0 9 0
2 0 dn
2 0 8 0
1 0 dn
2 0 5 0
8/10/2019 Rsh Qam11 Module02
13/24
M2.5 KNAPSACK PROBLEM M2-
The lower value for each decision is zero, and the upper value is the total number availab
The coefficient is the profit per item shipped. The actual return functions are:
STAGE-BY-STAGE SOLUTION As you would expect, the return at any stage is equal to t
profit per unit at that stage multiplied by the number of units shipped at that stage, With t
information, we can solve Rollers Air Transport problem, starting with stage 1 (item 4). T
following table shows the solution for the first stage. You may wish to refer to Figure M2.8
this discussion.
dn.
rn,
r1 = 5d1
r2 = 8d2
r3 = 9d3
r4 = 3d4
bn
Because we dont know how many tons will be available at stage 1, we must consider
possibilities. Thus, the number of tons available at stage 1, can vary from 1 to 10. This is se
in the first column of numbers for stage 1. The number of units that we ship at stage 1, c
vary from 0 to 2. We cant go over 2 because the number available is only 2. For any decisi
we compute the return at stage 1, by multiplying the number of items shipped by 5, the pro
per item. The profit at this stage will be 0, 5, or 10, depending on whether 0, 1, or 2 items a
shipped. Note that the total return at this stage, is the same as because this is the only sta
we are considering so far. Also note that the total return before stage 1, is 0 because thisthe beginning of the solution and we are shipping nothing at this point.
f0,
r1f1,
r1,
d1,
s1,
We consider all possibilities.
is the total return at the first
stage. The total return before the
first stage is f0.
f1
STAGE 1
S1
d1
r1
s0
f0
f1
0 0 0 0 0 0
1 0 0 0 0 0
2 0 0 0 0 01 5 0 0 5
3 0 0 0 0 0
1 5 0 0 5
4 0 0 0 0 0
1 5 0 0 5
2 10 0 0 10
5 0 0 0 0 0
1 5 0 0 5
2 10 0 0 10
6 0 0 0 0 01 5 0 0 5
2 10 0 0 10
STAGE 1
S1
d1
r1
s0
f0
f
7 0 0 0 0
1 5 0 0
2 10 0 0 18 0 0 0 0
1 5 0 0
2 10 0 0 1
9 0 0 0 0
1 5 0 0
2 10 0 0 1
10 0 0 0 0
1 5 0 0
2 10 0 0 1
8/10/2019 Rsh Qam11 Module02
14/24
M2-14 MODULE 2 DYNAMIC PROGRAMMING
The solution for stage 1 shows the best decision, the return for this stage, and the total re-
turn, given all possible number of tons available (0 to 10 tons). Using the results of stage 1, we
can now proceed to stage 2. The solution for stage 2 is as follows:
STAGE 2
S2
d2
r2
s1
f1
f2
0 0 0 0 0 0
1 0 0 1 0 0
2 0 0 2 2 5
3 0 0 3 5 5
1 8 0 0 8
4 0 0 4 10 10
1 8 1 0 8
5 0 0 5 10 10
1 8 2 5 13
6 0 0 6 10 10
1 8 3 5 13
2 16 0 0 16
STAGE 2
S2
d2
r2
s1
f1
f2
7 0 0 7 10 10
1 8 4 10 18
2 16 1 0 16
8 0 0 8 10 10
1 8 5 10 18
2 16 2 5 21
9 0 0 9 10 10
1 8 6 10 18
2 16 3 5 21
10 0 0 10 10 10
1 8 7 10 18
2 16 4 10 26
The solution for stage 2 is found in exactly the same way as for stage 1. At stage 2 we still
have to consider all possible number of tons available (from 0 to 10). See the column (the first
column). At stage 2 (item 3) we still only have 2 units that can be shipped. Thus, (second col-
umn) can range from 0 to a maximum of 2. The return for each and combination at stage 2,
is shown in the third column. These numbers are the profit per item at this stage, 8, times the
number of items shipped. Because items shipped at stage 2 can be 0, 1, or 2, the profit at this
stage can be 0, 8, or 16. The return for stage 2 can also be computed from the return function:
Now look at the fourth column, which lists the number of items available after stage 2.This is also the number of items available for stage 1. To get this number, we have to subtract
the number of tons we are shipping at stage 2 (which is the tonnage per unit times the number of
units) from the number of tons available before stage 2 Look at the row in which is 6 and
is 1. We have 6 tons available before stage 2, and we are shipping 1 item, which weighs 3
tons. Thus we will have 3 tons still available after this stage. The values can also be deter-
mined using the transformation function, which is
The last two columns of stage 2 contain the total return. The return afterstage 1 and before
stage 2 is These are the same values that appeared under the column for stage 1. The re-
turn after stage 2 is It is equal to the return from stage 2 plus the total return before stage 2.
Stage 3 is solved in the same way as stages 1 and 2. The following table presents the
solution for stage 3; look at each row and make sure that you understand the meaning of each
value.
f2.
f1f1.
s1 = s2 - 3d2.
s1
d2
s2(s2).
s1,
r2 = 8d2.
r2,
d2s2
d2
s2
8/10/2019 Rsh Qam11 Module02
15/24
8/10/2019 Rsh Qam11 Module02
16/24
M2-16 MODULE 2 DYNAMIC PROGRAMMING
We start with shipping 6 units at stage 4. Note that is 4 from the stage 4 calculations,
given that is 6. We use this value of 4 and go to the stage 3 calculations. We find the rows in
which is 4 and pick the row with the highest total return, In this row is 0 items with a
total return of 10. As a result, the number of units available, is still 4. We next go to the
calculations for stage 2 and then stage 1 in the same way. This gives us the optimal solution al-
ready described. see if you can find one of the alternate optimal solutions.
s2,(f3)
d3f3.s3
d4
s3
Key Equations
(M2-1)
The input to stage n. This is also the output from stage
(M2-2)
The decision at stage n.
(M2-3)
The return function, usually profit or loss, at stage n.
(M2-4)
The output from stage n. This is also the input to stagen - 1.
sn-1 = Outputfromstagen
rn = Returnatstagen
dn = Decisionatstagen
n + 1.
sn = Inputtostagen (M2-5)
A transformation function that allows us to go from
one stage to another.
(M2-6)
The general relationship that shows how the outputfrom any stage is a function of the input to the stage
and the decisions made at that stage.
(M2-7)
This equation gives the total return (profit or costs) at
any stage. It is obtained by summing the return at each
stage, rn.
fn = Totalreturnatstagen
sn-1 = tn1sn, dn2
tn = Transformationfunctionatstagen
Summary
Dynamic programming is a flexible, powerful technique. A
large number of problems can be solved using this technique,
including the shortest-route and knapsack problems. The
shortest-route problem finds the path through a network that
minimizes total distance traveled, and the knapsack problem
maximizes the value or return. An example of a knapsack prob-
lem is to determine what to ship in a cargo plane to maximize
total profits given the weight and size constraints of the cargo
plane. The dynamic programming technique requires four
steps: (1) Divide the original problem into stages, (2) solve the
last stage first, (3) work backward solving each subsequent
stage, and (4) obtain the optimal solution after all stages havebeen solved.
Dynamic programming requires that we specify stages,
state variables, decision variables, decision criteria, an optimal
policy, and a transformation function for each specific prob-
lem. Stages are logical subproblems. State variables are possi-
ble input values or beginning conditions. The decision
variables are the alternatives that we have at each stage, such
as which route to take in a shortest-route problem. The deci-
sion criterion is the objective of the problem, such as finding
the shortest route or maximizing total return in a knapsack
problem. The optimal policy helps us obtain an optimal solu-
tion at any stage, and the transformation function allows us to
go from one stage to the next.
Glossary
Decision Criterion A statement concerning the objective of
a dynamic programming problem.
Decision Variable The alternatives or possible decisions that
exist at each stage of a dynamic programming problem.
Dynamic Programming A quantitative technique that
works backward from the end of the problem to the begin-
ning of the problem in determining the best decision for a
number of interrelated decisions.
Optimal Policy A set of decision rules, developed as a result
of the decision criteria, that gives optimal decisions at any
stage of a dynamic programming problem.
Stage A logical subproblem in a dynamic programming
problem.
State Variable A term used in dynamic programming to
describe the possible beginning situations or conditions
of a stage.Transformation An algebraic statement that shows the
relationship between stages in a dynamic programming
problem.
8/10/2019 Rsh Qam11 Module02
17/24
SOLVED PROBLEM M2-
Solved Problem
Solved Problem M2-1
Lindsey Cortizan would like to travel from the university to her hometown over the holiday season.
road map from the university (node 1) to her home (node 7) is shown in Figure M2.9. What is the b
route that will minimize total distance traveled?
1
2
3
4
5
6
7
20
18
22
39
28
36
18
13
10
FIGURE M2.9
Road Map forLindsey Cortizan
1
4
3
2
6
5
7
20
18
22
39
28
36
18
13
10
Stage 3 Stage 2 Stage 1
50 41
28
52
10
13
FIGURE M2.10
Solution for theLindsey CortizanProblem
Solution
The solution to this problem is identical to the one presented earlier in the chapter. First, the problem
broken into three stages. See the network in Figure M2.10. Working backward, we start by solving sta
3. The closest and only distance from node 6 to 7 is 13, and the closest and only distance from node 5
node 7 is 10. We proceed to stage 2. The minimum distances are 52, 41, and 28 from node 4, node
and node 2 to node 7. Finally, we complete stage 3. The optimal solution is 50. The shortest route
1257 and is shown in the following network. This problem can also be solved using tables, as show
following the network solution.
Problem type: Minimization network
Number of stages: 3
Transition function type:
Recursion function type: fn = rn + fn-1
sn-1 = sn - dn
8/10/2019 Rsh Qam11 Module02
18/24
M2-18 MODULE 2 DYNAMIC PROGRAMMING
STAGE NUMBER OF DECISIONS
3 3
2 4
1 2
STAGE STARTING NODE ENDING NODE RETURN VALUE
3 1 2 22
1 3 18
1 4 20
2 2 5 18
2 6 36
3 6 28
4 6 39
1 5 7 10
6 7 13
STAGE 1
s1
d1
r1
s0
f0
f1
6 67 13 7 0 13
5 57 10 7 0 10
STAGE 2
s2
d2
r2
s1
f1
f2
4 46 39 6 13 52
3 36 28 6 13 41
2 26 36 6 13 49
25 18 5 10 28
STAGE 3
s3
d3
r3
s2
f2
f3
1 14 20 4 52 72
13 18 3 41 59
12 22 2 28 50
FINAL SOLUTION
STAGE OPTIMAL DECISION OPTIMAL RETURN
3 12 22
2 25 18
1 57 10
Total 50
8/10/2019 Rsh Qam11 Module02
19/24
SELF-TEST M2-
Self-Test
Before taking the self-test, refer back to the learning objectives at the beginning of the module, the notes in the margins, and
the glossary at the end of the module. Use the key at the back of the book to correct your answers. Restudy pages that correspond to any questions that you answered incorrectly or material you feel uncertain about.
c. a decision variable.
d. the optimal policy.e. a stage.
9. The statement that the distance from the beginning sta
is equal to the distance from the preceding stage to t
last node plus the distance from the given stage to the p
ceding stage is called
a. the transformation.
b. state variables.
c. decision variables.
d. the optimal policy.
e. stages.
10. In dynamic programming, is
a. the input to the stage n.
b. the decision at stage n.
c. the return at stage n.
d. the output of stage n.
e. none of the above.
11. The distance from the beginning stage is equal to the d
tance from the preceding stage to the last node plus t
distance for the given stage to the preceding stage. Th
relationship is used to solve which type of problem?
a. knapsack
b. JIT
c. shortest route
d. minimal spanning tree
e. maximal flow12. In using dynamic programming to solve a shortest-ro
problem, the distance from one point to the next would
called a
a. state.
b. stage.
c. return.
d. decision.
13. In using dynamic programming to solve a shortest-ro
problem, the decision variables at one stage of the pro
lem would be
a. the distances from one node to the next.
b. the possible arcs or routes that can be selected.c. the number of possible destination nodes.
d. the entering nodes.
14. In using dynamic programming to solve a shortest-ro
problem, the entering nodes would be called
a. the stages.
b. the state variables.
c. the returns.
d. the decision variables.
sn
1. Dynamic programming divides problems into
a. nodes.b. arcs.
c. decision stages.
d. branches.
e. variables.
2. Possible beginning situations or conditions of a dynamic
programming problem are called
a. stages.
b. state variables.
c. decision variables.
d. optimal policy.
e. transformation.
3. The statement concerning the objective of a dynamic pro-
gramming problem is called
a. stages.
b. state variables.
c. decision variables.
d. optimal policy.
e. decision criterion.
4. The first step of a dynamic programming problem is to
a. define the nodes.
b. define the arcs.
c. divide the original problem into stages.
d. determine the optimal policy.
e. none of the above.
5. In working a problem with dynamic programming, weusually
a. start at the first part of the problem and work forward
to the next parts.
b. start at the end of the problem and work backward.
c. start at the most expensive part of the problem.
d. start at the least expensive part of the problem.
6. An algebraic statement that reveals the relationship be-
tween stages is called
a. the transformation.
b. state variables.
c. decision variables.
d. the optimal policy.e. the decision criterion.
7. In this chapter, dynamic programming is used to solve
what type of problem?
a. quantity discount
b. just-in-time inventory
c. shortest route
d. minimal spanning tree
e. maximal flow
8. In dynamic programming terminology, a period or logical
subproblem is called
a. the transformation.
b. a state variable.
8/10/2019 Rsh Qam11 Module02
20/24
M2-20 MODULE 2 DYNAMIC PROGRAMMING
Discussion Questions and Problems
Discussion Questions
M2-1 What is a stage in dynamic programming?
M2-2 What is the difference between a state variable and a
decision variable?
M2-3 Describe the meaning and use of a decision crite-
rion.M2-4 Do all dynamic programming problems require an
optimal policy?
M2-5 Why is transformation important for dynamic pro-
gramming problems?
Problems
M2-6 Refer to Figure M2.1. What is the shortest route be-
tween Rice and Dixieville if the road between Hope
and Georgetown is improved and the distance is re-
duce to 4 miles?
M2-7 Due to road construction between Georgetown and
Dixieville, a detour must be taken through countryroads (Figure M2.1). Unfortunately, this detour has
increased the distance from Georgetown to Dix-
ieville to 14 miles. What should George do? Should
he take a different route?
M2-8 The Rice brothers have a gold mine between Rice
and Brown. In their zeal to find gold, they have
blown up the road between Rice and Brown. The
road will not be in service for five months. What
should George do? Refer to Figure M2.1.
M2-9 The Rice brothers (Figure M2.1) wish to determine
the potential savings from using the shortest route
from Rice to Dixieville rather than just randomly se-lecting any route. Use dynamic programming to find
the longest route from Rice to Dixieville. How much
farther is this than the shortest route?
M2-10 Solve the shortest-route problem for the network
shown in Figure M2.11.
M2-11 Solve the shortest-route problem for the network
shown in Figure M2.12.
M2-12 Mail Express, an overnight mail service, delivers
mail to customers throughout the United States,Canada, and Mexico. Fortunately, Mail Express has
additional capacity on one of its cargo planes. To
maximize profits, Mail Express takes shipments
from local manufacturing plants to warehouses for
other companies. Currently, there is room for an-
other 6 tons. The following table shows the items
that can be shipped, their weights, the expected
profit for each, and the number of available parts.
How many units of each item do you suggest that
Mail Express ship?
1 5
2
3
7
4
8
6
9
5
4
11
10
4
12
6
10
4
6
2
4
6
FIGURE M2.11
(for Problem M2-10)
ITEMS TO BE SHIPPED
WEIGHT NUMBERITEM (TONS) PROFIT/UNIT AVAILABLE
1 1 $3 6
2 2 9 1
3 3 8 2
4 1 2 2
M2-13 Leslie Bessler must travel from her hometown to
Denver to see her friend Austin. Given the road map
of Figure M2.13, what route will minimize the dis-
tance that Leslie travels?
M2-14 An air cargo company has the shipping requirementsshown in the accompanying table. Two planes are
available with a total capacity of 11 tons. How many
of each item should be shipped to maximize profits?
8/10/2019 Rsh Qam11 Module02
21/24
DISCUSSION QUESTIONS AND PROBLEMS M2-
1
3
2
6
5
4
7
8
9
10
11
3
4
5
3
2
2
7
5
2
2
4 3
3
3
6
FIGURE M2.12
(for Problem M2-11)
M2-15 Because of a new manufacturing and packaging pro-
cedure, the weight of item 2 in Problem M2-14 can
be cut in half. Does this change the number or types
of items that can be shipped by the air transport
company?
M2-16 What is the shortest route through the network in
Figure M2.14?
M2-17 Refer to Problem M2-16. The road between node 6
and node 11 is no longer in service due to construc-
tion. What is the shortest route, given this situation?
M2-18 In Problem M2-16, what is the shortest distan
from node 6 to the ending node? How does th
change if the road between node 6 and node 11 is
longer in service (see Problem M2-17)?
M2-19 Paige Schwartz is planning to go camping in BBend National Park. She will carry a backpack w
food and other items, and she wants to carry no mo
than 18 pounds of food. The four possible food item
she is considering each have certain nutritional ing
dients. These items and their weights are as follow
How many of each item should paige carry if s
wishes to maximize the total nutritional units?
1
4
3
2
5
6
9
8
7
10
11
12
2
1
3 3
4
4
3 3
2
3
2
3
4
2
3
3
FIGURE M2.13
(for Problem M2-13)
ITEMS TO BE SHIPPED
WEIGHT NUMBERITEM (TONS) PROFIT/UNIT AVAILABLE
1 1 $3 62 2 9 1
3 3 8 2
4 2 5 5
5 5 8 6
6 1 2 2
ITEM WEIGHT (POUNDS) NUTRITIONAL UNIT
A 3 600
B 4 1,000
C 5 1,500
D 2 300
8/10/2019 Rsh Qam11 Module02
22/24
M2-22 MODULE 2 DYNAMIC PROGRAMMING
1 3
2
47
8
6
59
10
11
12
13
16
15
14
17
18
19 20
4
2
3 3
3
4
3
4
3
3
4
3
3
2
4
3
4
4
4
3
2
5
2
2
2
FIGURE M2.14
(for Problem M2-16)
Case Study
United Trucking
Like many other trucking operations, United Trucking gotstarted with one truck and one ownerJudson Maclay. Judson
is an individualist and has always liked to do things his way.
He was a fast driver, and many people called the 800 number
on the back of his truck when he worked for Hartmann Truck-
ing. After two years with Hartmann and numerous calls about
his bad driving, Judson decided to go out on his own. He
started United Trucking.
In the early days of United Trucking, Judson was the only
driver. On the back of his truck was the message: How do you
like my driving? Call 1-800-AMI-FAST. He was convinced
that some people actually tried to call the number. Soon, a num-
ber of truck operators had the same or similar messages on the
back of their trucks. After three years of operation, Judson had
15 other trucks and drivers working for him. He traded his driv-
ing skills for office and management skills. Although 1-800-
AMI-FAST was no longer visible, Judson decided to never
place an 800 number on the back of any of his trucks. If some-
one really wanted to complain, they could look up United
Trucking in the phone book.
Judson liked to innovate with his trucking company. He
knew that he could make more money by keeping his trucks
full. Thus he decided to institute a discount trucking service. He
gave substantial discounts to customers that would accept deliv-
ery to the West Coast within two weeks. Customers got a great
price, and he made more money and kept his trucks full. Overtime, Judson developed steady customers that would usually
have loads to go at the discounted price. On one shipment, hehad an available capacity of 10 tons in several trucks going to
the West Coast. Ten items can be shipped at discount. The
weight, profit, and number of items available are shown in the
following table:
WEIGHTITEM (TONS) PROFIT/UNIT AVAILABLE
1 1 $10 2
2 1 10 1
3 2 5 3
4 1 7 20
5 3 25 2
6 1 11 1
7 4 30 2
8 3 50 1
9 1 10 2
10 1 5 4
Discussion Questions
1. What do you recommend that Judson do?
2. If the total available capacity were 20 tons, how wouldthis change Judsons decision?
See our Internet home page at www.pearsonhighered.com/render for this additional case study:
Briarcliff Electronics.
Internet Case Study
8/10/2019 Rsh Qam11 Module02
23/24
BIBLIOGRAPHY M2-
Bibliography
Bellman, R. E.Dynamic Programming. Princeton, NJ: Princeton University
Press, 1957.
Bourland, Karla. Parallel-Machine Scheduling with Fractional Operator Re-
quirements,IEE Transactions (September 1994): 56.
Elmaghraby, Salah. Resource Allocation via Dynamic Programming,
European Journal of Operations Research (January 1993): 199215.
El-Rayes, Khaled, et al. Optimized Scheduling for Highway Construction,Transactions of AACE International (February 1997): 311315.
Hillard, Michael R., et al. Scheduling the Operation Desert Storm Airlift: An
Advanced Automated Scheduling Support System,Interfaces 21, 1
(JanuaryFebruary 1992): 131146.
Howard, R. A.Dynamic Programming. Cambridge, MA: The MIT Press,
1960.
Ibarake, Toshihide, and Yuichi Nakamura. A Dynamic Programming Meth
for Single Machine Scheduling,European Journal of Operations
Research (July 1994): 7282.
Idem, Fidelis, and A. M. Reisman. An Approach to Planning for Physician
Requirements in Developing Countries Using Dynamic ProgrammingOperations Research (July 1990): 607618.
Stokes, Jeffrey, et al. Optimal Marketing of Nursery Crops from Container
Based Production Systems,American Journal of Agricultural
Economics (February 1997): 235246.
8/10/2019 Rsh Qam11 Module02
24/24