1

421-682 Engineering Systems

Management

Better decision-making

Associate Professor Andrew Wirth,

Room 125 CMC House Building 173

telephone 8344 4852

email:wirtha@unimelb.edu.au,

www.mame.mu.oz.au/~wirth

Recommended text: W.L. Winston Operations Research (Thomson, 4th ed , 2004) (with student editions of LINGO, LINDO…) [En: 003 WINS]

2

OPERATIONS RESEARCH OR

THE SCIENCE OF BETTER

Operations research (OR) is a scientific approach to the analysis of decision-making problems. Such an analysis will often involve the development of a mathematical

model. The early OR workers were a group of very bright engineers and physical scientists (five later won Nobel Prizes) who investigated operational aspects of certain military problems in the Second World War. An early OR success was the formulation, in 1947, of the simplex algorithm by George Dantzig to solve linear programs. The solution of mathematical models of OR problems generally involves the execution of an algorithm, a well-defined procedure.

3

Unfortunately certain algorithms require an inordinate amount of time to terminate. We say a problem is polynomial time (in class P) if there exists an algorithm which requires no more than kn

p elementary steps for some constants k and p, for all n sufficiently large, where n is the ‘problem size’. In such a case we say the algorithm is O(np). Note that it is the problem that is in class P (not the algorithm). Note also that we have been evasive about specifying what we mean by problem size. As a simple example, sorting is in P and standard sorting algorithms are O(n log n). Many OR problems are in P, but many, such as the travelling salesperson problem, discussed below, belong to a class called NP-hard for which no polynomial time exact algorithms are known. In such cases we often resort to heuristic (or approximate) solutions.

4

It is known that if we could find a fast, that is, polynomial time algorithm for any one NP-hard problem we could find one for all NP-hard problems. Most researchers believe that NP-hard problems are inherently more difficult and no fast algorithms exist for them. However, this has not yet been proved. In fact, if you could solve this problem, about problems, you would win $US 1 million. For further details see www.claymath.org It is fascinating to note that we can sometimes make practical use of NP-hardness. Current encryption algorithms use prime decompositions of large numbers. To decode the message requires knowledge of the prime factors of large numbers. To find prime factors of an n digit number may require an amount of time that grows exponentially with n.

5

Linear and integer programming and

sensitivity analysis

Linear Programming A vast number of decision-making problems may be modelled as linear programs (LPs) or their extensions. Very efficient solution algorithms are readily available to handle LPs. So it is useful to be able to model problems as LPs, whenever appropriate. We introduce the topic through a simple example. Example Pollution control

The combustion of fuel A (B) produces 3 (5) kg of pollutant per tonne of A (B). We wish to maximise power output given that one tonne of A (B) yields 10 (12) units of energy, at most 4 tonnes of fuel may be

6

processed per hour and no more than 15 kg of pollutant may be generated per hour. We can now identify three key features of an LP. Decision variables These are sometimes called control

variables. The decision maker has at least partial control over these entities. In this example we define two such variables: Let x (y) be the number of tonnes of fuel A (B) processed each hour. Constraints The values of x and y are restricted by certain conditions. Processing: x + y ≤ 4 (1) Pollution: 3x + 5y ≤ 15 (2)

7

Also we must have Non-negativity: x, y ≥ 0 (3) Objective function We wish to maximise the power, that is maximise z = 10x + 12y (4) Since there are only two variables we may easily represent the problem graphically:

3

4

4 5x

y

The set of points satisfying all the constraints is called the feasible region. It is convex and as will be stated below an

8

optimal solution may be found at a vertex of the feasible region. In this case we see, by simple enumeration

of the vertices, that 5

2,3

2

is optimal with

z = 43. That is, we should process 5

2 tonnes

of A and 3

2 tonnes of B per hour.

More generally, we have

Decision variables: x1, x2, …, xn ∈ R

That is we assume for now that the decision variables are continuous. Integer

programming considers the important but more difficult problem of discrete variables. In many practical problems n may be of the order of 104 or 105 or larger.

9

Constraints The variables are restricted through a number of inequalities

≥…

≤+…+

≤+…+

0,,&

&

...&

1

11

11111

n

mnmnm

nn

xx

bxaxa

bxaxa

(5)

where the aijs are constants Note that we may always replace

a11x1+…+a1nxn ≥ b1 by

−a11x1−…−a1nxn ≤ −b1

and also note that a11x1+…+a1nxn = b1

is equivalent to

10

a11x1+…+a1nxn ≤ b1

& − a11x1−…−a1nxn ≤ −b1

So we see that (5) is quite general, incorporating ≤ , = and ≥ constraints. Objective Function We shall, generally, assume that we wish to maximise z = c1x1 + … + cnxn (6) where the ci are constants. In brief a linear program is:

max c~

T x~

(7)

subject to A x

~≤ b

~ (8)

and x

~≥ 0

~ (9)

11

The set of points x~

∈Rn satisfying A x

~≤ b

~

and x~

≥ 0~

is called the feasible region.

The feasible region is convex.

Recall that a set S ⊆ Rn is convex if,

( ) SyxSyx ∈−+⇒≤≤∈~~~~

110,, λλλ

Note that the feasible region may be empty or it may be unbounded. Further, simple linear (and non-linear) programs are readily available as Solver

examples in Excel (see the Tools menu for Solver).

The feasible region is a convex set. If the feasible region is non-empty and bounded then an optimal solution to the linear program (7) – (9) exists at a vertex of the feasible region.

12

It is intuitively clear what a vertex is. Formally, a point v of S is a vertex (extreme

point) of S if v~

= λ x~

+ 1− λ( )y~ for some

10 and S,~~

<<∈ λyx

imply that x~

= y~

= v~.

We say S is bounded if for all x~

∈S, x~

≤ κ

for some κ, where x~

= x12 +…+xn

2 .

Note that the feasible region may be empty, or it may be unbounded. Note also that analysis (calculus) methods are not useful in the solution of LPs. They simply tell us that the optimum is at a vertex. However, for our problem with n variables and m equations

there may be as many as m

nC vertices (if

nm ≤ ). So, for a relatively small problem with, say, n = 100, m = 20 we may have more than 1019 vertices. If we check 109 vertices per second it would take over 300 years to check all vertices.

13

In fact, LP is a polynomial time problem, that is, it is in the class P. The first practically effective solution method for LPs, the simplex algorithm, was proposed in 1947 by George Dantzig. It has been verified experimentally that on average the

algorithm is O (m × (m + n) × min (m, n)). (In the worst case the simplex algorithm is exponential time but this does not appear to occur in practical problems.) The simplex algorithm climbs up the exterior of the feasible region from one vertex to a better vertex until it reaches the ‘top’. If the problem is feasible and bounded the simplex algorithm terminates after finitely many steps at an optimal vertex. On average the simplex algorithm is polynomial time, as stated earlier. Other LP algorithms, which are always polynomial time, have also been proposed.

14

Example Project Management with

Crashing

We can represent activities and precedence of, say, a construction project by the following type of activity on arc network

In the diagram activity B must be preceded by activities A and C. Let the duration of activity A be denoted by dA and so on. How soon can the project be finished? We introduce the following variables: let tA … be start time of A K

3B2

1

A

C

15

The precedence relations yield the following constraints

B A At t d≥ + K We wish to minimise

iii

dt + max

However, max is a nonlinear function! To model this problem as a linear program we introduce an extra variable y:

i it d y+ ≤ for all i and now

minimize y Clearly at the optimum y is, in fact, the maximum of the activity completion times.

16

Suppose that with the current available resources the earliest finish time exceeds some target time T. Can we, by crashing, that is, by decreasing activity times by xi

at cost cixi, finish in time and at what minimum cost? This problem can also be modelled as a linear program:

min i i

i

c x∑

such that

B A A At t d x≥ + − …

A Ax d≤ …

i i it d x T+ − ≤ for all i

, 0i it x ≥ for all i

17

Integer programming examples

An integer program is just like a linear program except for the fact that some of the variables must take integer values. In some cases they must take binary (0 or 1) values. Computationally, this apparently small change, may have serious consequences. Integer programming is NP-hard. That is, the fastest currently known algorithms are exponential time. So we may need to rely on heuristic methods to find fast, approximate solutions. Example Bin Packing

Suppose we have n objects of mass q1,…,qn and we wish to store them in as few bins as possible. Each bin has a capacity of mass Q. We assume we cannot split the objects between bins. For there to be any solution

18

we must assume qi ≤Q for all i. Clearly n bins will suffice but can we do better? Suppose that we use m bins and try to minimise m. We may then construct the following model. Let

→

=otherwise0

bin object if1 jixij

ixm

j

ij allfor 11

∑=

=

jQqxn

i

iij allfor 1

∑=

≤

xij = 0 or 1 for all i,j

The objective is now to

minimize m

19

Unfortunately this is function is not a linear function of the decision variables, but rather of the number of such variables. We reformulate the above to convert it to an integer program. Recall that n bins will certainly suffice. So suppose that we have n bins available. Let

= otherwise 0

used is bin if 1 jy j j = 1,…,n

So the IP is:

∑=

n

j

jy1

min

subject to xij ≤yj for all i, j

ix

n

j

ij ∀=∑=

11

jQqxn

i

iij ∀≤∑=

1

xij , yj = 0 or 1 for all i, j

20

The following is essentially the bin packing problem with additional constraints.

Example Assembly line balancing

Suppose that to complete production we are required to carry out n tasks with durations

1, , np pK , and we must produce at least one unit in time C (cycle time). We assume that

Cpi ≤ for all i. We also assume that certain tasks precede others. Clearly n workstations suffice, but what is the smallest number of workstations that will achieve this cycle time?

Workstation i

Workstation j

21

Introduce the following decision variables

1 task station

0 otherwiseij

i jx

→=

= otherwise 0

used is station if 1 jy j

Constraints

ixn

j

ij allfor 11

=∑=

jCxpn

i

iji allfor 1

≤∑=

xij ≤yj for all i, j

.....

3213

212

11

αααβ

ααβ

αβ

xxxx

xxx

xx

++≤

+≤

≤

for every pair of tasks where α precedes β xij , yj = 0 or 1 for all i, j

22

Objective function

∑=

n

j

jy1

min

What if tasks α and β must [must not] be carried out at the same workstation? Example Scheduling

Most scheduling problems are NP-hard. The following is arguably the simplest possible scheduling problem. It is, in fact, in P, that is, a polynomial time algorithm exists for this problem, but we shall, somewhat perversely, model it as an IP. Single machine, mean completion time

minimisation

Given n jobs with duration times ti, and one

processor, minimize the mean completion time.

23

We assume the jobs cannot be split and the processor can only do one job at a time. We assume all jobs are available at time t = 0. For example, if

t2 = 1 t1 = 3 t3 = 2 → Processor

Then mean completion time = [2 + (2 + 3) + (2 + 3 + 1)]/3 = 13/3 More generally, we can construct the following IP model. Let

=otherwise0

positionth in the is job if1 jixij

24

minimize ( ) njnxt

n

j

n

i

iji −+∑∑= =

11 1

subject to

ixn

j

ij allfor 11

=∑=

jxn

i

ij allfor 11

=∑=

and xij = 0 or 1 for all i, j For this problem there is a fast (non-IP) solution. Can you guess what it is? Generally scheduling problems have no exact polynomial time algorithms. For example, if we have two or more parallel processors and we wish to minimise the largest completion time. This new problem is, in fact, in some ways analogous to bin packing. However, now we have only two bins whose capacity can vary. This similarity has been put to use in a powerful heuristic. But that is another story!

25

Example Either Or Constraint Suppose we wish to solve an LP with the following constraint:

x ≤ a or x ≥ b (a < b) We introduce a 0–1 variable y:

x ≤ a + My and x ≥ b – (1 – y)M

where M is LARGE (in the context of the particular problem) and y = 0 or 1. We have shifted the ‘or’ so that we now have a 0–1 integer program. What if we have: a1 ≤ x ≤ a2 or b1 ≤ x ≤ b2 ?

26

Example The Travelling Salesperson

Problem (TSP)

This is probably the most famous integer programming problem of all.

i

j

cij

Assume we have a network of n nodes, some of which are linked by arcs. The length of the arc from i and j (if it exists) is denoted by cij. A tour is a directed path through the network that visits each node exactly once and returns to the starting point. What is the shortest tour?

27

We shall model this as an IP. Let

=otherwise 0

to fromdirectly goes tour theof arc one if 1 jixij

So we wish to

minimize ∑∑= =

n

i

n

j

ijij xc1 1

subject to

ixn

j

ij allfor 11

=∑=

jxn

i

ij allfor 11

=∑=

and xij = 0 or 1 Unfortunately the above constraints are insufficient. They allow subtours:

28

For example, if n = 7 and

x13 = x32 = x21 = x46 = x65 = x57 = x74 = 1

all other xij = 0

we satisfy all of the above constraints and yet we have

3

2

1

6

7

5

4

To avoid this problem we may use xijk

variables (the kth arc goes from node i to node j) or we introduce n additional non-negative variables ui and (n – 1) (n – 2)

constraints: ui − u j + nxij ≤ n −1, 2 ≤ i, j ≤ n, i ≠ j

29

So, if n = 7, say, we cannot have x46 = x65 = x57 = x74 = 1, as in the

example above, since u4 – u6 + 7x46 ≤6,

u6 – u5 + 7x65 ≤ 6,

u5 – u7 + 7x57 ≤ 6

and u7 – u4 + 7x74 ≤ 6

imply that 28 ≤24 , a contradiction. If we let ui be the position in a tour when

node i is visited with say node 1 visited in position 1 then we have a solution to the u

constraints. So adding these constraints to the earlier ones will eliminate subtours but still allow for all genuine tours. Hence we have formulated the TSP as an integer program. Note that since we have

n − 1( )! ~ 2π n − 1( )

n − 1

e

n−1

possible different tours, total enumeration is infeasible for large n.

30

Sensitivity Analysis

How does the solution to a linear program change with respect changes to A, b, c? We shall consider changes in b and c only.

For a max problem the marginal price is

the quantity i

opt

b

z

∂

∂

Note that for a ≤ constraint the marginal price is non-negative. Since increasing bi

gives a larger feasible region. If bi were the

available amount of a scarce resource, and z were profit, then the marginal price would be the highest price that should be paid

for one extra unit of the resource. Note also that we change one constraint at a time. Marginal prices are piecewise constant functions. Outside the allowable range, different variables may enter the solution.