Fall 2006, Sep. 5 and 7Fall 2006, Sep. 5 and 7 ELEC5270-001/6270-001 Lecture 4ELEC5270-001/6270-001 Lecture 4 11
ELEC 5270-001/6270-001 (Fall 2006)ELEC 5270-001/6270-001 (Fall 2006)Low-Power Design of Electronic CircuitsLow-Power Design of Electronic Circuits
(Formerly ELEC 5970-003/6970-003)(Formerly ELEC 5970-003/6970-003)
Linear Programming – A Mathematical Linear Programming – A Mathematical Optimization TechniqueOptimization Technique
Vishwani D. AgrawalVishwani D. AgrawalJames J. Danaher ProfessorJames J. Danaher Professor
Department of Electrical and Computer Department of Electrical and Computer EngineeringEngineering
Auburn University, Auburn, AL 36849Auburn University, Auburn, AL 36849http://www.eng.auburn.edu/~vagrawalhttp://www.eng.auburn.edu/~vagrawal
[email protected]@eng.auburn.edu
Fall 2006, Sep. 5 and 7Fall 2006, Sep. 5 and 7 ELEC5270-001/6270-001 Lecture 4ELEC5270-001/6270-001 Lecture 4 22
What is Linear ProgrammingWhat is Linear Programming Linear programming (LP) is a Linear programming (LP) is a
mathematical method for selecting mathematical method for selecting the best solution from the available the best solution from the available solutions of a problem.solutions of a problem.
Method:Method: State the problem and define variables State the problem and define variables
whose values will be determined.whose values will be determined. Develop a linear programming model:Develop a linear programming model:
Write the problem as an optimization formula (a Write the problem as an optimization formula (a linear expression to be minimized or maximized)linear expression to be minimized or maximized)
Write a set of linear constraintsWrite a set of linear constraints An available LP solver (computer program) An available LP solver (computer program)
gives the values of variables.gives the values of variables.
Fall 2006, Sep. 5 and 7Fall 2006, Sep. 5 and 7 ELEC5270-001/6270-001 Lecture 4ELEC5270-001/6270-001 Lecture 4 33
Types of LPsTypes of LPs LP – all variables are real.LP – all variables are real. ILP – all variables are integers.ILP – all variables are integers. MILP – some variables are integers, MILP – some variables are integers,
others are real.others are real. A reference:A reference:
S. I. Gass, An Illustrated Guide to Linear S. I. Gass, An Illustrated Guide to Linear Programming, New York: Dover, 1990.Programming, New York: Dover, 1990.
Fall 2006, Sep. 5 and 7Fall 2006, Sep. 5 and 7 ELEC5270-001/6270-001 Lecture 4ELEC5270-001/6270-001 Lecture 4 44
A Single-Variable ProblemA Single-Variable Problem
Consider variable xConsider variable x Problem: find the maximum value of Problem: find the maximum value of
x subject to constraint, 0 ≤ x ≤ 15.x subject to constraint, 0 ≤ x ≤ 15. Solution: x = 15.Solution: x = 15.
Fall 2006, Sep. 5 and 7Fall 2006, Sep. 5 and 7 ELEC5270-001/6270-001 Lecture 4ELEC5270-001/6270-001 Lecture 4 55
Single Variable Problem Single Variable Problem (Cont.)(Cont.)
Consider more complex constraints:Consider more complex constraints: Maximize x, subject to following constraintsMaximize x, subject to following constraints
x ≥ 0x ≥ 0 (1)(1) 5x ≤ 755x ≤ 75 (2)(2) 6x ≤ 306x ≤ 30 (3)(3) x ≤ 10x ≤ 10 (4)(4)
0 5 10 15x (1)
(2)(3)
(4)
All constraints satisfied Solution, x = 5
Fall 2006, Sep. 5 and 7Fall 2006, Sep. 5 and 7 ELEC5270-001/6270-001 Lecture 4ELEC5270-001/6270-001 Lecture 4 66
A Two-Variable ProblemA Two-Variable Problem Manufacture of xManufacture of x11 chairs and x chairs and x22
tables:tables: Maximize profit, P = 45xMaximize profit, P = 45x11 + 80x + 80x22 dollars dollars Subject to resource constraints:Subject to resource constraints:
400 boards of wood,400 boards of wood, 5x5x11 + 20x + 20x22 ≤ 400 ≤ 400 (1)(1) 450 man-hours of labor,450 man-hours of labor, 10x10x11 + 15x + 15x22 ≤ ≤
450450 (2)(2) xx11 ≥ 0 ≥ 0 (3)(3) xx22 ≥ 0 ≥ 0 (4)(4)
Fall 2006, Sep. 5 and 7Fall 2006, Sep. 5 and 7 ELEC5270-001/6270-001 Lecture 4ELEC5270-001/6270-001 Lecture 4 77
Solution: Two-Variable Solution: Two-Variable ProblemProblem
Chairs, x1
Tab
les,
x2
(1)
(2)
0 10 20 30 40 50 60 70 80 90
40
30
20
10
0
(24, 14)
Profi
t increasing
decresing
P = 2200
P = 0
Best solution: 24 chairs, 14 tablesProfit = 45×24 + 80×14 = 2200 dollars
Fall 2006, Sep. 5 and 7Fall 2006, Sep. 5 and 7 ELEC5270-001/6270-001 Lecture 4ELEC5270-001/6270-001 Lecture 4 88
Change Chair Profit, Change Chair Profit, $64/Unit$64/Unit
Manufacture of xManufacture of x11 chairs and x chairs and x22 tables:tables: Maximize profit, P = 64xMaximize profit, P = 64x11 + 80x + 80x22 dollars dollars Subject to resource constraints:Subject to resource constraints:
400 boards of wood,400 boards of wood, 5x5x11 + 20x + 20x22 ≤ 400 ≤ 400 (1)(1) 450 man-hours of labor,450 man-hours of labor, 10x10x11 + 15x + 15x22 ≤ ≤
450450 (2)(2) xx11 ≥ 0 ≥ 0 (3)(3) xx22 ≥ 0 ≥ 0 (4)(4)
Fall 2006, Sep. 5 and 7Fall 2006, Sep. 5 and 7 ELEC5270-001/6270-001 Lecture 4ELEC5270-001/6270-001 Lecture 4 99
Solution: $64 Profit/ChairSolution: $64 Profit/Chair
Chairs, x1
Tab
les,
x2
(1)
(2)
Profi
t increasing
decresing
P = 2880
P = 0
Best solution: 45 chairs, 0 tablesProfit = 64×45 + 80×0 = 2880 dollars
0 10 20 30 40 50 60 70 80 90
(24, 14)
40
30
20
10
0
Fall 2006, Sep. 5 and 7Fall 2006, Sep. 5 and 7 ELEC5270-001/6270-001 Lecture 4ELEC5270-001/6270-001 Lecture 4 1010
Primal-Dual ProblemsPrimal-Dual Problems Primal problemPrimal problem Variables:Variables:
xx11 (number of chairs) (number of chairs) xx22 (number of tables) (number of tables)
Maximize profit 45xMaximize profit 45x11 + 80x + 80x22
Subject to:Subject to: 5x5x11 + 20x + 20x22 ≥ 400≥ 400 10x10x11 + 15x + 15x22 ≥ 450≥ 450 xx11 ≥ 0≥ 0 xx22 ≥ 0≥ 0
Solution:Solution: xx11 = 24 chairs, x = 24 chairs, x22 = 14 = 14
tablestables Profit = $2200Profit = $2200
Dual ProblemDual Problem Variables:Variables:
ww11 ($ cost/board of wood) ($ cost/board of wood) ww22 ($ cost/man-hour) ($ cost/man-hour)
Minimize cost 400w1 + Minimize cost 400w1 + 450w2450w2
Subject to:Subject to: 5w5w11 +10w +10w22 ≥ 45≥ 45 20w20w11 + 15w + 15w2 2 ≥ 80≥ 80 ww11 ≥ 0≥ 0 ww22 ≥ 0≥ 0
Solution:Solution: ww11 = $1, w = $1, w22 = $4 = $4 Cost = $2200Cost = $2200
Fall 2006, Sep. 5 and 7Fall 2006, Sep. 5 and 7 ELEC5270-001/6270-001 Lecture 4ELEC5270-001/6270-001 Lecture 4 1111
The Duality TheoremThe Duality Theorem
If the primal has a finite optimal If the primal has a finite optimal solution, so does the dual, and the solution, so does the dual, and the optimum values of the objective optimum values of the objective functions are equal.functions are equal.
Fall 2006, Sep. 5 and 7Fall 2006, Sep. 5 and 7 ELEC5270-001/6270-001 Lecture 4ELEC5270-001/6270-001 Lecture 4 1212
LP for LP for nn Variables Variables n
minimize Σ cj xj Objective functionj =1 n
subject to Σ aij xj ≤ bi, i = 1, 2, . . ., mj =1
n
Σ cij xj = di, i = 1, 2, . . ., pj =1
Variables: xjConstants: cj, aij, bi, cij, di
Fall 2006, Sep. 5 and 7Fall 2006, Sep. 5 and 7 ELEC5270-001/6270-001 Lecture 4ELEC5270-001/6270-001 Lecture 4 1313
Algorithms for Solving LPAlgorithms for Solving LP
Simplex methodSimplex method G. B. Dantzig, G. B. Dantzig, Linear Programming and ExtensionLinear Programming and Extension, Princeton, , Princeton,
New Jersey, Princeton University Press, 1963.New Jersey, Princeton University Press, 1963. Ellipsoid methodEllipsoid method
L. G. Khachiyan, “A Polynomial Algorithm for Linear L. G. Khachiyan, “A Polynomial Algorithm for Linear Programming,” Programming,” Soviet Math. DoklSoviet Math. Dokl., vol. 20, pp. 191-194, ., vol. 20, pp. 191-194, 1984.1984.
Interior-point methodInterior-point method N. K. Karmarkar, “A New Polynomial-Time Algorithm for N. K. Karmarkar, “A New Polynomial-Time Algorithm for
Linear Programming,” Linear Programming,” CombinatoricaCombinatorica, vol. 4, pp. 373-395, , vol. 4, pp. 373-395, 1984.1984.
Course website of Prof. Lieven Vandenberghe (UCLA), Course website of Prof. Lieven Vandenberghe (UCLA), http://www.ee.ucla.edu/ee236a/ee236a.htmlhttp://www.ee.ucla.edu/ee236a/ee236a.html
Fall 2006, Sep. 5 and 7Fall 2006, Sep. 5 and 7 ELEC5270-001/6270-001 Lecture 4ELEC5270-001/6270-001 Lecture 4 1414
Basic Ideas of Solution Basic Ideas of Solution methodsmethods
Constraints
Extreme points
Objective function Constraints
Extreme points
Objective function
Simplex: search on extreme points.Interior-point methods: Successively iterate with interior spaces of analytic convex boundaries.
Fall 2006, Sep. 5 and 7Fall 2006, Sep. 5 and 7 ELEC5270-001/6270-001 Lecture 4ELEC5270-001/6270-001 Lecture 4 1515
Integer Linear Programming Integer Linear Programming (ILP)(ILP)
Variables are integers.Variables are integers. Complexity is exponential – higher Complexity is exponential – higher
than LP.than LP. LP relaxationLP relaxation
Convert all variables to real, preserve Convert all variables to real, preserve ranges.ranges.
LP solution provides guidance.LP solution provides guidance. Rounding LP solution can provide a non-Rounding LP solution can provide a non-
optimal solution.optimal solution.
Fall 2006, Sep. 5 and 7Fall 2006, Sep. 5 and 7 ELEC5270-001/6270-001 Lecture 4ELEC5270-001/6270-001 Lecture 4 1616
ILP Example: Test ILP Example: Test MinimizationMinimization
A combinational circuit has n test vectors that A combinational circuit has n test vectors that detect m faults. Each test detects a subset of detect m faults. Each test detects a subset of faults. Find the smallest subset of test vectors faults. Find the smallest subset of test vectors that detects all m faults.that detects all m faults.
ILP model:ILP model: Assign an integer ti Assign an integer ti εε [0,1] variable to ith test [0,1] variable to ith test
vector such that ti = 1, if we select ti, otherwise ti= vector such that ti = 1, if we select ti, otherwise ti= 0.0.
Define an integer constant fij Define an integer constant fij εε [0,1] such that fij = [0,1] such that fij = 1, if ith vector detects jth fault, otherwise fij = 0. 1, if ith vector detects jth fault, otherwise fij = 0. Values of constants fij are determined by fault Values of constants fij are determined by fault simulation. simulation.
Fall 2006, Sep. 5 and 7Fall 2006, Sep. 5 and 7 ELEC5270-001/6270-001 Lecture 4ELEC5270-001/6270-001 Lecture 4 1717
Test Minimization by ILPTest Minimization by ILP n
minimize Σ ti Objective function i=1 n
subject to Σ fij ti ≥ 1, j = 1, 2, . . ., mi=1
Fall 2006, Sep. 5 and 7Fall 2006, Sep. 5 and 7 ELEC5270-001/6270-001 Lecture 4ELEC5270-001/6270-001 Lecture 4 1818
3V3F: A 3-Vector 3-Fault 3V3F: A 3-Vector 3-Fault ExampleExample
fijfij i=1i=1 i=2i=2 i=3i=3
j=1j=1 11 11 00
j=2j=2 00 11 11
j=3j=3 11 00 11
Test vector i
Fau
lt j
Variables: t1, t2, t3 εε [0,1]
Minimize t1 + t2 + t3
Subject to:
t1 + t2 ≥ 1
t2 + t3 ≥ 1
t1 + t3 ≥ 1
Fall 2006, Sep. 5 and 7Fall 2006, Sep. 5 and 7 ELEC5270-001/6270-001 Lecture 4ELEC5270-001/6270-001 Lecture 4 1919
3V3F: Solution Space3V3F: Solution Space
Non-optimum solution
t1
t2
t3
1
1
1
LP solution(0.5, 0.5, 0.5) and iterative rounding
ILP solutions(optimum)
Fall 2006, Sep. 5 and 7Fall 2006, Sep. 5 and 7 ELEC5270-001/6270-001 Lecture 4ELEC5270-001/6270-001 Lecture 4 2020
Characteristics of ILPCharacteristics of ILP Worst-case complexity is exponential in Worst-case complexity is exponential in
number of variables.number of variables. Linear programming (LP) relaxation, where Linear programming (LP) relaxation, where
integer variables are treated as real, gives integer variables are treated as real, gives a lower bound on the objective function.a lower bound on the objective function.
Recursive roundingRecursive rounding of relaxed LP solution of relaxed LP solution to nearest integers gives an approximate to nearest integers gives an approximate solution to the ILP problem.solution to the ILP problem. K. R. Kantipudi and V. D. Agrawal, “A Reduced K. R. Kantipudi and V. D. Agrawal, “A Reduced
Complexity Algorithm for Minimizing Complexity Algorithm for Minimizing NN-Detect -Detect Tests,” Tests,” Proc. 20Proc. 20thth International Conf. VLSI International Conf. VLSI DesignDesign, January 2007, pp. 492-497., January 2007, pp. 492-497.
Fall 2006, Sep. 5 and 7Fall 2006, Sep. 5 and 7 ELEC5270-001/6270-001 Lecture 4ELEC5270-001/6270-001 Lecture 4 2121
3V3F: LP Relaxation and 3V3F: LP Relaxation and RoundingRounding
ILP – Variables: t1, t2, t3 εε [0,1]
Minimize t1 + t2 + t3
Subject to:
t1 + t2 ≥ 1
t2 + t3 ≥ 1
t1 + t3 ≥ 1
LP relaxation: t1, t2, t3 εε (0.0, 1.0)
Solution: t1 = t2 = t3 = 0.5
Iterative rounding:
(1) round one variable, t1 = 1.0 Two-variable LP problem: Minimize t2 + t3 subject to t2 + t3 ≥ 1.0 LP solution t2 = t3 = 0.5
(2) round a variable, t2 = 1.0 ILP constraints are satisfied solution is t1 = 1, t2 = 1, t3 = 0
Fall 2006, Sep. 5 and 7Fall 2006, Sep. 5 and 7 ELEC5270-001/6270-001 Lecture 4ELEC5270-001/6270-001 Lecture 4 2222
Recursive Rounding Recursive Rounding AlgorithmAlgorithm
1.1. Obtain a relaxed LP solution. Obtain a relaxed LP solution. StopStop if if each variable in the solution is an each variable in the solution is an integer.integer.
2.2. Round the variable closest to an Round the variable closest to an integer.integer.
3.3. Remove any constraints that are Remove any constraints that are now unconditionally satisfied.now unconditionally satisfied.
4.4. Go to step 1.Go to step 1.
Fall 2006, Sep. 5 and 7Fall 2006, Sep. 5 and 7 ELEC5270-001/6270-001 Lecture 4ELEC5270-001/6270-001 Lecture 4 2323
Recursive RoundingRecursive Rounding ILP has exponential complexity.ILP has exponential complexity. Recursive rounding:Recursive rounding:
ILP is transformed into k LPs with ILP is transformed into k LPs with progressively reducing number of variables.progressively reducing number of variables.
Number of LPs, k, is the size of the final Number of LPs, k, is the size of the final solution, i.e., the number of non-zero solution, i.e., the number of non-zero variables in the test minimization problem.variables in the test minimization problem.
Recursive rounding complexity is k × O(nRecursive rounding complexity is k × O(npp), ), where k ≤ n, n is number of variables.where k ≤ n, n is number of variables.
Fall 2006, Sep. 5 and 7Fall 2006, Sep. 5 and 7 ELEC5270-001/6270-001 Lecture 4ELEC5270-001/6270-001 Lecture 4 2424
Four-Bit ALU CircuitFour-Bit ALU Circuit
Initial Initial vectorsvectors
ILPILP Recursive roundingRecursive rounding
VectorsVectors CPU sCPU s VectorsVectors CPU sCPU s
285285 1414 0.650.65 1414 0.420.42
400400 1313 1.071.07 1313 1.001.00
500500 1212 4.384.38 1313 3.003.00
1,0001,000 1212 4.174.17 1212 3.003.00
5,0005,000 1212 12.9512.95 1212 9.009.00
10,00010,000 1212 34.6134.61 1212 17.017.0
16,38416,384 1212 87.4787.47 1212 37.037.0
Fall 2006, Sep. 5 and 7Fall 2006, Sep. 5 and 7 ELEC5270-001/6270-001 Lecture 4ELEC5270-001/6270-001 Lecture 4 2525
CPU Time: ILP vs. Recursive CPU Time: ILP vs. Recursive RoundingRounding
0 5,000 10,000 15,000 Vectors
100
75
50
25
0
ILP
Recursive Rounding
CPU
s
Fall 2006, Sep. 5 and 7Fall 2006, Sep. 5 and 7 ELEC5270-001/6270-001 Lecture 4ELEC5270-001/6270-001 Lecture 4 2626
N-Detect Tests (N = 5)N-Detect Tests (N = 5)
CircuiCircuitt
UnoptimizUnoptimized vectorsed vectors
Relaxed LP/Recur. Relaxed LP/Recur. roundingrounding ILP (exact)ILP (exact)
Lower Lower boundbound
Min. Min. vectorvector
ssCPU sCPU s
Min. Min. vectorvector
ssCPU sCPU s
c432c432 608608 196.38196.38 197197 1.01.0 197197 1.01.0
c499c499 379379 260.00260.00 260260 1.21.2 260260 2.32.3
c880c880 1,0231,023 125.97125.97 128128 14.014.0 127127 881.8881.8
c1355c1355 755755 420.00420.00 420420 3.23.2 420420 4.44.4
c1908c1908 1,0551,055 543.00543.00 543543 4.64.6 543543 6.96.9
c2670c2670 959959 477.00477.00 477477 4.74.7 477477 7.27.2
c3540c3540 1,9711,971 467.25467.25 477477 72.072.0 471471 20008.20008.
55
c5315c5315 1,0791,079 374.33374.33 377377 18.018.0 376376 40.740.7
c6288c6288 243243 52.5252.52 5757 39.039.0 5757 34740.34740.
00
c7552c7552 2,1652,165 841.00841.00 841841 52.052.0 841841 114.3114.3
Fall 2006, Sep. 5 and 7Fall 2006, Sep. 5 and 7 ELEC5270-001/6270-001 Lecture 4ELEC5270-001/6270-001 Lecture 4 2727
Finding LP/ILP SolversFinding LP/ILP Solvers R. Fourer, D. M. Gay and B. W. Kernighan, R. Fourer, D. M. Gay and B. W. Kernighan,
AMPL: A Modeling Language for AMPL: A Modeling Language for Mathematical ProgrammingMathematical Programming, South San , South San Francisco, California: Scientific Press, 1993. Francisco, California: Scientific Press, 1993. Several of programs described in this book Several of programs described in this book are available to Auburn users.are available to Auburn users.
MATLAB?MATLAB? Search the web. Many programs with small Search the web. Many programs with small
number of variables can be downloaded free.number of variables can be downloaded free.