July 2013
Improvements to Benders' decomposition: systematic classification
and performance comparison in a Transmission Expansion Planning
problem
Sara Lumbreras & Andrés Ramos
2 Instituto de Investigación Tecnológica
Escuela Técnica Superior de Ingeniería ICAI
Sara Lumbreras, July 2013
Motivation
A quick introduction
Master problem improvement techniques
Subproblem improvement techniques
Case Study: Transmission Expansion Planning
Results
Conclusions
Agenda
4 Instituto de Investigación Tecnológica
Escuela Técnica Superior de Ingeniería ICAI
Sara Lumbreras, July 2013
It was first proposed in 1962
In its 50 years of history it has been applied to very diverse fields:
Scheduling
Routing (e.g. traveling salesman) & vehicle assignment
Computer network design
Capacity allocation in telecommunications networks
Manufacturing system design
Portfolio optimization
Specially, in Power systems
Generation, Transmission & Distribution Expansion Planning [Pereira et al, ‘A
decomposition approach to automated generation/ transmission expansion planning’, 1985]
Hydrothermal co-ordination
Unit Commitment
Many improvements to the basic strategy have been proposed in an uncoordinated way, so they are not easily accessible or related to the cases where they can be useful
Benders, ‘Partitioning procedures for solving mixed-variables programming problems’, 1962
Motivation
MotivationA quick
introduction
Master problem
techniques
Subproblem
techniques
Case Study:
TEPResults Conclusions
5 Instituto de Investigación Tecnológica
Escuela Técnica Superior de Ingeniería ICAI
Sara Lumbreras, July 2013
This work aims at filling this gap:
Classifies the improvements to Benders’ decomposition that appear in the literature
Proposes new improvements to add to the ones in the literature
Links these methodologies to the cases where they can be useful
Compares their performance in a particularly relevant case study based on Transmission Expansion Planning
Given its characteristics this problem has been extensively solved with Benders’ decomposition
Motivation (II)
MotivationA quick
introduction
Master problem
techniques
Subproblem
techniques
Case Study:
TEPResults Conclusions
7 Instituto de Investigación Tecnológica
Escuela Técnica Superior de Ingeniería ICAI
Sara Lumbreras, July 2013
Benders’ decomposition is applied to a two-stage stochastic (linear) problem
First-stage decisions (also known as investment decisions) are taken before an uncertain event occurs
Second-stage decisions (also known as operation decisions or recourse decisions) are adjusted after the uncertainty has been revealed
Second-stage scenarios can be solved independently
Both stages are coupled through the tender constraints
First-stage decisions “complicate” the resolution of the problem
A quick introduction
1st stage 2nd stage Stochastic
scenarios
Constraints of the 1st stage
Constraints that link both
stages (tender constraints)
Constraints of the 2nd stage
MotivationA quick
introduction
Master problem
techniques
Subproblem
techniques
Case Study:
TEPResults Conclusions
8 Instituto de Investigación Tecnológica
Escuela Técnica Superior de Ingeniería ICAI
Sara Lumbreras, July 2013
Benders’ decomposition divides the two-stage stochastic linear programming problem in two parts (master problem & subproblems) that are solved iteratively until convergence
The process builds an increasingly accurate piecewise linear approximation of the recourse problem (Benders’ cuts)
A quick introduction (cont’ed)
lower
bound
upper
bound
Complete problem
Master problem
Subproblem (solved independently for each scenario)
iterations
current iteration
MotivationA quick
introduction
Master problem
techniques
Subproblem
techniques
Case Study:
TEPResults Conclusions
9 Instituto de Investigación Tecnológica
Escuela Técnica Superior de Ingeniería ICAI
Sara Lumbreras, July 2013
Feasibility cuts
When a certain 1st-stage solution is infeasible in the subproblem, a feasibility cut is generated with the aim of eliminating solutions that do not abide the constraints
Usually, the sum of infeasibilities is minimized
The resulting feasibility cut has the form:
A quick introduction (cont’ed)
infeasibility
1lδ =
0lδ =
optimality cut
feasibility cut
For uniform notation
MotivationA quick
introduction
Master problem
techniques
Subproblem
techniques
Case Study:
TEPResults Conclusions
10 Instituto de Investigación Tecnológica
Escuela Técnica Superior de Ingeniería ICAI
Sara Lumbreras, July 2013
This method is likely to yield time savings in the following cases:
First-stage variables complicate the resolution of the problem
Decoupling the resolution of 2nd stage scenarios is specially important
Computational time taken by this method is related to the number of complicating variables
Therefore, it is likely to yield time savings if the number of complicating variables is small
The master problem and the subproblem have different natures (and hence it would be convenient to solve them with different methods, e.g. NLP / MIP vs LP)
Both conditions apply to TEP
Scope of Benders’ decomposition
MotivationA quick
introduction
Master problem
techniques
Subproblem
techniques
Case Study:
TEPResults Conclusions
11 Instituto de Investigación Tecnológica
Escuela Técnica Superior de Ingeniería ICAI
Sara Lumbreras, July 2013
Improvement techniques respond to the need to reduce either:
Master problem solution time (solution time can be long because of size, integral variables or a large number of cuts), or
Subproblem solution time (solution time can be long because of a large number of scenarios or 2nd stage conditions)
Guides for their suitability will be provided
However, the practical benefit achieved will have to be assessed on a case-by-case basis
Most of them involve trade-offs that must be assessed individually
A case study based on Transmission Expansion Planning will be presented
Classification of improvement techniques
MotivationA quick
introduction
Master problem
techniques
Subproblem
techniques
Case Study:
TEPResults Conclusions
13 Instituto de Investigación Tecnológica
Escuela Técnica Superior de Ingeniería ICAI
Sara Lumbreras, July 2013
Master problem relaxations
Binary variables complicate the resolution, so solving the LP relaxation is much quicker
It yields valid Benders’ cuts
Linear-first approaches solve first the relaxed problem
Recently, [Lumbreras & Ramos, ‘Optimal design of an offshore wind farm applying
decomposition strategies’, 2012] proposed a progressive discretization of variables to improve convergence (semi-relaxed cuts)
Improvements that modify the solution technique (I)
upper bound
lower bound
variable
discretization
convergence
Can be useful if the
linearized problem can
be solved much quicker
and the integrality gap is
small
MotivationA quick
introduction
Master problem
techniques
Subproblem
techniques
Case Study:
TEPResults Conclusions
14 Instituto de Investigación Tecnológica
Escuela Técnica Superior de Ingeniería ICAI
Sara Lumbreras, July 2013
Sub-optimal master problem solutions
Early terminations of the master problem might improve convergence
Using any feasible solution [Fortz & Poss, An improved Benders decomposition applied to a
multi-layer network design problem, 2008]
Rounding linearized solutions [Costa et al, ‘Accelerating Benders decomposition with heuristic master solutions’, 2012]
(feasibility must be checked)
Alternative strategies to find master proposals
Non-classical optimization techniques (e.g., metaheuristics) can be applied to find near-optimal solutions in affordable times
[Poojari & Beasley,’ Improving Benders’ decomposition using a Genetic Algorithm’, 2009]
Improvements that modify the solution technique (II)
gap
time
Can be useful if the sub-
optimal solution is not far
from the optimal one and is
obtained in a reduced time
MotivationA quick
introduction
Master problem
techniques
Subproblem
techniques
Case Study:
TEPResults Conclusions
15 Instituto de Investigación Tecnológica
Escuela Técnica Superior de Ingeniería ICAI
Sara Lumbreras, July 2013
Box-step method (introduction of additional constraints):
Ideally they are conditions that should be met at the optimum, so they only eliminate not useful zones of the feasible region
From expert opinion
Data mining
If the additional constraint is active at the optimal solution the constraint should be relaxed by an specified amount (the step) and the problem resolved.
Use of a more suitable solution technique
E.g. Constraint programming and logic-based methods [Benoist et al, ‘Constraint programming contribution to Benders’ decomposition’, 2002]
In many cases, complex problems include many logical constraints that make use of auxiliary binary variables
This greatly complicates the problem
There are techniques that have been specially developed for these problems, where the logical constraints are included explicitly (e.g. LOGMIP)
Improvements that modify the solution technique (III)
Can be useful if there is a technique that
is better suited to the specific problem
Can be useful if these constraints
are available
MotivationA quick
introduction
Master problem
techniques
Subproblem
techniques
Case Study:
TEPResults Conclusions
16 Instituto de Investigación Tecnológica
Escuela Técnica Superior de Ingeniería ICAI
Sara Lumbreras, July 2013
Extracting non-dominated cuts / Pareto-optimal cuts [Sherali & Fraticelli, ‘On generating maximal nondominated Benders cuts’, 2011]
A cut or constraint dominates another if any evaluation of first stage decisions is larger than or equal to the previous one
Generating covering cuts [Saharidis et al, Accelerating benders method using covering cut bundle generation, 2010]
Generating cuts so that they carry the maximum amount of information possible
They include the maximum possible number of 1st-stage variables
Removing inactive cuts [Marin & Salmeron, ‘Electricity capacity expansion under uncertain demand: decomposition approaches’, 1998]
Or dynamically defining the master problem so that only the cuts that are likely to be active constraints are taken into account
Improvements that modify Benders’ cuts (I)
Can be useful if there
are too many cuts and
most are dominated
Can be useful if there are too many cuts
Can be useful if there are too many cuts
MotivationA quick
introduction
Master problem
techniques
Subproblem
techniques
Case Study:
TEPResults Conclusions
17 Instituto de Investigación Tecnológica
Escuela Técnica Superior de Ingeniería ICAI
Sara Lumbreras, July 2013
Minimal Infeasible Subsystems (MIS) can be used to modify the way feasibility cuts are calculated
[Saharidis & Ierapitrou, ‘Improving benders decomposition using maximum feasible subsistem (MFS) cut generation, 2009]
Instead of minimizing the sum of infeasibilities the problem minimizes the number of equations that are infeasible
This enables faster convergence in some cases
Conversely, if most of the solutions are infeasible, it is possible to keep a maximum feasible set to derive optimality cuts to better guide the search
Modifications to Benders’ cuts (II)
Can be useful if most of
the cuts are feasibility cuts
MotivationA quick
introduction
Master problem
techniques
Subproblem
techniques
Case Study:
TEPResults Conclusions
19 Instituto de Investigación Tecnológica
Escuela Técnica Superior de Ingeniería ICAI
Sara Lumbreras, July 2013
Subtree partition [Birge & Loveaux, ‘A multicut algorithm for stochastic linear programs’, 1988]
The second stage scenarios can be arranged differently
In general, the most efficient arrangement cannot be known beforehand (tradeoff between the accuracy of the cuts and solution time for the master problem)
Scenario structure design (I)
ws: wind scenario
ss: system state
MotivationA quick
introduction
Master problem
techniques
Subproblem
techniques
Case Study:
TEPResults Conclusions
20 Instituto de Investigación Tecnológica
Escuela Técnica Superior de Ingeniería ICAI
Sara Lumbreras, July 2013
Scenario aggregation: the second stage corresponding to the scenarios with the highest impact on the final design are added to the master problem
[Cerisola & Ramos, ‘Node aggregation in stochastic nested benders decomposition applied to hydrothermal coordination’,2000]
The master problem proposes solutions that are closer to the optimal
Convergence speed can be increased
Scenario structure design (II)
Can be useful if one of the
scenarios has a much
higher impact on the final
solution
MotivationA quick
introduction
Master problem
techniques
Subproblem
techniques
Case Study:
TEPResults Conclusions
21 Instituto de Investigación Tecnológica
Escuela Técnica Superior de Ingeniería ICAI
Sara Lumbreras, July 2013
Solution technique modifications
Bunching: if the 2nd stage scenarios are similar we can solve only one scenario and calculate the others using the calculated sensitivities
[Birge & Loveaux, ‘Introduction to Stochastic Programming’, 1988]
Application of specific algorithms [Marin & Salmeron, ‘Electricity Capacity Expansion Models Under Uncertain Demand. Decomposition Approaches’, 1988]
Or even a series of increasingly accurate versions of the subproblem (as long as they have increasing values of the o.f.) so that time is not wasted in the first few iterations
[Romero & Monticelli, ‘A hierarchical decomposition approach for transmission network expansion planning’, 1994]
Sub-optimal subproblem solutions (Zakeri’s cuts) [Zakeri et al, ‘Inexact Cuts in Benders Decomposition’, 1999]
Any infeasible solution in the subproblem will give a valid cut (can use IPM)
Can be useful if there are
many similar scenarios being
calculated
Can be useful if there is an specific more efficient technique available
Can be useful if solving the subproblem
until optimality, even for one scenario, is
computationally very expensive
MotivationA quick
introduction
Master problem
techniques
Subproblem
techniques
Case Study:
TEPResults Conclusions
23 Instituto de Investigación Tecnológica
Escuela Técnica Superior de Ingeniería ICAI
Sara Lumbreras, July 2013
The power system in a nutshell
Supply is composed by generators of different technologies with different operation costs
Demand has to be served instantaneously (there is no possibility for storage)
The transmission network enables (and constrains) the physical transactions (and the operation outcome)
Flows follow Kirchhoff’s laws, so bilateral transactions do not have a physical meaning
MotivationA quick
introduction
Master problem
techniques
Subproblem
techniques
Case Study:
TEPResults Conclusions
24 Instituto de Investigación Tecnológica
Escuela Técnica Superior de Ingeniería ICAI
Sara Lumbreras, July 2013
Transmission Expansion Planning
The Transmission Expansion Planning (TEP) problem consists in deciding the optimal transmission investments (lines or otherwise) that should be added to the existing transmission network in order to minimize the total investment and operation costs (generation costs and reliability).
MotivationA quick
introduction
Master problem
techniques
Subproblem
techniques
Case Study:
TEPResults Conclusions
25 Instituto de Investigación Tecnológica
Escuela Técnica Superior de Ingeniería ICAI
Sara Lumbreras, July 2013
Simplified formulation
The TEP problem can be formulated as MIP in a centralized, cost-minimization framework
Minimize the sum of investment and operation costs
Subject to Kirchhoff’s laws
Respecting generation and transmission limits
MotivationA quick
introduction
Master problem
techniques
Subproblem
techniques
Case Study:
TEPResults Conclusions
min InvC OpC+ijc ijc
ijc CL
InvC ic x∈
= ∑
( )i i i
i
OpC g cg pns pnlt= +∑
:jic ijc i i i i
jic ijc
f f g d pns σ∈
− + = −∑ ∑
i j
ijc
ijc
f ijc ELX
θ θ−= ∈
( )
( )
1 :
1 :
i j
ijc ijc ijc ijc
ijc
i j
ijc ijc ijc ijc
ijc
f M xX
f M xX
ijc CL
θ θρ
θ θρ
+
−
−− ≥− −
−− ≤ −
∈
:
:
( )
ijc ijc ijc ijc
ijc ijc ijc ijc
f f x
f f x
ijc CL EL
π
π
+
−
≥−
≤
∈ ∪
, 0i i i ii
g g g pns d≤ ≤ ≤ ≤
26 Instituto de Investigación Tecnológica
Escuela Técnica Superior de Ingeniería ICAI
Sara Lumbreras, July 2013
Benders’ decomposition applied to TEP
The master problem takes the form:
And the subproblem:
2
2 2
min
( )
( ) ( )
1, ...,
ijc ijcijc CL
l lT lT
ijc ijc ijcijc
lT lT l
ijc ijc ijc ijc ijc
c x P
Z f
M x x
l n
ω ω
ω
ω ω ω ω
ω ω
π π
ρ ρ
∈ ∈Ω
+ −
+ −
+ Θ
Θ ≥ + − +
− −=
∑ ∑
∑
2,min ( )i i
n
i i ig pns
i
Z g cg pns pnltω ω
ω ω ω
ω
= +∑
:jic ijc i i i i
jic ijc
f f g d pnsω ω ω ω ω ωσ− + = −∑ ∑
i j
ijc
ijc
f ijc ELX
ω ω
ωθ θ−
= ∈
( )
( )
1 :
1 :
i j
ijc ijc ijc ijc
ijc
i j
ijc ijc ijc ijc
ijc
f M xX
f M xX
ijc CL
ω ω
ω ω
ω ω
ω ω
θ θρ
θ θρ
+
−
−− ≥− −
−− ≤ −
∈
:
:
( )
ijc ijc ijc ijc
ijc ijc ijc ijc
f f x
f f x
ijc CL EL
ω ω ω
ω ω ω
π
π
+
−
≥−
≤
∈ ∪
MotivationA quick
introduction
Master problem
techniques
Subproblem
techniques
Case Study:
TEPResults Conclusions
, 0i i i ii
g g g pns d≤ ≤ ≤ ≤
27 Instituto de Investigación Tecnológica
Escuela Técnica Superior de Ingeniería ICAI
Sara Lumbreras, July 2013
The ratio of time spent solving the master problem and the subproblem is the most important factor in the decision
In all cases, time spent in the subproblem is less than 12%
Master problem modification techniques are more appropriate.
Selection of the improvement techniques
Master Subproblem Decomposition
Case Eq Var D var Time (s) Eq Var Time (s) It. Time
(s)
Time per it.
Garver 14 14 9 0.6 15 45 0.4 5 5.1 1.0
46N 162 82 79 1.0 208 359 0.0 161 162.6 1.0
46N 100
scenarios
7402 181 79 9.7 208 359 1.2 75 820.6 10.9
87N 124 155 152 2.9 181 748 0.0 123 364.5 3.0
MotivationA quick
introduction
Master problem
techniques
Subproblem
techniques
Case Study:
TEPResults Conclusions
28 Instituto de Investigación Tecnológica
Escuela Técnica Superior de Ingeniería ICAI
Sara Lumbreras, July 2013
Are relaxations potentially interesting?
The largest case study, with 152 discrete variables in the master problem, is solved in 2.9s for MIP and 0.1s in a relaxed version (85% savings)
Master problem relaxations should be explored
- Linear first
- Semi-relaxed cuts
Are suboptimal master solutions interesting?
We examine the tolerance-performance curve
Values around 2.5% seem interesting
Selection of the improvement techniques (cont’ed)
0
20
40
60
80
100
120
140
160
180
0
0.2
0.4
0.6
0.8
1
1.2
1.4
1.6
1.8
0% 1% 2% 3% 4% 5% 6% 7% 8% 9% 10%
Iterations
Solution tim
e (s)
Relative optimality tolerance
MotivationA quick
introduction
Master problem
techniques
Subproblem
techniques
Case Study:
TEPResults Conclusions
29 Instituto de Investigación Tecnológica
Escuela Técnica Superior de Ingeniería ICAI
Sara Lumbreras, July 2013
Selection of the improvement techniques (cont’ed)
Other master problem modification techniques
Box-step techniques could be interesting in general in problems where an additional constraint can be easily imposed. In this case we can limit investment. Values of 85%, 100% and 115% of optimal investment were used
Alternative solution techniques (other than MIP) are not available for this problem
Modifications to Benders cuts are not interesting in this case
The number of generated cuts is not excessively large, so cover cuts or cut removal are not indicated
The subproblem is feasible in all cases, so MIS is not applicable
ij ijij
c x MaxInv∈
≤∑C
MotivationA quick
introduction
Master problem
techniques
Subproblem
techniques
Case Study:
TEPResults Conclusions
31 Instituto de Investigación Tecnológica
Escuela Técnica Superior de Ingeniería ICAI
Sara Lumbreras, July 2013
Results
Only the largest case study is reported for the sake of clarity
Semi-relaxed cuts and suboptimal master solutions offer consistent savings that can be above 50% of solution time
Linear-first approaches and box-step methods are not able to provide consistent results
87N Time (s) % Savings
Benders' decomposition 364.5
Linear first 374.7 -3%
Semi relaxed cuts 162.4 55%
Suboptimal master εMaster = 5%
εStep = 0.025%
106.5 71%
Suboptimal master εMaster = 2.5%
εStep = 0%
232.5 36%
Suboptimal master Highest tolerance 237.1 35%
Boxstep Initial maximum investment
= 85% optimal
193.0 47%
Boxstep Initial maximum investment
= 100% optimal
135.8 63%
Boxstep Initial maximum investment
= 115% optimal
551.3 -51%
MotivationA quick
introduction
Master problem
techniques
Subproblem
techniques
Case Study:
TEPResults Conclusions
33 Instituto de Investigación Tecnológica
Escuela Técnica Superior de Ingeniería ICAI
Sara Lumbreras, July 2013
Benders’ decomposition is a key tool in stochastic optimization which divides the problem in:
A master problem that optimizes the first stage and a piecewise linear approximation of the second stage costs
A subproblem which optimizes the second stage and creates the piecewise approximation by means of primal and marginal information
The method can bring substantial benefits when:
First-stage variables complicate the resolution of the problem
Master problem and subproblems have a different nature
Conclusions (I)
MotivationA quick
introduction
Master problem
techniques
Subproblem
techniques
Case Study:
TEPResults Conclusions
34 Instituto de Investigación Tecnológica
Escuela Técnica Superior de Ingeniería ICAI
Sara Lumbreras, July 2013
A slow master problem can be accelerated:
Modifying the solution method
Relaxations
Sub-optimal solutions
Box-step
Use of a more suitable technique
Modifying the calculation of cuts:
Nondominated cuts / Pareto optimal cuts
Covering cuts
Removing inactive cuts
Minimal Infeasible Subsystems
Conclusions (II)
MotivationA quick
introduction
Master problem
techniques
Subproblem
techniques
Case Study:
TEPResults Conclusions
35 Instituto de Investigación Tecnológica
Escuela Técnica Superior de Ingeniería ICAI
Sara Lumbreras, July 2013
A slow subproblem can be accelerated:
Selecting the most suitable structure for the scenario tree
Scenario aggregation
Bunching
Application of specific algorithms
Sub-optimal solutions
A case study based on Transmission Expansion Planning has demonstrated
How to select the most promising improvement techniques for a particular problem
Results show that for this case the semi-relaxed cuts (proposed by the authors) and sub-optimal master resolution were able to consistently offer savings above 50%
Conclusions (III)
MotivationA quick
introduction
Master problem
techniques
Subproblem
techniques
Case Study:
TEPResults Conclusions
Instituto de Investigación Tecnológica
Santa Cruz de Marcenado, 26
28015 Madrid
Tel +34 91 542 28 00
Fax + 34 91 542 31 76
www.upcomillas.es