Date post: | 06-May-2015 |
Category: |
Documents |
Upload: | alkis-vazacopoulos |
View: | 308 times |
Download: | 5 times |
Advanced Modeling of Industrial Optimization Problems:
Beyond Algebraic & Algorithmic Modeling Languages
J.D. Kelly & A. Vazacopoulos
Industrial [email protected] & [email protected]
September 10, 2012
UOPSS: Modeling the Objects
● UOPSS = Unit-Operation-Port-State Superstructure.● Unit-operations are the structural objects combining
"physical" units with "procedural" operations (eg. Equipment x Activities or Machine x Job).
● Port-states are the structural points where resources, etc. are consumed & produced.
● UOPS superstructure models the hyper-network of UOPS to PSUO connectivity.
● UOPSS objects define the "keys" of the problem or model.
UOPSS: Modeling the Objects
Unit-Operation 1 Unit-Operation 2
Port-State 1
Port-State 2
UOPSS: Modeling the Objects
● UOPSS is designed for batch, continuous and dimensional process industries and has the following types & sub-types:● ProcessB, ProcessC, ProcessD, Perimeter,
Pool, Pipeline, Pileline, Parcel & Port (In/Out).● Blender, Splicer, Splitter, Separator, Reactor,
Fractionator & Blackbox for Processes.● Stock, Utility, Utensil, Time & Task for Ports.
QLQP: Modeling the Attributes ● QLQP = Quantity-Logic-Quality Phenomena.
● Phenomenological is different from/orthogonal to hierarchical, structural, spatial & temporal dimensions.
● Quanties = rates, flows, holdups, yields, durations,● Logic = setups, startups, switchovers, shutdowns,
sequences, slots,● Qualities = densities, components, properties,
conditions, coefficients,● Logistics is the combination of Quanity & Logic
(MILP) and Quality is the combination of Quantity & Quality (NLP).
QLQP: Modeling the Attributes U O
P S
Attributes
U O
P S Attributes
U O
P S
AttributesUO
PS
UO
PS
Phenomenological Attributes of Quantity, Logic and Quality
Advanced Modeling
Baker, T.E., Ladson, L.S., “Successive Linear Programming at Exxon”, Management Science, 31, 1985.
Algorithmic-Form
Algebraic-Form
Allegoric-Form (Figurative Language)
Advanced Modeling
● Philosophy = "Configure v. Code".● Allows the modeler or user the ability to focus on
the important aspects of the industrial optimization problem (IOP) instead of its algebra & algorithms.
● Most of the variables and constraints found in an IOP can be generalized or formalized using UOPSS objects and QLQP attributes.● We currently have over 125 different variable-types &
over 185 different constraint-types including over 100 sets/lists & 150 parameters (and counting ...).
Advanced Modeling System: IMPRESS
● IMPRESS = Industrial Modeling & Presolving System.
● Problems are configured either interfacing with a flat-file language (IML = Industrial Modeling Language) or interactively using a programming language such as Python, Java, C#, C++, C or Fortran (IMI = Industrial Modeling Interchange).
● We currently have bindings to several linear and nonlinear programming solvers such as COINMP, GLPK, LPSOLVE, SCIP, XPRESS, XPRESS-SLP, CONOPT, IPOPT, KNITRO, NOVA & SLPQP.
Advanced Modeling System: IMPRESS
● IMPRESS uses the following resources or entities to formulate or model the IOP:
– SETS & LISTS = integer- & string-keyed to store integers.– CATALOGS = integer-keyed to store strings.– PARAMETERS = integer-keyed to store reals.– VARIABLES = integer-keyed to store complexes (Re/Im).– CONSTRAINTS = integer-keyed to store complexes.– DERIVATIVES = 2 integer & 1 real parallel vectors.– EXPRESSIONS = 1 integer & 1 real parallel vectors.– FORMULAS = integer-keyed to store 1 integer & 1 complex
parallel vectors.
Key Differences with AML's ● AML's use sorted database table technology to store
and manipulate sparse data.● IMPRESS also stores the sparse data in a database
table format (i.e., coordinate, triplets, ordered-pairs, non-zeros, etc.) but manipulates the data using a proprietary referencing technique for spot or random-access.
● AML's manage the time-period dimension like any other key, index or cursor in the sparse data table.● IMPRESS inherently manages the time-period dimension
as a separate appendable-array or vector which is manipulated using whole-array or vectorized processing (much faster as the number of time-periods increases).
Key Differences with AML's ● AML's always export each nonlinear constraint
instance as a prefix/postfix (RPN) tokenized expression in byte-code.● IMPRESS also does this for nonlinear solvers such as
XPRESS-SLP & LINDO-SLP but can also be embedded directly into function callbacks required by CONOPT, IPOPT, KNITRO, NOVA & SLPQP in machine-code (requires substantially less memory).
● Some AML's perform their own mostly linear presolving calculations (AMPL & AIMMS).● IMPRESS performs LP primal presolving as well as
nonlinear presolving such as detecting if nonlinearly declared constraints are linear after presolve.
Important Features ● Digitization Engine:
● All time-varying data such as orders, transactions or events are entered in continuous-time where they are either discretized (uniform) or distributed (non-uniform) into time-periods over the time-horizon.– This also includes a compression & continuosization on the
solution data to remove temporal redundancy.
● Differentiation Engine: ● All 1st-order partial derivatives are either supplied
analytically or computed numerically of analytical quality using complex-step differencing & graph-colouring. * The modeler or user is not responsible for providing derivatives.
Important Features
● Documentation Engine:● All constraints are externalized into a "human-
readable" form as well as all other entities such as sets, lists, parameters, variables & formulas have annotations available.
● Diagnostic Engine (Work-in-Progress ...):● Experience says that over 95% of IOP's have
infeasibilities that occur in the linear part of the model where these can be quickly identified in LP primal presolve. The rest require artificial or what we call excursion variables in both the integer and nonlinear parts of the model and are generated automatically.
Important Features ● Demarcation Engine (Openings, Wet-Plant Problem):
● Optimizing into the future is the goal but respecting the past & present is vital to properly achieving this i.e., the "demarcation" between the past & future time-lines.
Addressing the previous activities currently occurring in the problem must be respected as we look-ahead into the future.
For example, if a unit-operation has a minimum run-length of 10-hours & if it started 2-hours in the past then there is a minimum of 8-hours left i.e., it can be shutdown after 8-hours into the future.– It also includes respesting future operations when at a certain
future time a known event is to occur (predictive- or preventative- maintenance).
System Architecture: SIIMPL
● SIIMPL = Server, Interfacer, Interacter, Modeler, Presolver Libraries.
● There are five DLL components in IMPRESS:● Server = data modeling & presolving routines.● Interfacer = parsing for the language.● Interacter = inserting, updating & viewing routines for
the interchange. ● Modeler = formulating of the variables, constraints,
derivatives & expressions including "dependent" sets, lists & parameters.
● Presolver = bindings for 3rd-party solving-systems.
Types of Optimization & Estimation ● IMPRESS is designed both for industrial decision-making & data-mining problems deployed off-line, in-line & on-line such as:● Planning & Scheduling Optimization (active).● Data Reconciliation & Regression Estimation (passive).● Real-Time Control & Optimization (active).● Monitoring, Tracking & Tracing (passive).
– The terms active & passive imply the "degree of causality". Active models must be "causal" (cause & effect amongst variables) & passive models may or may not be causal (no cause & effect required) also known as "observational".
Types of Optimization & Estimation ● IMPRESS directly manages the integration of
planning & scheduling decision-making by exploiting its hierarchical nature:● Planning decisions are implemented as either lower &
upper hard bounds (min 1-norm excursions) or as target soft bounds (min 2-norm deviations) in the scheduling.
● IMPRESS can also model "hybrid" planning & scheduling problems i.e., a mix of planning & scheduling constraints in the same model/horizon:● Both "big" & "small" time-buckets or periods are included
in the problem by allowing one or more operations on the same unit for the same time-period (single or multi-use).
Types of Optimization & Estimation ● IMPRESS inherently manages the integration of the
logistics & quality QLQ phenomena in either planning or scheduling problems:● Logistics has quality proxy'd and Quality has logic fixed
but quanties are free/finite. This is a "truncated" Bender's Decomposition heuristic to avoid MINLP but is very effective in practice and is a natural mechanism to manage complexity.
● IMPRESS includes estimatibility diagnostics:● Observability, redundancy and variability estimates are
calculated as a postsolve for all measured (reconciled) and unmeasured (regressed) variables using novel sparse matrix techniques (no GJ, QR or SVD required).
Complexity Management ● IMPRESS is intended to assist with the ubiquitous
issue of "complexity management" in IOP's.● Managing uncertainty & hierarchy are somewhat
addressed but managing complexity is difficult:– how to balance accuracy, tractability & operability when
modeling & implementing the solution.
● What looks like uncertainty & hierarchy management issues may be the inability to model the IOP's complexity ...
– Perceived supply & demand uncertainty maybe the inability of the upstream producer & downstream consumer to manage their production or manufacturing as well as intermediate transportation or distribution.
Poor Man's Parallelism ● MPP = Multi-Problem Parallelism.● MPP does not require IMPRESS to be multi-
threaded unless the solver is multi-threaded (parallel Barrier or B&B) but allows multiple processes to be run concurrently managed by the operating-system.
● IMPRESS has a low memory footprint allowing many problems to be run simultaneously on multi-core, shared-memory computers with different starting-values, settings and/or solvers.● Due to the nonlinear and nonconvex nature of these
problems local solutions can and will occur so extra runs can be beneficial to potentially find better solutions.
MILP Example - SeqDepSwo ● SeqDepSwo = Sequence-dependent switchovers
w/ "repetitive-maintenance" (setup).– This instance can be considered as the "Prize-Collecting
Traveling-Salesman Problem" (PCTSP).– This formulation is from Kelly & Zyngier, "An Improved MILP
Modeling of Sequence-Dependent Switchovers for Discrete-Time Scheduling Problems",I&ECR, 46, 4964-4973, (2007).
● Horizon duration = 30-days, period duration = 1-day.● Production-line rate (semi-continuous) = 18-tons/day.● Ten-materials (3..5-day run-lengths), 3-families (3-3-4),
1-cleanout task (1-day run-length, rate = 2-tons/day).● Demands of 0..72-tons w/ release,due-dates = 0,30-days
w/ prices of $+1/ton & cleanout costs of $-1/ton.
MILP Example - SeqDepSwo ● Maximum profit = $540 = $1/ton * 18-tons/day * 30-days● Provably optimal profit = $502 = 1 * 18 * 28 + -1 * 2 * 1
– Schedules 3rd family and either the 1st or 2nd family in any sequence (3rd then 1st or 2nd OR 1st or 2nd then 3rd) with one cleanout instance & one idle day.
Cleanout
IdlePython 2.7 w/ Matplotlib 1.1.0
MILP Example - SeqDepSwo
Python 2.3 w/ Dia 0.97.2
UOPSS Stencil Sheet
Cleanout
Creates a *.UPS file which can be used in
both IML & IMI
MILP Example - SeqDepSwo ● Python 2.7 w/ ctypes w/ IMPRESS-IMI:
from ctypes import *
# Number of materials (including maintenance) and families.
nmaterials = 10+1nfamilies = 3
# Include IMPRESS constants and callbacks.
execfile("IMPRSimi.py")
# Set the problem path and name.
problem = c_char_p(b"C:/IndustrialAlgorithms/Problems/seqdepswo")
# Root the problem (initializes).
rtnstat = imis.IMPRSroot(problem)
# Reserve the problem memory.
rtnstat = imis.IMPRSreserve(problem,IMPRSall)
# Receive the chronological data.
dthp = c_double(-1.)dthf = c_double(30.)dtp = c_double(1.0)rtnstat = imii.IMPRSreceiveT(byref(dthp),byref(dthf),byref(dtp))
Past & Future Time Horizon Duration &
Time-Period Duration
MILP Example - SeqDepSwo ● Python 2.7 w/ ctypes w/ IMPRESS-IMI:
# Receive the construction data.
for i in range(nmaterials): if i < nmaterials-1: j = i+1 else: j = -1
... add Sj UO and UOPS
uname = c_char_p(b"PL") oname = c_char_p(str(j)) utype = c_char_p(b"processc") usubtype = c_char_p(b"") uuse = c_char_p(b"") rtnstat = imii.IMPRSreceiveUO(uname,oname,utype,usubtype,uuse,IMPRSkeep) pname = c_char_p(b"IN") sname = c_char_p(str(j)) ptype = c_char_p(b"in") psubtype = c_char_p(b"") puse = c_char_p(b"") rtnstat = imii.IMPRSreceiveUOPS(uname,oname,pname,sname,ptype,psubtype,puse,IMPRSkeep) pname = c_char_p(b"OUT") sname = c_char_p(str(j)) ptype = c_char_p(b"out") rtnstat = imii.IMPRSreceiveUOPS(uname,oname,pname,sname,ptype,psubtype,puse,IMPRSkeep)
... add Dj UO and UOPS
“PL” is a Continuous-Process
One inlet & one outlet port
“Keep” or “Erase”
MILP Example - SeqDepSwo ● Python 2.7 w/ ctypes w/ IMPRESS-IMI:
# Receive the connection data.
for i in range(nmaterials): if i < nmaterials-1: j = i+1 else: j = -1
uname = c_char_p(b"S"+str(j)) oname = c_char_p(str(j)) pname = c_char_p(b"OUT") sname = c_char_p(str(j)) uname2 = c_char_p(b"PL") oname2 = c_char_p(str(j)) pname2 = c_char_p(b"IN") sname2 = c_char_p(str(j)) rtnstat = imii.IMPRSreceiveUOPSPSUO(uname,oname,pname,sname,pname2,sname2,uname2,oname2,IMPRSkeep) uname = c_char_p(b"PL") oname = c_char_p(str(j)) pname = c_char_p(b"OUT") sname = c_char_p(str(j)) uname2 = c_char_p(b"D"+str(j)) oname2 = c_char_p(str(j)) pname2 = c_char_p(b"IN") sname2 = c_char_p(str(j)) rtnstat = imii.IMPRSreceiveUOPSPSUO(uname,oname,pname,sname,pname2,sname2,uname2,oname2,IMPRSkeep)
Routes, Paths, Streams, Transfers,
Movements, etc.
MILP Example - SeqDepSwo ● Python 2.7 w/ ctypes w/ IMPRESS-IMI:
# Receive the compatibility data.
j = 1for i in range(nmaterials-1): uname = c_char_p(b"PL") oname = c_char_p(str(i+1)) ogname = c_char_p(b"F"+str(j)) rtnstat = imii.IMPRSreceiveUOOG(uname,oname,ogname,IMPRSkeep) if divmod(i+1,round(float(nmaterials-1)/float(nfamilies)))[1] == 0: j = j + 1 j = min(j,nfamilies)
for i in range(nfamilies): for j in range(nfamilies): if i != j: uname = c_char_p(b"PL") ogname = c_char_p(b"F"+str(i+1)) ogname2 = c_char_p(b"F"+str(j+1)) oname = c_char_p(b"-1") rtnstat = imii.IMPRSreceiveUOGOGO(uname,ogname,ogname2,oname,IMPRSkeep)
Family = Operation-Group
“From-Family” to “To-Family” w/Repetitive Maintenance-Operation
F1 --> F2F1 --> F3F2 --> F1F2 --> F3F3 --> F1F3 --> F2
MILP Example - SeqDepSwo ● Python 2.7 w/ ctypes w/ IMPRESS-IMI:
# Receive the capacity data. upper = byref(c_double(18.))for i in range(nmaterials-1): uname = c_char_p(b"PL") oname = c_char_p(str(i+1)) rtnstat = imii.IMPRSreceiveUOrate(uname,oname,upper,upper,IMPRSkeep)uname = c_char_p(b"PL")oname = c_char_p(str(-1))upper = byref(c_double(2.))rtnstat = imii.IMPRSreceiveUOrate(uname,oname,upper,upper,IMPRSkeep)
lower = byref(c_double(1.))upper = byref(c_double(18.))for i in range(nmaterials): if i < nmaterials-1: j = i+1 else: j = -1 uname = c_char_p(b"PL") oname = c_char_p(str(j)) pname = c_char_p(b"IN") sname = c_char_p(str(j)) rtnstat = imii.IMPRSreceiveUOPSteerate(uname,oname,pname,sname,lower,upper,IMPRSkeep) rtnstat = imii.IMPRSreceiveUOPStotalrate(uname,oname,pname,sname,lower,upper,IMPRSkeep) rtnstat = imii.IMPRSreceiveUOPSyield(uname,oname,pname,sname,byref(c_double(1.0)),byref(c_double(1.0)),byref(c_double(0.0)),IMPRSkeep) pname = c_char_p(b"OUT") sname = c_char_p(str(j)) rtnstat = imii.IMPRSreceiveUOPSteerate(uname,oname,pname,sname,lower,upper,IMPRSkeep) rtnstat = imii.IMPRSreceiveUOPStotalrate(uname,oname,pname,sname,lower,upper,IMPRSkeep) rtnstat = imii.IMPRSreceiveUOPSyield(uname,oname,pname,sname,byref(c_double(1.0)),byref(c_double(1.0)),byref(c_double(0.0)),IMPRSkeep)
PL's Charge or Throughput Rate of 18.0 tons/day
Port Flow Rates & Yields
Total
TeeYield
MILP Example - SeqDepSwo ● Python 2.7 w/ ctypes w/ IMPRESS-IMI:
# Receive the constricture data.
lower = byref(c_double(3.))upper = byref(c_double(5.))for i in range(nmaterials-1): uname = c_char_p(b"PL") oname = c_char_p(str(i+1)) rtnstat = imii.IMPRSreceiveUOuptime(uname,oname,lower,upper,IMPRSkeep)uname = c_char_p(b"PL")oname = c_char_p(str(-1))lower = byref(c_double(1.))upper = byref(c_double(1.))rtnstat = imii.IMPRSreceiveUOuptime(uname,oname,lower,upper,IMPRSkeep)
# Receive the cost data. for i in range(nmaterials): if i < nmaterials-1: j = i+1 else: j = -1 uname = c_char_p(b"D"+str(j)) oname = c_char_p(str(j)) pname = c_char_p(b"IN") sname = c_char_p(str(j)) if i < nmaterials-1: rtnstat = imii.IMPRSreceiveUOPSflowweight(uname,oname,pname,sname,
byref(c_double(1.)),byref(c_double(0.)),byref(c_double(0.)),byref(c_double(0.)),IMPRSkeep) else: rtnstat = imii.IMPRSreceiveUOPSflowweight(uname,oname,pname,sname,
byref(c_double(-1.)),byref(c_double(0.)),byref(c_double(0.)),byref(c_double(0.)),IMPRSkeep)
PL's Min/Max Run-Lengths
Prices & Costs (Profit-Weights)
“Cleanout” Task's Setup/Downtime of 1-day
MILP Example - SeqDepSwo ● Python 2.7 w/ ctypes w/ IMPRESS-IMI:
# Receive the command data.
lower = byref(c_double(0.))upper = byref(c_double(1.))begin = byref(c_double(0.))end = byref(dthf)for i in range(nmaterials): if i < nmaterials-1: j = i+1 else: j = -1 uname = c_char_p(b"S"+str(j)) oname = c_char_p(str(j)) rtnstat = imii.IMPRSreceiveUOsetuporder(uname,oname,lower,upper,begin,end,IMPRSkeep) uname = c_char_p(b"PL") oname = c_char_p(str(j)) rtnstat = imii.IMPRSreceiveUOsetuporder(uname,oname,lower,upper,begin,end,IMPRSkeep) uname = c_char_p(b"D"+str(j)) oname = c_char_p(str(j)) rtnstat = imii.IMPRSreceiveUOsetuporder(uname,oname,lower,upper,begin,end,IMPRSkeep)
Setup-Orders “Free” or “Fix” a Unit-Operation
MILP Example - SeqDepSwo ● Python 2.7 w/ ctypes w/ IMPRESS-IMI:
# Receive the command data.
for i in range(nmaterials): if i < nmaterials-1: j = i+1 else: j = -1 uname = c_char_p(b"S"+str(j)) oname = c_char_p(str(j)) pname = c_char_p(b"OUT") sname = c_char_p(str(j)) uname2 = c_char_p(b"PL") oname2 = c_char_p(str(j)) pname2 = c_char_p(b"IN") sname2 = c_char_p(str(j)) rtnstat = imii.IMPRSreceiveUOPSPSUOsetuporder(uname,oname,pname,sname,pname2,sname2,uname2,oname2,
lower,upper,begin,end,IMPRSkeep) uname = c_char_p(b"PL") oname = c_char_p(str(j)) pname = c_char_p(b"OUT") sname = c_char_p(str(j)) uname2 = c_char_p(b"D"+str(j)) oname2 = c_char_p(str(j)) pname2 = c_char_p(b"IN") sname2 = c_char_p(str(j)) rtnstat = imii.IMPRSreceiveUOPSPSUOsetuporder(uname,oname,pname,sname,pname2,sname2,uname2,oname2,
lower,upper,begin,end,IMPRSkeep)
MILP Example - SeqDepSwo ● Python 2.7 w/ ctypes w/ IMPRESS-IMI:
# Receive the command data.
lower = byref(c_double(0.))upper = byref(c_double(72.))target = IMPRSrnnonbegin = byref(c_double(0.))end = byref(dthf)for i in range(nmaterials): if i < nmaterials-1: j = i+1 else: j = -1 uname = c_char_p(b"D"+str(j)) oname = c_char_p(str(j)) pname = c_char_p(b"IN") sname = c_char_p(str(j)) if i < nmaterials-1: rtnstat = imii.IMPRSreceiveUOPSholduporder(uname,oname,pname,sname,lower,upper,target,begin,end,IMPRSkeep) else: rtnstat = imii.IMPRSreceiveUOPSholduporder(uname,oname,pname,sname,lower,upper,target,begin,end,IMPRSkeep)
Demands w/ Release (begin)& Due-Dates (end)
MILP Example - SeqDepSwo ● Python 2.7 w/ ctypes w/ IMPRESS-IMI:
# Model the problem.
rtnstat = imim.IMPRSmodeler(problem,IMPRSsparsic,IMPRSdiscrete,IMPRSlogistics,IMPRSoptimization)
# Presolve and solve the problem.
rtnstat = imip.IMPRSpresolver(problem,IMPRSsparsic,IMPRSdiscrete,IMPRSlogistics,IMPRSoptimization,IMPRSsemisolverless,IMPRSxpress,IMPRSfirstsession,IMPRSflatfile,IMPRSfeedback)
# Retrieve the objective function terms.
profit = c_double()performance1 = c_double()performance2 = c_double()penalty = c_double()total = c_double()rtnstat = imii.IMPRSretrieveOBJterms(byref(profit),byref(performance1),byref(performance2),byref(penalty),byref(total))
print(profit.value)print(performance1.value)print(performance2.value)print(penalty.value)print(total.value)
...
# Release the problem memory.
rtnstat = imis.IMPRSrelease(IMPRSall)
IMPRSfeedback points to a Python coded “callback” function
to display solver progress
Model = Variables, Constraints & Derivatives
(& Expressions)
MILP Example - SeqDepSwo ● XPRESS-Mosel 7.3:
Coding New Variables & Constraints ● Although IMPRESS is not an AML, coding or
creating your own variables, constraints, derivatives & expressions as well as sets, lists, parameters & formulas is possible.
● Coding/creating a variable:– vregister(number,name,dimension,type)– vreceive(index,value)– vrestrain(index,lower,upper,weight)
● Coding/creating a constraint (f(x)+...+A*x+b ~ 0):– cregister(number,name,dimension,type)– creceive(index,value)– dratio(cindex,vindices,derivatives optional)– erelate(cindex,ntokens,instructions,values)
Defines Sparsity-Pattern
Objective Function Weights
Type = Continuous, Binary, etc.
Defines Expression
Challenges ● Understanding how to configure IOP's to improve
business performance (economics, efficiency, etc.).● Understanding how to configure IOP's for
tracatability (good solutions in reasonable-time).● Troubleshooting IOP's for inconsistent & incorrect
results (not infeasible but not expected either).● Incrementing & iterating the IOP from solution to
solution (manually or automatically) to improve its solution accuracy & reality.
● Improving the formulation of the IOP's model to help in the above (tighter, smaller, faster, smarter, ...)
● Thank You!