A Brief Introduction to Mathematical Optimization in Julia ...

Operated by Los Alamos National Security, LLC for the U.S. Department of Energy's NNSA

A Brief Introduction to Mathematical Optimization in Julia (Part 2)

Carleton Coffrin, et. al.Advanced Network Science Initiative

https://lanl-ansi.github.io/

Critical Infrastructure Modeling in Julia

LA-UR-21-20300

https://lanl-ansi.github.io/

Warning: This is not a technical talk!

National Infrastructure Networks

Power System Natural Gas

Communications (Fiber)

• Goals • Optimization of Network

Design and Operations • Understand

Interdependencies

Common Threads

Graph Structure

Sij = Y ⇤ijViV

⇤i � Y ⇤

ijViV⇤j (i, j) 2 E [ ER

Sgi � Sd

i =X

(i,j)2E[ER

Sij 8i 2 N

Physics Equations

I.A. Hiskens and R.J. Davy, "Exploring the power flow solution space boundary” IEEE Transactions on Power Systems, Vol. 16, No. 3, August 2001, pp. 389-395

Challenges

• Explosion of Power Flow physics formulations in recent years

Taxonomy of Power Flow Formulations (2014)

Also see: D.K. Molzahn and I.A. Hiskens, “A Survey of Relaxations and Approximations of the Power Flow Equations,” to appear in Foundation and Trends in Electric Power Systems, 2018.

More Challenges

• Extremely Wide Range of Optimization Problem Classes • NLP: nonlinear physics in operations • Mixed-Integer: network design decisions • QP / SOC / SDP: convex relaxations • LP: linear approximations

• Every problem class has a different state-of-the-art solver… • solver scalability varies widely

Even More Challenges

• One formulation does not fit all problems • approximation vs relaxation • may need to explore many variants

• Many technical implementation details • e.g. QC (14 voltage constraints for each branch) • best done by original authors

• Comparing Formulations • same equations? (bus shunts, line charging, HVDC lines) • same constraints? (thermal limits, angle limits) • same objective function? (linear, quadratic, PWL) • runtime evaluation (matlab, python, AMPL)

Our Inspiration (in 2016)

?

Foundational Language Infrastructure Modeling

Core Design Ideas

• Develop a Infrastructure Network Optimization modeling layers • all formulations share a common mathematical program specification • switching formulations (and problem class) is effortless

• Open-source and community driven • formulation authors contribute their implementation • most well suited to provide the best version of their formulation idea

• Common implementation platform • shared problem specification ensure same equations and constraints • side-by-side comparisons (e.g. benchmarking) are much more accurate

InfrastructureModels.jl

Open-source Julia packages for Infrastructure Network Optimization

Why Julia?

• Benefits of the Julia Language • Recall part 1 of this talk

• We didn’t have a choice • See the LANL postdoc revolt of 2014

• JuMP • fantastic optimization modeling layer • LP, SOCP, SDP, NLP (and Mixed-Integer variants)

9/25/2019 Thin-Border-Logo-Text

https://camo.githubusercontent.com/31d60f762b44d0c3ea47cc16b785e042104a6e03/68747470733a2f2f7777772e6a756c69616f70742e6f72672f696d616765732f6a7… 1/1

Mathematical Optimization

Infrastructure Models Packages

• Open-Source Infrastructure Optimization Software Foundation

Russell Bent @rb004f

Kaarthik Sundar @kaarthiksundar

Carleton Coffrin @ccoffrin

David Fobes @pseudocubic

Byron Tasseff @tasseff

External Contributors @frederikgeth,

@hakanergun, @jd-lara, @kersulis, @mlubin,

@rofinn, @yeesian, …

Theme of InfrastructureModels Packages

• Infrastructure-aware JuMP-like modeling layer

• Minimalist Matlab data formats for R&D

• Parsers for Established Data Files • e.g. PSSE v33 / Matpower, GasLib / MatGas, OpenDSS, EPANET

• Cross Infrastructure Modeling Standards • Lists of components, organized by component type • Universal fields

• “index” - unique integer id for component • “status” - makes components inactive • “name” - human readable name

• Goal: Usage of all InfrastructureModels packages “feels” similar • Easy to integrate and study multi-infrastructure problems

Example - Simple OPF Study

# Installation] add PowerModels, Ipopt

using PowerModels

network_data = parse_file(“pglib_opf_case5_pjm.m”)print_summary(network_data)println(network_data[“bus”][“1”][“va”])

using Ipopt

ac_result = run_ac_opf(network_data, Ipopt.Optimizer)print_summary(ac_result[“solution”])

dc_result = run_dc_opf(network_data, Ipopt.Optimizer)print_summary(dc_result[“solution”])

Example - Reading Data

using PowerModels

network_data = parse_file(“pglib_opf_case5_pjm.m”)

function mpc = pglib_opf_case5_pjmmpc.version = '2';mpc.baseMVA = 100.0;

mpc.bus = [ 1 2 0.0 0.00 0.0 0.0 1 1.00000 0.00000 230.0 1 1.10000 0.90000; 2 1 300.0 98.61 0.0 0.0 1 1.00000 0.00000 230.0 1 1.10000 0.90000; 3 2 300.0 98.61 0.0 0.0 1 1.00000 0.00000 230.0 1 1.10000 0.90000; 4 3 400.0 131.47 0.0 0.0 1 1.00000 0.00000 230.0 1 1.10000 0.90000; 5 2 0.0 0.00 0.0 0.0 1 1.00000 0.00000 230.0 1 1.10000 0.90000;];

mpc.branch = [ 1 2 0.00281 0.0281 0.00712 400 400 400 0.0 0.0 1 -30.0 30.0; 1 4 0.00304 0.0304 0.00658 426 426 426 0.0 0.0 1 -30.0 30.0; 1 5 0.00064 0.0064 0.03126 426 426 426 0.0 0.0 1 -30.0 30.0; 2 3 0.00108 0.0108 0.01852 426 426 426 0.0 0.0 1 -30.0 30.0; 3 4 0.00297 0.0297 0.00674 426 426 426 0.0 0.0 1 -30.0 30.0; 4 5 0.00297 0.0297 0.00674 240 240 240 0.0 0.0 1 -30.0 30.0;];

...

Dict{String,Any} with 13 entries: "source_type" => "matpower" "name" => "pglib_opf_case5_pjm" "source_version" => v"2.0.0" "baseMVA" => 100.0 "per_unit" => true "bus" => Dict{String,Any}(...) "branch" => Dict{String,Any}(...) "dcline" => Dict{String,Any}(...) "gen" => Dict{String,Any}(...) "load" => Dict{String,Any}(...) "shunt" => Dict{String,Any}(...) "storage" => Dict{String,Any}(...)

network_data["bus"]["1"]["vm"] ... 1.0network_data["branch"]["3"]["br_x"] ... 0.0064

PowerModels Mathematical Modeling Layer

function build_opf(pm::GenericPowerModel) variable_bus_voltage(pm) variable_gen_power(pm) variable_branch_power(pm)

objective_min_fuel_cost(pm)

constraint_model_voltage(pm)

for i in ids(pm, :ref_buses) constraint_theta_ref(pm, i) end

for i in ids(pm, :bus) constraint_power_balance(pm, i) end

for i in ids(pm, :branch) constraint_ohms_yt_from(pm, i) constraint_ohms_yt_to(pm, i)

constraint_voltage_angle_difference(pm, i)

constraint_thermal_limit_from(pm, i) constraint_thermal_limit_to(pm, i) endend

Infrastructure Aware JuMP-like Modeling

Variable Declaration

Objective Function

ConstraintsComponent-wise Constraints

Real-World Applications

Severe Contingency Solver

• PowerModelsRestoration - Maximal Load Delivery • Highly reliable solver for N-k (k in the range of 10-1000) • Designed for extreme event analysis (e.g. natural disasters)

• Solves networks with 3,000 to 10,000 buses in seconds to minutes • Validated on over 10,000 of contingency scenarios

40 50 60 70 80 90 100

0.00

0.05

0.10

0.15

0.20

0.25

0.30

40 50 60 70 80 90 100

0.00

0.05

0.10

0.15

0.20

0.25

0.30

40 50 60 70 80 90 100

0.00

0.05

0.10

0.15

0.20

0.25

0.30

40 50 60 70 80 90 100

0.00

0.05

0.10

0.15

0.20

0.25

0.30

40 50 60 70 80 90 100

0.00

0.05

0.10

0.15

0.20

0.25

0.30

40 50 60 70 80 90 100

0.00

0.05

0.10

0.15

0.20

0.25

0.30

Distributions of Active Power Delivery after Severe Contingencies

Active Power Delivered (% of total)

Freq

uenc

y (n

orm

alize

d, n

=100

0)

40 50 60 70 80 90 100

0.00

0.05

0.10

0.15

0.20

0.25

0.30

IEEE RTS 96PSERC 240PEGASE 1354RTE 1888Polish 2383wpPolish 3120spRTE 6468• 2019 •

FINALIST

• Solving Real-World Power Network Optimization • https://gocompetition.energy.gov/ • 1,000,000,000 - 5,000,000,000 decision variables • Comparable number of constraints • Solved in 5 minutes or less • Well beyond available commercial tools

• PowerModelsSecurityConstrained.jl • Built on top of PowerModels.jl (open sourced) • Challenge 1, finished 10th • Challenge 2, underway

• Finished 2nd in Trial 1

Security Constraint Optimal Power Flow

https://gocompetition.energy.gov/

Other Notable Julia Developments

Infrastructure Modeling in Julia

• NREL’s SIIP Project • SIIP = Scalable Integrated Infrastructure Planning • Data wrangling, investment planning, production cost modeling, dynamics modeling • https://www.nrel.gov/analysis/siip.html

• NREL’s REopt Lite API • Optimal investment and operation of renewable energy, conventional generation,

and energy storage technologies • https://github.com/NREL/REopt_Lite_API

• Other Efforts • https://github.com/JuliaEnergy • https://github.com/PyPSA/PSA.jl • https://github.com/PyPSA/EnergyModels.jl • https://github.com/Energy-MAC

https://www.nrel.gov/analysis/siip.html

https://github.com/NREL/REopt_Lite_API

https://github.com/JuliaEnergy

https://github.com/PyPSA/PSA.jl

https://github.com/PyPSA/EnergyModels.jl

https://github.com/Energy-MAC

Want to Learn More?

• LANL Grid Science Tutorials (InfrastructureModels.jl) • https://github.com/lanl-ansi/tutorial-grid-science

• NREL SIIP Tutorials • https://github.com/NREL-SIIP/SIIPExamples.jl

• PowerModelsAnnex • OPF “from scratch” using JuMP • https://github.com/lanl-ansi/PowerModelsAnnex.jl/tree/master/src/model

https://github.com/lanl-ansi/tutorial-grid-science

https://github.com/NREL-SIIP/SIIPExamples.jl

https://github.com/lanl-ansi/PowerModelsAnnex.jl/tree/master/src/model

Summary

• Infrastructure modeling and optimization is complicated!

• We are developing the InfrastructureModels.jl packages to… • Help researchers to explore and contribute mathematical programs • Rigorously benchmark formulations with a shared problem specification and

computational platform • Provide a unified software architecture for infrastructure modeling and

optimization • Model infrastructure interdependencies through common mathematical

model

• Special thanks to the LANL team and 10+ contributors to the codebase! • Open source is great!

Thanks!

What do you want to know about infrastructure modeling in Julia?

PowerModels Validation

Validation Study

• Validation Study • Compare AC Polar and DC Power Flow • https://arxiv.org/abs/1711.01728

• IEEE PES PGLib OPF cases • Focus on those with over 1000 buses

using PowerModels; using Ipopt ipopt = IpoptSolver(tol=1e-6) case = PowerModels.parse_file(“pglib_opf_case24_ieee_rts.m”)

result_acp = run_opf(case, ACPPowerModel, ipopt)result_dcp = run_opf(case, DCPPowerModel, ipopt)

Base lines

Power Grid Libraryhttps://github.com/power-grid-lib

https://arxiv.org/abs/1711.01728

Validation Study

TABLE IQUALITY AND RUNTIME COMPARISON OF MATPOWER AND POWERMODELS ON AC AND DC OPTIMAL POWER FLOW

$/h Runtime (seconds)MP PM MP PM MP PM MP PM

Test Case |N | |E| AC-P AC-P DC DC AC-P AC-P DC DCTypical Operating Conditions (TYP)

case1354 pegase 1354 1991 1.3640e+06 1.3640e+06 1.3141e+06 1.3140e+06 4 6 2 <1case1888 rte 1888 2531 n.s. 1.5654e+06 1.5111e+06 1.5111e+06 6 14 2 <1case1951 rte 1951 2596 n.s. 2.3753e+06 2.3128e+06 2.3128e+06 3 18 2 <1

case2383wp k 2383 2896 1.8685e+06 1.8685e+06 1.7968e+06 1.8041e+06 5 9 2 <1case2736sp k 2736 3504 1.3079e+06 1.3079e+06 1.2760e+06 1.2760e+06 4 8 2 <1

case2737sop k 2737 3506 7.7763e+05 7.7763e+05 7.6401e+05 7.6401e+05 5 6 2 <1case2746wop k 2746 3514 1.2083e+06 1.2083e+06 1.1782e+06 1.1782e+06 5 7 2 <1case2746wp k 2746 3514 1.6318e+06 1.6318e+06 1.5814e+06 1.5814e+06 5 7 3 <1

case2848 rte 2848 3776 n.s. 1.3847e+06 1.3636e+06 1.3636e+06 8 19 2 <1case2868 rte 2868 3808 n.s. 2.2599e+06 2.2053e+06 2.2053e+06 12 21 3 <1

case2869 pegase 2869 4582 2.6050e+06 2.6050e+06 2.5167e+06 2.5166e+06 8 14 3 <1case3012wp k 3012 3572 2.6008e+06 2.6008e+06 2.5143e+06 2.5090e+06 7 11 3 <1case3120sp k 3120 3693 2.1457e+06 2.1457e+06 2.0891e+06 2.0880e+06 7 11 3 <1

case3375wp k 3375 4161 n.s. 7.4357e+06 7.3166e+06 7.3170e+06 2 14 3 <1case6468 rte 6468 9000 n.s. 2.2623e+06 2.1797e+06 2.1619e+06 7 80 4 <1case6470 rte 6470 9005 n.s. 2.5558e+06 2.4520e+06 2.4454e+06 32 47 4 2case6495 rte 6495 9019 n.s. 3.4777e+06 3.0085e+06 2.8481e+06 16 89 4 2case6515 rte 6515 9037 n.s. 3.1971e+06 2.9493e+06 2.8484e+06 30 73 4 2

case9241 pegase 9241 16049 n.s. 6.7747e+06 6.5411e+06 6.5179e+06 68 62 7 2case13659 pegase 13659 20467 1.0781e+07 1.0781e+07 1.0587e+07 1.0565e+07 51 96 8 3

Congested Operating Conditions (API)case1354 pegase api 1354 1991 – 1.8041e+06 1.7479e+06 1.7485e+06 – 6 2 <1

case1888 rte api 1888 2531 n.s. 2.2566e+06 2.1930e+06 2.1930e+06 9 10 2 <1case1951 rte api 1951 2596 – 2.8005e+06 n.s. 2.7027e+06 – 124 2 <1

case2383wp k api 2383 2896 2.7913e+05 2.7913e+05 2.7913e+05 2.7913e+05 3 31 2 <1case2736sp k api 2736 3504 6.3847e+05 6.3847e+05 6.1252e+05 5.9774e+05 5 9 3 <1

case2737sop k api 2737 3506 4.0282e+05 4.0282e+05 3.7767e+05 3.7570e+05 5 8 3 <1case2746wop k api 2746 3514 5.1166e+05 5.1166e+05 5.1166e+05 5.1166e+05 3 3 3 <1case2746wp k api 2746 3514 5.8183e+05 5.8183e+05 5.8183e+05 5.8183e+05 4 5 3 <1

case2848 rte api 2848 3776 – 1.7169e+06 1.6822e+06 1.6822e+06 – 35 3 <1case2868 rte api 2868 3808 – 2.7159e+06 n.s. 2.6357e+06 – 29 3 <1

case2869 pegase api 2869 4582 – 3.3185e+06 3.2137e+06 3.2154e+06 – 16 3 <1case3012wp k api 3012 3572 7.2887e+05 7.2887e+05 7.2887e+05 7.2887e+05 7 5 3 <1case3120sp k api 3120 3693 9.2026e+05 9.2026e+05 8.9103e+05 8.5997e+05 7 15 3 <1

case3375wp k api 3375 4161 n.s. 5.8861e+06 5.8117e+06 5.7641e+06 2 14 3 <1case6468 rte api 6468 9000 n.s. 2.7102e+06 2.6078e+06 2.6081e+06 45 84 4 <1case6470 rte api 6470 9005 n.s. 3.1603e+06 3.0266e+06 3.0333e+06 18 58 4 <1case6495 rte api 6495 9019 n.s. 3.6263e+06 3.4265e+06 3.4186e+06 7 77 4 2case6515 rte api 6515 9037 n.s. 3.5904e+06 3.3662e+06 3.3777e+06 42 72 4 2

case9241 pegase api 9241 16049 – 8.2656e+06 7.9822e+06 7.9822e+06 – 73 13 2case13659 pegase api 13659 20467 – 1.1209e+07 1.0903e+07 1.0895e+07 – 79 11 3

Small Angle Difference Conditions (SAD)case1354 pegase sad 1354 1991 1.3646e+06 1.3646e+06 n.s. inf. 4 6 2 <1

case1888 rte sad 1888 2531 n.s. 1.5806e+06 1.5122e+06 1.5123e+06 7 16 2 <1case1951 rte sad 1951 2596 n.s. 2.3820e+06 n.s. inf. 3 25 2 <1

case2383wp k sad 2383 2896 1.9165e+06 1.9165e+06 n.s. inf. 5 10 2 2case2736sp k sad 2736 3504 1.3294e+06 1.3294e+06 n.s. inf. 5 10 2 2

case2737sop k sad 2737 3506 7.9267e+05 7.9267e+05 n.s. inf. 5 9 2 2case2746wop k sad 2746 3514 1.2344e+06 1.2344e+06 n.s. inf. 5 8 2 2case2746wp k sad 2746 3514 1.6674e+06 1.6674e+06 n.s. inf. 5 9 3 19

case2848 rte sad 2848 3776 n.s. 1.3879e+06 n.s. inf. 6 21 3 2case2868 rte sad 2868 3808 n.s. 2.2707e+06 n.s. inf. 7 20 3 2

case2869 pegase sad 2869 4582 2.6198e+06 2.6198e+06 n.s. inf. 9 14 3 4case3012wp k sad 3012 3572 2.6213e+06 2.6213e+06 n.s. inf. 7 12 3 3case3120sp k sad 3120 3693 2.1755e+06 2.1755e+06 n.s. inf. 7 14 3 3

case3375wp k sad 3375 4161 n.s. 7.4357e+06 7.3181e+06 7.3197e+06 2 14 3 <1case6468 rte sad 6468 9000 n.s. 2.2623e+06 2.1797e+06 2.1619e+06 7 131 4 <1case6470 rte sad 6470 9005 n.s. 2.5597e+06 2.4595e+06 2.4483e+06 22 48 4 2case6495 rte sad 6495 9019 n.s. 3.4777e+06 3.0995e+06 2.8482e+06 12 88 4 2case6515 rte sad 6515 9037 n.s. 3.2679e+06 3.1394e+06 2.8486e+06 11 78 4 2

case9241 pegase sad 9241 16049 n.s. 6.9170e+06 n.s. inf. 38 70 5 10case13659 pegase sad 13659 20467 1.0901e+07 1.0901e+07 n.s. inf. 51 79 6 12

MP = MatPowerPM = PowerModels

AC-P = AC Polar FormDC-P = DC Power Flow

Validation Study

• AC Solutions are identical, when Matpower converges • Matpower’s default solver can be faster, but less reliable • Some minor differences in DC Power Flow implementationTABLE I

QUALITY AND RUNTIME COMPARISON OF MATPOWER AND POWERMODELS ON AC AND DC OPTIMAL POWER FLOW














































































Date post:	20-Oct-2021
Category:	Documents
Upload:	others
View:	5 times
Download:	0 times

A Brief Introduction to Mathematical Optimization in Julia ...

Documents