+ All Categories
Home > Documents > Large-scale stochastic PDE-constrained optimization · Outline 1 Examples of stochastic PDE...

Large-scale stochastic PDE-constrained optimization · Outline 1 Examples of stochastic PDE...

Date post: 27-Jun-2020
Category:
Upload: others
View: 5 times
Download: 0 times
Share this document with a friend
54
Large-scale stochastic PDE-constrained optimization Omar Ghattas and Peng Chen Oden Institute for Computational Engineering & Sciences The University of Texas at Austin Collaborators on previous work: Alen Alexanderian (NCSU), Noemi Petra (UC Merced), Georg Stadler (NYU), Umberto Villa (Washington Univ) 18 July 2019 International Congress on Industrial and Applied Mathematics (ICIAM 2019) Valencia, Spain Supported by AFOSR, DARPA, DOE, NSF Ghattas (Oden Inst/UT Austin) Stochastic PDE-constrained optimization ICIAM 2019 1 / 47
Transcript
  • Large-scale stochastic PDE-constrained optimization

    Omar Ghattas and Peng Chen

    Oden Institute for Computational Engineering & SciencesThe University of Texas at Austin

    Collaborators on previous work: Alen Alexanderian (NCSU),Noemi Petra (UC Merced), Georg Stadler (NYU), Umberto Villa (Washington Univ)

    18 July 2019International Congress on Industrial and Applied Mathematics (ICIAM 2019)

    Valencia, Spain

    Supported by AFOSR, DARPA, DOE, NSF

    Ghattas (Oden Inst/UT Austin) Stochastic PDE-constrained optimization ICIAM 2019 1 / 47

  • Introduction

    Classes of PDE-constrained optimization under uncertainty:

    Inverse problem: Given observations, a random/stochastic forward PDEmodel, and prior information, find model parameters that minimize datamisfit

    Optimal experimental design problem: Design data acquisition system thatmaximizes information gain in Bayesian inverse problem

    Optimal design problem: Find the configuration of a system described by arandom/stochastic PDE model that maximizes desired performance (subjectto constraints)

    Optimal control problem: Find the operation of a system described by arandom/stochastic PDE model that maximizes desired performance (subjectto constraints)

    Fundamental difficulty:

    Both optimization variable and random parameter are often (inf-dim) fields

    OUU amounts to numerous forward uncertainty propagation problems

    Forward UQ amounts to numerous deterministic forward problems

    Ghattas (Oden Inst/UT Austin) Stochastic PDE-constrained optimization ICIAM 2019 2 / 47

  • Outline

    1 Examples of stochastic PDE constrained optimization problems

    2 Mean-variance PDE-constrained optimization via Taylor approximation

    3 Taylor approximation as a control variate

    4 Optimal design of acoustic metamaterial cloak under uncertainty

    5 Chance constraints with PDE models

    Ghattas (Oden Inst/UT Austin) Stochastic PDE-constrained optimization ICIAM 2019 3 / 47

  • Ice core locations to optimally infer Antarctic basal frictionw/ T. Isaac (Georgia Tech), N. Petra (UC Merced), G. Stadler (NYU), H. Zhu (UTRC)

    Ghattas (Oden Inst/UT Austin) Stochastic PDE-constrained optimization ICIAM 2019 4 / 47

  • Seismometer locations to optimally infer Earth structurew/ T. Bui-Thanh (UT Austin), C. Burstedde (Bonn), G. Stadler (NYU), L. Wilcox (NPS)

    Ghattas (Oden Inst/UT Austin) Stochastic PDE-constrained optimization ICIAM 2019 5 / 47

  • Optimal management of groundwater resourcesw/ A. Alghamdi, M. Hesse, A. Chen, P. Chen, G. Stadler (NYU)

    Ghattas (Oden Inst/UT Austin) Stochastic PDE-constrained optimization ICIAM 2019 6 / 47

  • Optimal stellerator design with uncertain coil geometryw/ G. Stadler (NYU) and Simons Collaboration (PI: A. Bhattacharjee, Princeton)

    Ghattas (Oden Inst/UT Austin) Stochastic PDE-constrained optimization ICIAM 2019 7 / 47

  • Optimal directed self-assembly of block copolymersw/ AEOLUS center: T. Oden, F. Alexander (BNL), K. Willcox, P. Chen, et. al

    Ghattas (Oden Inst/UT Austin) Stochastic PDE-constrained optimization ICIAM 2019 8 / 47

  • Outline

    1 Examples of stochastic PDE constrained optimization problems

    2 Mean-variance PDE-constrained optimization via Taylor approximation

    3 Taylor approximation as a control variate

    4 Optimal design of acoustic metamaterial cloak under uncertainty

    5 Chance constraints with PDE models

    Ghattas (Oden Inst/UT Austin) Stochastic PDE-constrained optimization ICIAM 2019 9 / 47

  • Mean-variance PDE-constrained optimal control

    Weak form of forward PDE model with random and control variables:

    find u ∈ U such that r(u, v,m, z) = 0 ∀v ∈ V

    where u ∈ U is state, v ∈ V adjoint, m ∈M random field, z ∈ Z controlObjective function: Consider mean–variance of control functionalQ(·, ·) : U×M→ R:

    J(z) = E[Q] + βVar[Q] + P(z)

    where P(z) is cost of controls (or regularization)

    Optimal control problem: find z∗ ∈ Z, s.t.

    z∗ = arg minz∈Z J(z), subject to r(u, v,m, z) = 0

    Sample average approximation (SAA) is prohibitive: entails as many(nonlinear) PDE constraints as required for accurate estimation of E[Q]

    z∗ = arg minz∈Z JMC(z), subject to r(u, v,mi, z) = 0 i = 1, . . . ,M

    =⇒ “Many-PDE-constrained optimization”Ghattas (Oden Inst/UT Austin) Stochastic PDE-constrained optimization ICIAM 2019 10 / 47

  • Mean-variance PDE-constrained optimal control

    Weak form of forward PDE model with random and control variables:

    find u ∈ U such that r(u, v,m, z) = 0 ∀v ∈ V

    where u ∈ U is state, v ∈ V adjoint, m ∈M random field, z ∈ Z controlObjective function: Consider mean–variance of control functionalQ(·, ·) : U×M→ R:

    J(z) = E[Q] + βVar[Q] + P(z)

    where P(z) is cost of controls (or regularization)

    Optimal control problem: find z∗ ∈ Z, s.t.

    z∗ = arg minz∈Z J(z), subject to r(u, v,m, z) = 0

    Sample average approximation (SAA) is prohibitive: entails as many(nonlinear) PDE constraints as required for accurate estimation of E[Q]

    z∗ = arg minz∈Z JMC(z), subject to r(u, v,mi, z) = 0 i = 1, . . . ,M

    =⇒ “Many-PDE-constrained optimization”Ghattas (Oden Inst/UT Austin) Stochastic PDE-constrained optimization ICIAM 2019 10 / 47

  • Some existing approaches for PDE-constrained OUUSchulz & Schillings, Problem formulations and treatment of uncertainties in aerodynamic design, AIAA J, 2009.Borz̀ı & von Winckel, Multigrid methods and sparse-grid collocation techniques for parabolic optimal control problemswith random coefficients, SISC, 2009.Borz̀ı, Schillings, & von Winckel, On the treatment of distributed uncertainties in PDE-constrained optimization,GAMM-Mitt. 2010.Borz̀ı & von Winckel, A POD framework to determine robust controls in PDE optimization, Computing andVisualization in Science, 2011.Gunzburger & Ming, Optimal control of stochastic flow over a backward-facing step using ROM, SISC 2011.Hou, Lee, & Manouzi, Finite element approximations of stochastic optimal control problems constrained by stochasticelliptic PDEs, J Math Anal Appl, 2011.Gunzburger, Lee, & Lee, Error estimates of stochastic optimal Neumann boundary control problems, SINUM, 2011.Rosseel & Wells, Optimal control with stochastic PDE constraints and uncertain controls, CMAME, 2012.Tiesler, Kirby, Xiu, & Preusser, Stochastic collocation for optimal control problems with stochastic PDE constraints,SICON, 2012.Kouri, Heinkenschloss, Ridzal, & Van Bloemen Waanders, A trust-region algorithm with adaptive stochastic collocationfor PDE optimization under uncertainty, SISC, 2013.Chen, Quarteroni, & Rozza, Stochastic optimal Robin boundary control problems of advection-dominated ellipticequations, SINUM, 2013.Kunoth & Schwab, Analytic regularity and gPC approximation for control problems constrained by linear parametricelliptic and parabolic PDEs, SICON, 2013.Kouri, A multilevel stochastic collocation algorithm for optimization of PDEs with uncertain coefficients, JUQ, 2014.Chen & Quarteroni, Weighted reduced basis method for stochastic optimal control problems with elliptic PDEconstraint, JUQ, 2014.Ng & Willcox, Multifidelity approaches for optimization under uncertainty, IJNME, 2014.Kouri, Heinkenschloss, Ridzal, & van Bloemen Waanders, Inexact objective function evaluations in a trust-regionalgorithm for PDE-constrained optimization under uncertainty, SISC, 2014.Chen, Quarteroni, & Rozza, Multilevel and weighted reduced basis method for stochastic optimal control problemsconstrained by Stokes equations, Num. Math. 2015.Ng & Willcox, Monte Carlo information-reuse approach to aircraft conceptual design optimization under uncertainty, JAircraft, 2015.P. Benner, A. Onwunta, and M. Stoll. Block-diagonal preconditioning for optimal control problems constrained by PDEswith uncertain inputs. SIMAX, 2016.A.A. Ali, E. Ullmann, & M. Hinze, Multilevel Monte Carlo analysis for optimal control of elliptic PDEs with randomcoefficients, SIAM/ASA JUQ, 2017.A. Alexanderian, N. Petra, G. Stadler, & O. Ghattas. Mean-variance risk-averse optimal control of systems governed byPDEs with random parameter fields using quadratic approximations. SIAM/ASA JUQ, 2017.

    Ghattas (Oden Inst/UT Austin) Stochastic PDE-constrained optimization ICIAM 2019 11 / 47

  • Quadratic approximation in infinite dimensions

    We approximate Q by a quadratically-truncated Taylor expansion w.r.t. m:

    Q(m) ≈ Qquad(m) = Q(m̄) + 〈gm(m̄),m− m̄〉

    +1

    2〈Hm(m̄)(m− m̄),m− m̄〉

    For a Gaussian random field m with m ∼ N(m̄,C), Qquad is non-Gaussian,but we can still express1

    E[Qquad] = Q(m̄) +1

    2tr(H̃)

    Var[Qquad] = 〈gm(m̄),Cgm(m̄)〉+1

    2tr(H̃2)

    where H̃ = C1/2Hm(m̄)C1/2 is the covariance-preconditioned Hessian

    Need to efficiently evaluate tr(H̃) and tr(H̃2) and their gradients w.r.t. z

    Qquad is corrected by using it as a control variate (cf. multifidelity methods2)

    1A. Alexanderian, N. Petra, G. Stadler, and O. Ghattas, Mean-variance risk-averse optimal control of systems governed by

    PDEs with random parameter fields using quadratic approximations, SIAM/ASA JUQ, 2017.2

    L. Ng & K. Willcox, Multifidelity approaches for optimization under uncertainty, IJNME, 2014.

    Ghattas (Oden Inst/UT Austin) Stochastic PDE-constrained optimization ICIAM 2019 12 / 47

  • Quadratic approximation in infinite dimensions

    We approximate Q by a quadratically-truncated Taylor expansion w.r.t. m:

    Q(m) ≈ Qquad(m) = Q(m̄) + 〈gm(m̄),m− m̄〉

    +1

    2〈Hm(m̄)(m− m̄),m− m̄〉

    For a Gaussian random field m with m ∼ N(m̄,C), Qquad is non-Gaussian,but we can still express1

    E[Qquad] = Q(m̄) +1

    2tr(H̃)

    Var[Qquad] = 〈gm(m̄),Cgm(m̄)〉+1

    2tr(H̃2)

    where H̃ = C1/2Hm(m̄)C1/2 is the covariance-preconditioned Hessian

    Need to efficiently evaluate tr(H̃) and tr(H̃2) and their gradients w.r.t. z

    Qquad is corrected by using it as a control variate (cf. multifidelity methods2)

    1A. Alexanderian, N. Petra, G. Stadler, and O. Ghattas, Mean-variance risk-averse optimal control of systems governed by

    PDEs with random parameter fields using quadratic approximations, SIAM/ASA JUQ, 2017.2

    L. Ng & K. Willcox, Multifidelity approaches for optimization under uncertainty, IJNME, 2014.

    Ghattas (Oden Inst/UT Austin) Stochastic PDE-constrained optimization ICIAM 2019 12 / 47

  • Quadratic approximation in infinite dimensions

    We approximate Q by a quadratically-truncated Taylor expansion w.r.t. m:

    Q(m) ≈ Qquad(m) = Q(m̄) + 〈gm(m̄),m− m̄〉

    +1

    2〈Hm(m̄)(m− m̄),m− m̄〉

    For a Gaussian random field m with m ∼ N(m̄,C), Qquad is non-Gaussian,but we can still express1

    E[Qquad] = Q(m̄) +1

    2tr(H̃)

    Var[Qquad] = 〈gm(m̄),Cgm(m̄)〉+1

    2tr(H̃2)

    where H̃ = C1/2Hm(m̄)C1/2 is the covariance-preconditioned Hessian

    Need to efficiently evaluate tr(H̃) and tr(H̃2) and their gradients w.r.t. z

    Qquad is corrected by using it as a control variate (cf. multifidelity methods2)

    1A. Alexanderian, N. Petra, G. Stadler, and O. Ghattas, Mean-variance risk-averse optimal control of systems governed by

    PDEs with random parameter fields using quadratic approximations, SIAM/ASA JUQ, 2017.2

    L. Ng & K. Willcox, Multifidelity approaches for optimization under uncertainty, IJNME, 2014.

    Ghattas (Oden Inst/UT Austin) Stochastic PDE-constrained optimization ICIAM 2019 12 / 47

  • How to compute tr(H̃) efficiently?

    When the eigenvalues decay rapidly (as is common for Hessians), the tracecan be approximated efficiently with small N by

    tr(H̃) ≈N∑j=1

    λj(H̃) and tr(H̃2) ≈

    N∑j=1

    λ2j (H̃)

    where λj , j = 1, . . . , N , are the dominant eigenvalues of H̃, or thedominant generalized eigenvalues of

    Hm(m̄)ψj = λjC−1ψj

    where ψj are the C−1-orthonormal eigenfunctions, i.e., 〈ψi,C−1ψj〉 = δij

    Prohibitive to compute Hm by itself; instead can form action in a givendirection at cost of pair of linearized forward/adjoint PDE solves

    =⇒ Need operator-free eigensolver that can capture dominant spectrum innumber of operator applications that scales with effective rank

    Ghattas (Oden Inst/UT Austin) Stochastic PDE-constrained optimization ICIAM 2019 13 / 47

  • Computing the trace of H̃ via randomized SVD

    Double-pass randomized SVD algorithmestimates trace at cost of 2r products ofH̃ with random vectors (r = N + p,N is rank of H̃, p is oversampling #)Resulting cost is 2r pairs of incrementalforward/adjoint solves w/same PDEoperatorCovariance operator and Hessian are oftencompact (Q is sensitive to limited numberof modes) so composition is compact

    Randomized SVD (double pass algorithm)

    1 Generate i.i.d. Gaussian matrix R ∈ Rn×rwith r = numerical rank of H̃ (r � n)

    2 Form Y = H̃R

    3 Compute Q = orthonormal basis for Y

    4 Define B ∈ Rr×r := QT H̃Q5 Decompose B = ZΛZT

    6 Low-rank approximation: H̃ ≈ V ΛV T ,where V ∈ Rn×r := QZ

    7 Trace estimation: tr(H̃) ≈ tr(B)

    Thus often r � n and independent of parameter dimension n; with high probability

    |tr(H̃)− tr(B)| ≤ c(p)∑r

  • Eigenproblem-constrained optimization

    With the trace computed via randomized SVD, we obtain

    Jquad(z) = Q(m̄) +1

    2

    N∑j=1

    λj(H̃)︸ ︷︷ ︸E[Q]

    +β〈gm(m̄),Cgm(m̄)〉+β

    2

    N∑j=1

    λ2j (H̃)︸ ︷︷ ︸βVar[Q]

    +P(z)

    where Q(m̄) := Q̄ is obtained by solving the forward problem for u ∈ U

    〈ṽ, ∂v r̄(u, ṽ, z)〉 = 0, ∀ṽ ∈ V

    with r̄(u, ṽ, z) = r(u, ṽ, m̄, z) for short. By defining the Lagrangian

    L(u, v, m̄, z) = Q(u) + r̄(u, v, z)

    the gradient gm(m̄) is found from

    〈m̃, gm(m̄)〉 = 〈m̃, ∂mL〉 = 〈m̃, ∂mr̄(u, v, z)〉, ∀m̃ ∈M

    for which we need to compute v ∈ V by solving the adjoint problem

    〈ũ, ∂ur̄(u, v, z)〉 = −〈ũ, ∂uQ̄〉, ∀ũ ∈ U

    Ghattas (Oden Inst/UT Austin) Stochastic PDE-constrained optimization ICIAM 2019 15 / 47

  • Eigenproblem-constrained optimization

    With the trace computed via randomized SVD, we obtain

    Jquad(z) = Q(m̄) +1

    2

    N∑j=1

    λj(H̃)︸ ︷︷ ︸E[Q]

    +β〈gm(m̄),Cgm(m̄)〉+β

    2

    N∑j=1

    λ2j (H̃)︸ ︷︷ ︸βVar[Q]

    +P(z)

    where Q(m̄) := Q̄ is obtained by solving the forward problem for u ∈ U

    〈ṽ, ∂v r̄(u, ṽ, z)〉 = 0, ∀ṽ ∈ V

    with r̄(u, ṽ, z) = r(u, ṽ, m̄, z) for short. By defining the Lagrangian

    L(u, v, m̄, z) = Q(u) + r̄(u, v, z)

    the gradient gm(m̄) is found from

    〈m̃, gm(m̄)〉 = 〈m̃, ∂mL〉 = 〈m̃, ∂mr̄(u, v, z)〉, ∀m̃ ∈M

    for which we need to compute v ∈ V by solving the adjoint problem

    〈ũ, ∂ur̄(u, v, z)〉 = −〈ũ, ∂uQ̄〉, ∀ũ ∈ U

    Ghattas (Oden Inst/UT Austin) Stochastic PDE-constrained optimization ICIAM 2019 15 / 47

  • Eigenproblem constrained optimization

    To compute λj , which satisfies for j = 1, . . . , N

    Hm(m̄)ψj = λjC−1ψj , and 〈ψj ,C−1ψj〉 = δij

    we need Hessian action in a direction m̂, for which we form the Lagrangian

    LH(u, v,m, z; û, v̂, m̂) = 〈m̂, ∂mr̄〉︸ ︷︷ ︸gradient

    + 〈v̂, ∂v r̄〉︸ ︷︷ ︸forward

    + 〈û, ∂ur̄ + ∂uQ̄〉︸ ︷︷ ︸adjoint

    which involves the gradient, the forward problem, and the adjoint problem. TheHessian action is given by the variation of LH with respect to m:

    〈m̃,Hm(m̄) m̂〉 = 〈m̃, ∂mLH m̂〉 = 〈m̃, ∂mv r̄ v̂ + ∂mur̄ û+ ∂mmr̄ m̂〉, ∀m̃ ∈M

    where û ∈ U is the solution of the incremental forward problem, ∂uLH = 0

    〈ṽ, ∂vur̄ û〉 = −〈ṽ, ∂vmr̄ m̂〉, ∀ṽ ∈ V

    and v̂ ∈ V is the solution of the incremental adjoint problem, ∂vLH = 0

    〈ũ, ∂uv r̄ v̂〉 = −〈ũ, ∂uur̄ û+ ∂uuQ̄ û+ ∂umr̄ m̂〉, ∀ũ ∈ U

    Ghattas (Oden Inst/UT Austin) Stochastic PDE-constrained optimization ICIAM 2019 16 / 47

  • Eigenproblem constrained optimization

    To compute λj , which satisfies for j = 1, . . . , N

    Hm(m̄)ψj = λjC−1ψj , and 〈ψj ,C−1ψj〉 = δij

    we need Hessian action in a direction m̂, for which we form the Lagrangian

    LH(u, v,m, z; û, v̂, m̂) = 〈m̂, ∂mr̄〉︸ ︷︷ ︸gradient

    + 〈v̂, ∂v r̄〉︸ ︷︷ ︸forward

    + 〈û, ∂ur̄ + ∂uQ̄〉︸ ︷︷ ︸adjoint

    which involves the gradient, the forward problem, and the adjoint problem. TheHessian action is given by the variation of LH with respect to m:

    〈m̃,Hm(m̄) m̂〉 = 〈m̃, ∂mLH m̂〉 = 〈m̃, ∂mv r̄ v̂ + ∂mur̄ û+ ∂mmr̄ m̂〉, ∀m̃ ∈M

    where û ∈ U is the solution of the incremental forward problem, ∂uLH = 0

    〈ṽ, ∂vur̄ û〉 = −〈ṽ, ∂vmr̄ m̂〉, ∀ṽ ∈ V

    and v̂ ∈ V is the solution of the incremental adjoint problem, ∂vLH = 0

    〈ũ, ∂uv r̄ v̂〉 = −〈ũ, ∂uur̄ û+ ∂uuQ̄ û+ ∂umr̄ m̂〉, ∀ũ ∈ U

    Ghattas (Oden Inst/UT Austin) Stochastic PDE-constrained optimization ICIAM 2019 16 / 47

  • Eigenproblem constrained optimization

    To compute λj , which satisfies for j = 1, . . . , N

    Hm(m̄)ψj = λjC−1ψj , and 〈ψj ,C−1ψj〉 = δij

    we need Hessian action in a direction m̂, for which we form the Lagrangian

    LH(u, v,m, z; û, v̂, m̂) = 〈m̂, ∂mr̄〉︸ ︷︷ ︸gradient

    + 〈v̂, ∂v r̄〉︸ ︷︷ ︸forward

    + 〈û, ∂ur̄ + ∂uQ̄〉︸ ︷︷ ︸adjoint

    which involves the gradient, the forward problem, and the adjoint problem. TheHessian action is given by the variation of LH with respect to m:

    〈m̃,Hm(m̄) m̂〉 = 〈m̃, ∂mLH m̂〉 = 〈m̃, ∂mv r̄ v̂ + ∂mur̄ û+ ∂mmr̄ m̂〉, ∀m̃ ∈M

    where û ∈ U is the solution of the incremental forward problem, ∂uLH = 0

    〈ṽ, ∂vur̄ û〉 = −〈ṽ, ∂vmr̄ m̂〉, ∀ṽ ∈ V

    and v̂ ∈ V is the solution of the incremental adjoint problem, ∂vLH = 0

    〈ũ, ∂uv r̄ v̂〉 = −〈ũ, ∂uur̄ û+ ∂uuQ̄ û+ ∂umr̄ m̂〉, ∀ũ ∈ U

    Ghattas (Oden Inst/UT Austin) Stochastic PDE-constrained optimization ICIAM 2019 16 / 47

  • Eigenproblem constrained optimization

    To compute λj , which satisfies for j = 1, . . . , N

    Hm(m̄)ψj = λjC−1ψj , and 〈ψj ,C−1ψj〉 = δij

    we need Hessian action in a direction m̂, for which we form the Lagrangian

    LH(u, v,m, z; û, v̂, m̂) = 〈m̂, ∂mr̄〉︸ ︷︷ ︸gradient

    + 〈v̂, ∂v r̄〉︸ ︷︷ ︸forward

    + 〈û, ∂ur̄ + ∂uQ̄〉︸ ︷︷ ︸adjoint

    which involves the gradient, the forward problem, and the adjoint problem. TheHessian action is given by the variation of LH with respect to m:

    〈m̃,Hm(m̄) m̂〉 = 〈m̃, ∂mLH m̂〉 = 〈m̃, ∂mv r̄ v̂ + ∂mur̄ û+ ∂mmr̄ m̂〉, ∀m̃ ∈M

    where û ∈ U is the solution of the incremental forward problem, ∂uLH = 0

    〈ṽ, ∂vur̄ û〉 = −〈ṽ, ∂vmr̄ m̂〉, ∀ṽ ∈ V

    and v̂ ∈ V is the solution of the incremental adjoint problem, ∂vLH = 0

    〈ũ, ∂uv r̄ v̂〉 = −〈ũ, ∂uur̄ û+ ∂uuQ̄ û+ ∂umr̄ m̂〉, ∀ũ ∈ U

    Ghattas (Oden Inst/UT Austin) Stochastic PDE-constrained optimization ICIAM 2019 16 / 47

  • OUU problem with quadratic approximation Jquad

    minz∈Z

    Jquad(z) := Q(m̄) +1

    2

    N∑j=1

    λj(H̃) + β

    (〈gm(m̄),Cgm(m̄)〉+

    1

    2

    N∑j=1

    λ2j (H̃)

    )+ P(z)

    where:

    forward 〈v∗, ∂v r̄〉 = 0 ∀v∗ ∈ V

    adjoint 〈u∗, ∂ur̄ + ∂uQ̄〉 = 0 ∀u∗ ∈ U

    (gradient defn) 〈gm(m̄),C gm(m̄)〉 = 〈∂mr̄(u, v, z),C ∂mr̄(u, v, z)〉

    eigenvalue 〈ψ∗j , (Hm(m̄)− λjC−1)ψj〉 = 0 ∀ψ∗j ∈M j = 1, . . . , N

    orthonormality λ∗j (〈ψj ,C−1ψj〉 − 1) = 0 ∀λ∗j ∈ R j = 1, . . . , N

    incremental forw 〈v̂∗j , ∂vur̄ ûj + ∂vmr̄ ψj〉 = 0 ∀v̂∗j ∈ V j = 1, . . . , N

    incremental adj 〈û∗j , ∂uv r̄ v̂j + ∂uur̄ ûj + ∂uuQ̄ ûj + ∂umr̄ ψj〉 = 0 ∀û∗j ∈ U j = 1, N

    (Hessian defn) 〈ψ∗j ,Hm(m̄)ψj〉 = 〈ψ∗j , ∂mv r̄ v̂ + ∂mur̄ û+ ∂mmr̄ ψj〉 ∀ψ∗j ∈MGhattas (Oden Inst/UT Austin) Stochastic PDE-constrained optimization ICIAM 2019 17 / 47

  • Lagrangian of the OUU problem

    Lquad(u, v, {λj}, {ψj}, {ûj}, {v̂j}, u∗, v∗, {λ∗j}, {ψ∗j }, {û∗j}, {v̂∗j }, z

    ):=

    quad obj = Q(m̄) +1

    2

    N∑j=1

    λj(H̃) + β

    (〈gm(m̄),Cgm(m̄)〉+

    1

    2

    N∑j=1

    λ2j (H̃)

    )+ P(z)

    forward + 〈v∗, ∂v r̄〉adjoint + 〈u∗, ∂ur̄ + ∂uQ̄〉

    eigen. prob. +N∑j=1

    〈ψ∗j , (Hm(m̄)− λjC−1)ψj〉

    orth. cond. +N∑j=1

    λ∗j (〈ψj ,C−1ψj〉 − 1)

    inc. fwd. +N∑j=1

    〈v̂∗j , ∂vur̄ ûj + ∂vmr̄ ψj〉

    inc. adj. +N∑j=1

    〈û∗j , ∂uv r̄ v̂j + ∂uur̄ ûj + ∂uuQ̄ ûj + ∂umr̄ ψj〉

    Ghattas (Oden Inst/UT Austin) Stochastic PDE-constrained optimization ICIAM 2019 18 / 47

  • Gradient of Jquad (assuming λj distinct)

    Variation of Lquad wrt λj vanishes:

    ψ∗j =1 + 2βλj

    2ψj , j = 1, . . . , N

    Variation of Lquad wrt v̂j vanishes:

    û∗j =1 + 2βλj

    2ûj , j = 1, . . . , N

    Variation of Lquad wrt ûj vanishes:

    v̂∗j =1 + 2βλj

    2v̂j , j = 1, . . . , N

    Variation of Lquad wrt v vanishes: find u∗ ∈ U s.t. (incr forward operator)

    〈ṽ, ∂vur̄ u∗〉 = −2β〈ṽ, ∂vmr̄ (C∂mr̄)〉

    −N∑j=1

    〈ṽ, ∂vmur̄ ûj ψ∗j + ∂vmmr̄ ψj ψ∗j 〉

    −N∑j=1

    〈ṽ, ∂vuur̄ ûj û∗j + ∂vumr̄ ψj û∗j 〉, ∀ṽ ∈ V

    Ghattas (Oden Inst/UT Austin) Stochastic PDE-constrained optimization ICIAM 2019 19 / 47

  • Computing the gradient of the OUU problem

    Variation of Lquad wrt u vanishes: find v∗ ∈ V s.t. (incr adjoint operator)

    〈ũ, ∂uv r̄ v∗〉 =− 〈ũ, ∂uQ̄〉 − 2β〈ũ, ∂umr̄ (C∂mr̄)〉− 〈ũ, ∂uur̄ u∗ + ∂uuQ̄ u∗〉

    −N∑j=1

    〈ũ, ∂umv r̄ v̂j ψ∗j + ∂umur̄ ûj ψ∗j + ∂uumr̄ ψj ψ∗j 〉

    −N∑j=1

    〈ũ, ∂uvur̄ ûj v̂∗j + ∂uvmr̄ ψj v̂∗j 〉

    −N∑j=1

    〈ũ, ∂uuv r̄ v̂j û∗j + ∂uuur̄ ûj û∗j + ∂uuuQ̄ ûj û∗j + ∂uumr̄ ψj û∗j 〉,∀ũ ∈ U,

    Finally the gradient of the cost functional can be computed as

    DzJquad(z) = ∂zLquad(primal, dual, z)

    Total cost: 1 forward PDE solve, 1 + 4(N + p) + 2N + 2 linearized PDEsolves (independent of uncertain parameter or control dimensions!)

    Ghattas (Oden Inst/UT Austin) Stochastic PDE-constrained optimization ICIAM 2019 20 / 47

  • Outline

    1 Examples of stochastic PDE constrained optimization problems

    2 Mean-variance PDE-constrained optimization via Taylor approximation

    3 Taylor approximation as a control variate

    4 Optimal design of acoustic metamaterial cloak under uncertainty

    5 Chance constraints with PDE models

    Ghattas (Oden Inst/UT Austin) Stochastic PDE-constrained optimization ICIAM 2019 21 / 47

  • Quadratic approximation as a variance reduction

    Statistics computed by quadratic approximation may be biased

    Use Monte Carlo quadrature to correct quadratic approximation

    E[Q] = E[Qquad] + E[Q−Qquad︸ ︷︷ ︸Y

    ] ≈ E[Qquad] + Ŷ︸︷︷︸MC estimator

    Mean squared error (MSE) of MC estimate of E[Q] and E[Y ]

    MSE(Q) � 1M

    Var[Q] vs. MSE(Y ) � 1M

    Var[Y ]

    A much smaller number of MC samples is required for E[Y ] as

    Var[Y ]� Var[Q]

    provided Qquad is a good approximation of (highly correlated to) Q

    Similar variance reduction can be applied for the variance Var[Q].

    A form of Multifidelity Monte Carlo method (Willcox et al.)

    Ghattas (Oden Inst/UT Austin) Stochastic PDE-constrained optimization ICIAM 2019 22 / 47

  • The unbiased cost functional with variance reduction

    We obtain an unbiased evaluation of the cost functional as

    JMCquad(z) = Q̂quad + βV̂

    quadQ

    + P(z)

    where

    Q̂quad = Q(m̄) +1

    2tr(H)

    +1

    M

    M∑i=1

    (Q(mi)−Q(m̄)− 〈mi − m̄, gm(m̄)〉

    −1

    2〈mi − m̄,Hm(m̄) (mi − m̄)〉

    )and

    V̂quadQ

    := 〈Cgm(m̄), gm(m̄)〉 +1

    4(tr(H))2 +

    1

    2tr(H2)

    +1

    M

    M∑i=1

    ((Q(mi)−Q(m̄))

    2

    −(〈mi − m̄, gm(m̄)〉 +

    1

    2〈mi − m̄,Hm(m̄) (mi − m̄)〉

    )2)

    −( 1

    2tr(H) +

    1

    M2

    M2∑i=1

    (Q(mi)− 〈mi − m̄, gm(m̄)〉

    −1

    2〈mi − m̄,Hm(m̄) (mi − m̄)〉

    ))2Ghattas (Oden Inst/UT Austin) Stochastic PDE-constrained optimization ICIAM 2019 23 / 47

  • OUU Lagrangian w/variance reduction using quad approx

    LMCquad

    (u, v, {ui}, {λj}, {ψj}, {ûj}, {v̂j}, {ûi}, {v̂i},

    u∗, v

    ∗, {vi}, {λ

    ∗j }, {ψ

    ∗j }, {û

    ∗j }, {v̂

    ∗j }, {û

    ∗i }, {v̂

    ∗i }, z

    )= J

    MCquad + 〈v

    ∗, ∂v r̄〉 + 〈u∗, ∂ur̄ + ∂uQ̄〉 +

    M∑i=1

    r(ui, vi,mi, z).

    +N∑

    j=1

    〈ψ∗j , (Hm(m̄)− λjC−1

    )ψj〉

    +N∑

    j=1

    λ∗j (〈ψj ,C

    −1ψj〉 − 1)

    +N∑

    j=1

    〈v̂∗j , ∂vur̄ ûj + ∂vmr̄ ψj〉

    +N∑

    j=1

    〈û∗j , ∂uv r̄ v̂j + ∂uur̄ ûj + ∂uuQ̄ ûj + ∂umr̄ ψj〉

    +M∑i=1

    〈v̂∗i , ∂vur̄ ûi + ∂vmr̄ mi〉

    +M∑i=1

    〈û∗i , ∂uv r̄ v̂i + ∂uur̄ ûi + ∂uuQ̄ ûi + ∂umr̄ mi〉.

    Total: 1 +M forward PDE solves and 3 + 4(N + p) + 4N + 5M linearized PDE solves

    Ghattas (Oden Inst/UT Austin) Stochastic PDE-constrained optimization ICIAM 2019 24 / 47

  • Outline

    1 Examples of stochastic PDE constrained optimization problems

    2 Mean-variance PDE-constrained optimization via Taylor approximation

    3 Taylor approximation as a control variate

    4 Optimal design of acoustic metamaterial cloak under uncertainty

    5 Chance constraints with PDE models

    Ghattas (Oden Inst/UT Austin) Stochastic PDE-constrained optimization ICIAM 2019 25 / 47

  • Optimal design of acoustic metamaterial cloak: Setup

    • Helmholtz problem:

    ∆u+ k2u = (k20 − k2)uin in D∇u · n = −∇uin · n on Γobs

    limr→∞ r(d−1)

    2 (∂ru− iku) = 0

    • Absorbing BC on Γout via PML

    • u: (complex) scattered field =total field − incident field

    u = uto − uin

    • k: wavenumber ω/c, given by{k(x) = ω

    c(x)in metamaterial

    k0 =ωc0

    in medium

    • Wavespeed given by

    c(x) = c0em(x)−z(x)

    obstacle medium

    metamaterial

    PML

    PML

    PML

    inci

    dent

    pla

    ne w

    ave

    incident field total field

    Ghattas (Oden Inst/UT Austin) Stochastic PDE-constrained optimization ICIAM 2019 26 / 47

  • Optimal design of acoustic cloak: Setup

    The complex field u = ur + iui and adjoint v = vr + ivi, are defined in the Hilbertspace (ur, ui), (vr, vi) ∈ V = H1Γin ×H

    1Γin

    , where

    H1Γin = {w ∈ L2(D), |∇w| ∈ L2(D), w|Γin = 0}

    The weak form is given by: find (ur, ui) ∈ V such thatr(u, v,m, z) = 0 ∀(vr, vi) ∈ V

    with r(u, v,m, z) = r1(u, vr,m, z) + ir2(u, vi,m, z), where

    r1(u, vr,m, z) =

    ∫D

    Ar∇ur · ∇vr +Ai∇ui · ∇vrdx−∫D

    Krurvr +Kiuivrdx

    r2(u, vi,m, z) =

    ∫D

    −Ar∇ui · ∇vi +Ai∇ur · ∇vidx−∫D

    Kruivi −Kiurvidx

    where Ar, Ai,Kr,Ki depend on (m, z) through the wavenumber k

    The objective is to eliminate the scattered field in the background medium Dback

    Q(u(m, z)) =

    ∫Dback

    |u(m, z)|2 dx

    The penalty term promotes design sparsity in the metamaterial via L1-norm

    P(z) = α

    ∫Dmeta

    |z| dx

    Ghattas (Oden Inst/UT Austin) Stochastic PDE-constrained optimization ICIAM 2019 27 / 47

  • Optimal design of acoustic cloak: Samples

    DOF for FE discretization of state, random, and design variables (FEniCS)

    DOF mesh1 mesh2 mesh3 mesh4 mesh5u (P2) 40,194 159,746 636,930 2,543,618 10,166,274m, z (P1) 940 3,336 12,487 48,288 189,736

    The random field m ∼ N(m̄,C) with mean m̄ = 0 and covariance

    C = (−γ∆ + δI)−2

    where correlation length ∼√

    γδ and variance ∼ δ

    −2 (equiv. to Matérn family)

    Samples of the random field m(γ = δ = 50, corresponding to manufacturing error of 10% ∼ 15% of material property)

    Ghattas (Oden Inst/UT Austin) Stochastic PDE-constrained optimization ICIAM 2019 28 / 47

  • No cloak vs. optimal cloak: Total wave field

    Ghattas (Oden Inst/UT Austin) Stochastic PDE-constrained optimization ICIAM 2019 29 / 47

  • Optimal cloak: Wave fields

    Top: No cloak: Incident field and total field with obstacleBottom: Optimal cloak: Total field and scattered field

    Ghattas (Oden Inst/UT Austin) Stochastic PDE-constrained optimization ICIAM 2019 30 / 47

  • Optimal design of acoustic cloak: Optimal design

    Optimal design (∞-dim design field z) with different approximationsTop: Random design, deterministic; Bottom: quadratic, SAA

    Ghattas (Oden Inst/UT Austin) Stochastic PDE-constrained optimization ICIAM 2019 31 / 47

  • Deterministic vs stochastic and Quad vs SAA

    Figure: Std of the scattered field at optimal design zquad (left) and zdeter (right)

    Figure: Mean of the scattered field at optimal design zquad (left) and zsaa (right)Ghattas (Oden Inst/UT Austin) Stochastic PDE-constrained optimization ICIAM 2019 32 / 47

  • Variance reduction by quadratic approximation

    Table: Estimates of misfit Q and mean square errors with 100 samples

    design Q̂ MSE(Q̂) MSE(Q−Qlin) MSE(Q−Qquad)zrandom 6.56e+01 9.67e-02 9.80e-03 1.63e-05zdeter 2.55e+00 4.75e-02 4.75e-02 7.30e-04zquad 1.17e+00 4.85e-03 4.31e-03 6.74e-04zsaa 6.46e+00 1.01e-02 1.29e-03 3.37e-05

    =⇒ Variance reduction of 10X–1000X by quadratic approximation

    Table: Estimates of q = (Q− Q̄)2 and mean square errors with 100 samples

    design q̂ MSE(q̂) MSE(q − qlin) MSE(q − qquad)zrandom 1.01e+01 2.97e+00 1.90e+00 1.50e-03zdeter 1.13e+01 4.89e+00 4.89e+00 7.32e-02zquad 1.30e+00 4.06e-02 3.81e-02 1.07e-02zsaa 1.41e+00 2.54e-02 1.54e-02 2.89e-04

    =⇒ Variance reduction of 100X–1000X by quadratic approximation

    Ghattas (Oden Inst/UT Austin) Stochastic PDE-constrained optimization ICIAM 2019 33 / 47

  • Optimal design of acoustic cloak: Trace estimate

    0 20 40 60 80 100N

    10-5

    10-4

    10-3

    10-2

    10-1

    100at random design

    λ+

    λ−errorMC

    errorSVD

    0 20 40 60 80 100N

    10-8

    10-7

    10-6

    10-5

    10-4

    10-3

    10-2

    10-1

    100

    101at optimal design with constant approximation

    λ+

    λ−errorMC

    errorSVD

    0 20 40 60 80 100N

    10-6

    10-5

    10-4

    10-3

    10-2

    10-1

    100

    101at optimal design with quadratic approximation

    λ+

    λ−errorMC

    errorSVD

    0 20 40 60 80 100N

    10-6

    10-5

    10-4

    10-3

    10-2

    10-1

    100at optimal design with saa approximation

    λ+

    λ−errorMC

    errorSVD

    Eigenvalues λN (C1/2Hm(m̄)C

    1/2) (first 100 out of 189,736)and trace estimation errors by MC and randomized SVD

    Ghattas (Oden Inst/UT Austin) Stochastic PDE-constrained optimization ICIAM 2019 34 / 47

  • Compactness of the Hessian for inverse medium scattering

    Theorem

    Let (1− n) ∈ Cm,α0 , where n is the refractive index, m ∈ N ∪ {0}, α ∈ (0, 1).The Hessian is a compact operator everywhere.

    Time harmonic (Helmholtz equation), noise-free

    The proof uses Newton potential theory, Riesz-Fredholm theory andcompact embeddings in Hölder spaces.

    The theorem holds for both continuous and pointwise observations.

    The decay rate of the eigenvalues of the Hessian as a function of thesmoothness of the refractive index n are also obtained.

    If the medium refractive index is analytic, the decay is exponential

    T. Bui-Thanh and O. Ghattas, Analysis of the Hessian for inverse scattering problems. Parts I,II, III, Inverse Problems 2012a, 2012b; Inverse Problems and Imaging 2013.

    Ghattas (Oden Inst/UT Austin) Stochastic PDE-constrained optimization ICIAM 2019 35 / 47

  • How scalable (wrt to parameter/design dimension) is this?

    Complexity of OUU is measured in # of PDE solves

    Overall complexity: # of PDE solves per iteration × # of iterations# of PDE solves per iteration is:

    1 +M nonlinear forward PDE solves3 + 4(N + p) + 4N + 5M linearized forward/adjoint PDE solves

    N scales with dominant spectrum of covariance-preconditionedHessian (used to build quadratic approximation)

    M scales with # of MC samples needed using quadratic controlvariate

    Want to show that N , M , and # of optimization iterations scaleindependent of random parameter/design variable dimension

    Ghattas (Oden Inst/UT Austin) Stochastic PDE-constrained optimization ICIAM 2019 36 / 47

  • Optimal design of acoustic cloak: Scalability I

    0 20 40 60 80 100N

    10-4

    10-3

    10-2

    10-1

    100

    101|λN|

    at random design

    dim = 940

    dim = 3,336

    dim = 12,487

    dim = 48,288

    dim = 189,736

    0 20 40 60 80 100N

    10-7

    10-6

    10-5

    10-4

    10-3

    10-2

    10-1

    100

    101

    |λN|

    at optimal design with constant approximation

    dim = 940

    dim = 3,336

    dim = 12,487

    dim = 48,288

    dim = 189,736

    0 20 40 60 80 100N

    10-6

    10-5

    10-4

    10-3

    10-2

    10-1

    100

    101

    |λN|

    at optimal design with quadratic approximation

    dim = 940

    dim = 3,336

    dim = 12,487

    dim = 48,288

    dim = 189,736

    0 20 40 60 80 100N

    10-6

    10-5

    10-4

    10-3

    10-2

    10-1

    100

    101

    |λN|

    at optimal design with saa approximation

    dim = 940

    dim = 3,336

    dim = 12,487

    dim = 48,288

    dim = 189,736

    Spectrum decay of the covariance-preconditioned Hessian is scalableGhattas (Oden Inst/UT Austin) Stochastic PDE-constrained optimization ICIAM 2019 37 / 47

  • Optimal design of acoustic cloak: Scalability II

    Table: Estimates of misfit Q and mean square errors with 100 samples

    dimension Q̂ MSE(Q̂) MSE(Q−Qlin) MSE(Q−Qquad)940 7.33e+01 1.25e-01 7.01e-03 7.16e-05

    3,336 6.87e+01 1.56e-01 9.29e-03 7.51e-0512,487 6.56e+01 9.67e-02 9.80e-03 1.63e-0548,288 6.94e+01 1.00e-01 1.04e-02 1.13e-04

    Variance reduction of 1000X (at random design) is scalable

    Table: Estimates of q = (Q− Q̄)2 and mean square errors with 100 samples

    dimension q̂ MSE(q̂) MSE(q − qlin) MSE(q − qquad)940 1.44e+01 3.19e+00 1.42e+00 7.53e-03

    3,336 2.06e+01 1.13e+01 3.10e+00 1.99e-0212,487 1.01e+01 2.97e+00 1.90e+00 1.50e-0348,288 1.21e+01 4.82e+00 2.52e+00 4.92e-03

    Variance reduction of 1000X (at random design) is scalable

    Ghattas (Oden Inst/UT Austin) Stochastic PDE-constrained optimization ICIAM 2019 38 / 47

  • Optimal design of acoustic cloak: Scalability III

    0 5 10 15 20 25 30 35# BFGS iterations

    10-2

    10-1

    100

    101

    102

    cost

    at optimal design with constant approximation

    dim = 940

    dim = 3,336

    dim = 12,487

    dim = 48,288

    dim = 189,736

    0 5 10 15 20 25 30 35# BFGS iterations

    100

    101

    102

    103

    cost

    at optimal design with quadratic approximation

    dim = 940

    dim = 3,336

    dim = 12,487

    dim = 48,288

    dim = 189,736

    0 5 10 15 20 25 30 35# BFGS iterations

    100

    101

    102

    103

    cost

    at optimal design with saa approximation

    dim = 940

    dim = 3,336

    dim = 12,487

    dim = 48,288

    dim = 189,736

    Optimization (# BFGS inter) is scalable by quadratic approximationGhattas (Oden Inst/UT Austin) Stochastic PDE-constrained optimization ICIAM 2019 39 / 47

  • Optimal design of acoustic cloak: Complex geometry

    Top: Geometry and adaptive mesh for complex obstacle.Bottom: Optimal design with deterministic approximation Scattered wave without (T) and with (B) cloak.

    Ghattas (Oden Inst/UT Austin) Stochastic PDE-constrained optimization ICIAM 2019 40 / 47

  • Outline

    1 Examples of stochastic PDE constrained optimization problems

    2 Mean-variance PDE-constrained optimization via Taylor approximation

    3 Taylor approximation as a control variate

    4 Optimal design of acoustic metamaterial cloak under uncertainty

    5 Chance constraints with PDE models

    Ghattas (Oden Inst/UT Austin) Stochastic PDE-constrained optimization ICIAM 2019 41 / 47

  • Chance-constrained OUU: Approximations

    minz∈Z

    J(z) := E[q] + wP(z)

    subject to the PDE constraint and the inequality chance constraint

    P (f(·, z) ≥ 0) ≤ αThe failure probability can be equivalently written as

    P (f(·, z) ≥ 0) = E[I[0,∞)(f(·, z))] =∫M

    I[0,∞)(f(m, z))dµ(m)

    Smooth approximation:

    I[0,∞)(x) ≈ `β(x) =1

    1 + e−2βx

    By I[0,∞)(0) := 12 , we have

    limβ→∞

    `β(x) = I[0,∞)(x)

    limβ→∞

    ∇`β(x) =−2βe−2βx

    (1 + e−2βx)2= ∇I[0,∞)(x)

    2.0 1.5 1.0 0.5 0.0 0.5 1.0 1.5 2.0x

    0.0

    0.2

    0.4

    0.6

    0.8

    1.0

    ` β(x

    )

    I(−∞,0]

    β=1

    β=2

    β=4

    β=8

    Ghattas (Oden Inst/UT Austin) Stochastic PDE-constrained optimization ICIAM 2019 42 / 47

  • Chance-constrained OUU: Approximations

    Quadratic approximation:

    fquad(m, z) := f(m̄, z) + 〈m− m̄, gfm(m̄)〉+1

    2〈m− m̄,Hfm(m̄) (m− m̄)〉

    Low-rank approximation: by the generalized eigenvalue problems

    Hfm(m̄)ψ

    fn = λ

    fnC−1ψfn, n = 1, . . . , Nf ,

    we define the low-rank approximation of fquad(m, z) as

    fquad,Nf (m, z) := f(m̄, z) + 〈m− m̄, gfm(m̄)〉+

    1

    2

    Nf∑n=1

    λfn〈m− m̄,C−1ψfn〉2

    Sample average approximation:

    P (f(·, z) ≥ 0) ≈ fMf (z) :=1

    Mf

    Mf∑i=1

    I[0,∞)(f(mi, z))

    where mi, i = 1, . . . ,Mf , are i.i.d. samples from µ. Combine all approximations

    P (f(·, z) ≥ 0) ≈ fβ,Mf ,Nf ,quad(z) :=1

    Mf

    Mf∑i=1

    `β(fquad,Nf (mi, z))

    Only O(Nf ) PDEs need to be solved, with Nf �Mf for small chance α.Ghattas (Oden Inst/UT Austin) Stochastic PDE-constrained optimization ICIAM 2019 43 / 47

  • MS: PDE-constrained optimization under uncertainty

    Part 1: Thursday 14:30–16:30 in Room A6-1-1

    A scalable method for PDE-constrained optimization underhigh-dimensional uncertainty (Chen, Villa, Ghattas)A multilevel stochastic gradient algorithm for PDE-constrained optimalcontrol problems under uncertainty (Nobile, Martin, Tsilifis)Low-rank tensor methods for optimal control of uncertain flowproblems (Benner, Dolgov, Onwunta, Stoll)Bayesian search methods for engineering design (Cook, Marzouk)

    Part 2: Friday 11:00–12:30 in Room A6-1-1

    Adjustable stochastic optimal control with shared support (Stadler, Li)A robust optimization approach for PDE-constrained optimizationunder uncertainty (Ulbrich, Kolvenbach, Lass)A primal-dual algorithm for risk-averse PDE-constrained optimization(Surowiec, Kouri)

    Ghattas (Oden Inst/UT Austin) Stochastic PDE-constrained optimization ICIAM 2019 44 / 47

  • Summary of approach to PDE-constrained OUU

    Construct 2nd order Taylor approximation (wrt random parameters) ofcontrol objective, and use as a variance reduction tool for mean-varianceOUU

    Hessian of parameter-to-objective map is compact, with fast decayingeigenvalues.

    Randomized SVD used to accurately and efficiently capture the low-rank

    Leads to an PDE-constrained optimization problem constrained by a Hessianeigenvalue problem, with state and adjoint PDE constraints to define thegradient entering the objective approximation, and incremental state andadjoint PDE constraints to define the Hessian action

    Solved for sequence of OUU problems with up to 1 million randomparameters, demonstrated scalability (i.e., # of PDE solves constant withincreasing random parameter and control dimensions)

    Trace estimation by randomized SVD is scalableQuasi-Newton optimization iterations are weakly scalableVariance reduction is scalable=⇒ Overall method is scalable

    Ghattas (Oden Inst/UT Austin) Stochastic PDE-constrained optimization ICIAM 2019 45 / 47

  • Limitations and ongoing work

    Limitations:

    Taylor approximation is local; variance reduction can deteriorate for large3rd derivatives or large variances (M can be large)

    Even when the Hessian is compact, the eigenvalues may not decay rapidly incertain parameter regimes such as increasing frequency for Helmholtz,increasing Peclet number for advection diffusion, increasing Reynoldsnumber for Navier-Stokes, etc. (N can be large)

    Ongoing and future work:

    Newton (as opposed to quasi-Newton) for optimization

    Higher order Taylor approximations when applicable (Tensor compression)

    Multifidelity approximation with multiple quadratics Qquad(mi)

    Alternatives to low-rank approximation of Hessian including hierarchicalmatrices and translation-invariance

    Chance constraints

    Ghattas (Oden Inst/UT Austin) Stochastic PDE-constrained optimization ICIAM 2019 46 / 47

  • References

    P. Chen, M. Haberman, and O. Ghattas, Optimal Design of AcousticMetamaterials Under Uncertainty, manuscript, 2019.

    P. Chen, U. Villa, and O. Ghattas, Taylor approximation and variance reduction forPDE-constrained optimal control problems under uncertainty, Journal ofComputational Physics, 385:163–186, 2019.

    A. Alexanderian, N. Petra, G. Stadler, and O. Ghattas, Mean-variance risk-averseoptimal control of systems governed by PDEs with random parameter fields usingquadratic approximations, SIAM/ASA Journal on Uncertainty Quantification,5(1):1166–1192, 2017.

    A. Alexanderian, N. Petra, G. Stadler, and O. Ghattas, A fast and scalable methodfor A-optimal design of experiments for infinite-dimensional Bayesian nonlinearinverse problems, SIAM Journal on Scientific Computing, 38(1):A243–A272,2016.

    T. Bui-Thanh and O. Ghattas, Analysis of the Hessian for inverse scatteringproblems. Parts I, II, III, Inverse Problems 2012a, 2012b; Inverse Problems andImaging 2013.

    N. Alger, V. Rao, A. Myers, T. Bui-Thanh, and O. Ghattas, Scalable matrix-freeadaptive convolution-product approximation for locally translation-invariantoperators, SIAM Journal on Scientific Computing, to appear, 2019.

    Ghattas (Oden Inst/UT Austin) Stochastic PDE-constrained optimization ICIAM 2019 47 / 47

    Examples of stochastic PDE constrained optimization problemsMean-variance PDE-constrained optimization via Taylor approximationTaylor approximation as a control variateOptimal design of acoustic metamaterial cloak under uncertaintyChance constraints with PDE models

    fd@rm@0: fd@rm@1:


Recommended