Abstract We propose a power penalty method for an obstacle problem arising fromthe discretization of an infinite-dimensional optimization problem involving differen-tial operators in both its objective function and constraints. In this method we approx-imate the mixed nonlinear complementarity problem (NCP) arising from the KKTconditions of the discretized problem by a nonlinear penalty equation. We then showthe solution to the penalty equation converges exponentially to that of the mixed NCP.Numerical results will be presented to demonstrate the theoretical convergence ratesof the method.
Keywords Obstacle problem · Gradient constraints · Financial option pricing ·Variational inequality · Penalty method · Rate of convergence
1 Introduction
Consider the following constrained optimization problem:
minu∈H
F(u) subject to Lu ≤ g,
where H is a function space, F is a functional on H involving a differential operator,L is a given linear differential operator, and g is a known function. Numerical solutionof obstacle problems with the bound constraint u ≤ g has been discussed extensively[7]. However, numerical methods for solving the above problem when L is not the
S. Wang (B)School of Mathematics and Statistics, The University of Western Australia,35 Stirling Highway, Crawley, WA 6009, Australiae-mail: [email protected]
123
S. Wang
identity operator are scarce in the open literature. On the other hand, many real-world problems do contain derivative constraints. Examples are gradient constrainedoptimization problems [3] and the option pricing problems [6,4,11,12].
In practice, the above problem can be discretized by a proper discretization tech-nique such as those in [1,14,17] so that the discretized problem is of the form
minx∈Rm
F(x) subject to Ax ≤ b, (1)
where m is a positive integer, F : Rm �→ R is a nonlinear differentiable function,
A : Rm �→ R
n is an n × m matrix with n a positive integer and b ∈ Rn is a known
vector. We assume that m ≥ n. The KKT conditions for (1) are
f (x) + A�μ = 0, μ ≥ 0, Ax − b ≤ 0, μ�(Ax − b) = 0, (2)
where f (x) = ∇F(x) : Rm �→ R
m and μ ∈ Rn is an unknown multiplier.
The above KKT conditions form a mixed NCP. Theoretical and computationalaspects of conventional NCPs have been studies extensively and many of these resultscan be found in [7]. For mixed CPs, some numerical methods have also been developed(cf., for example [2,5,10,13]). Recently, a power penalty approach has been developedfor linear complementarity problems and NCPs in both finite and infinite dimensions(cf. [11,15,16]). The penalty method has the merits that it does not introduce any extraor auxiliary variables and the resulting algebraic equations are easily solvable by aconventional numerical method such as that of Newton. Most importantly, it has beenshown that the penalty approach has an exponential convergence rate depending onthe parameters used in the penalty terms. In [9], we extended the penalty method in[8,15] to a mixed NCP when the overall mapping involved is ξ -monotone. As can beseen in the next section, the monotonicity condition used in [9] is not satisfied by (2)and thus the results in [9] are not applicable to (2). The aim of this work is to developand analyze a penalty approach to (2). The rest of this paper is organized as follows.
In the next section, we will reformulate (2) as a variational inequality. We will thenpropose, in Sect. 3, a penalty equation approximating (2) and establish a convergencetheory for the penalty method. The special case that A in (2) is invertible will bediscussed in Sect. 4. In Sect. 5, we perform some numerical experiments using anon-trivial large scale problem to confirm the theoretical rates of convergence for thepenalty method.
2 The variational inequality problem
For simplicity, we assume Ax = b has a solution, denoted as xb. In this case, weassume without loss of generality that b = 0 in (2), as the case that b = 0 can betransformed into this under z = x − xb.
Let y = −μ, (2) (with b = 0) can be written as the following problem:
Problem 1 Find (x�, y�)� ∈ Rm × R
n such that
f (x) − A�y = 0, y ≤ 0, Ax ≤ 0, y� Ax = 0. (3)
123
A penalty method for a finite-dimensional obstacle problem
Let K = {(u�, v�)� ∈ Rm × R
n : v ≤ 0}. It is easily seen that K is a convexsubset of R
m × Rn . Using K we define a variational inequality as follows.
Problem 2 Find z = (x�, y�)� ∈ K, such that for all w ∈ K,
(w − z)�G(x, y) := (w − z)�(
f (x) − A�yAx
)≥ 0. (4)
The following theorem establishes the equivalence of Problems 1 and 2.
Theorem 1 A vector z = (x�, y�)� ∈ Rm × R
n is a solution to Problem 1 if andonly if it is a solution to Problem 2.
The proof of this theorem can be found in [9].In what follows we use || · ||p to denote the usual l p-norm on R
k or the subordinatematrix l p-norm for any p > 0 and positive integer k. For the given mappings f andA in Problem 1, we have the following assumptions:
A1. f (x) is continuous on Rm .
A2. f (x) is ξ -monotone, i.e. there exist constants α > 0 and ξ ∈ (1, 2] such that forall x1, x2 ∈ R
m .
(x1 − x2)�( f (x1) − f (x2)) ≥ α||x1 − x2||ξ2, (5)
A3. Without loss of generality, we assume that the linear independent constraintqualification (LICQ) holds for Ax ≤ 0 in (3), i.e., Rank(A) = n.
When Rank(A) < n, some of the constraints in Ax ≤ 0 are linearly dependenton others and they can be eliminated from the constraints. In the rest of this paperwe assume that A1, A2 and A3 are fulfilled. The following theorem establishes themonotonicity of G(x, y).
Lemma 1 For any z1 = (x�1 , y�
1 )�, z2 = (x�2 , y�
2 )� ∈ Rm × R
n, the vector-valuedfunction G defined in Theorem 1 satisfies
Proof From the definition of G it is trivial to show that
(z1 − z2)�[G(x1, y1) − G(x2, y2)] = [ f (x1) − f (x2)]�(x1 − x2) ≥ α||x1 − x2||ξ2
by (5). Thus, the lemma is proved.
123
S. Wang
Using this lemma, we establish the following theorem.
Theorem 2 There exists a unique solution to Problem 2.
Proof Since f is ξ -monotone by (5), F(x) is strictly convex, and so the minimiza-tion problem (1) has a solution. This implies that Problem 1 arising from the KKTconditions of (1) also has a solution. The solvability of Problem 2 then follows fromTheorem 1.
Let zi = (x�i , y�
i )�, i = 1, 2, be two solutions to Problem 2. Then, we have
(w − zi )�G(xi , yi ) ≥ 0, ∀w ∈ K, i = 1, 2. (6)
Replacing w in (6) with z2 and z1 respectively for i = 1, 2, we get
(z2 − z1)�G(x1, y1) ≥ 0 and (z1 − z2)
�G(x2, y2) ≥ 0.
Combining thses two inequalities gives
(z1 − z2)�(G(x1, y1) − G(x2, y2)) ≤ 0.
Therefore, from Lemma 1 we have ||x1 − x2||2 = 0.We denote the two solutions as (x�, y�
i )�, i = 1, 2. Let εk = (0, .., 1, .., 0)� bethe m × 1 matrix with the kth element equal to 1 and all other elements equal to 0for k = 1, 2, . . . , m. Then, ((x ± εk)
�, y�i )� ∈ K for i = 1, 2 and k = 1, . . . , m.
Replacing w in (6) with ((x ±εk)�, y�
i )� for all k = 1, . . . , m and using the definitionof G we have, respectively,
f (x) − A�yi ≥ 0 and f (x) − A�yi ≤ 0, i = 1, 2
This implies f (x) − A�yi = 0 for i = 1, 2, or A�y1 = A�y2. Since A is of rank nby Assumption A3, AA� is invertible. Therefore, left-multiplying A�y1 = A�y2 byA gives y1 = y2. Thus, we have proved this theorem.
3 The power penalty method and convergence rates
Consider the following nonlinear equation:
Problem 3 Find (x�λ , yλ)
� ∈ Rm × R
n , such that
G(xλ, yλ) + λΦ(yλ) := G(xλ, yλ) + λ
(0
[yλ]1/k+
)= 0, (7)
where λ ≥ 1 and k ≥ 0 are penalty parameters, [u]+ = max{u, 0} andyσ = (yσ
1 , . . . , yσn )� for any y = (y1, . . . , yn)� ∈ R
n and constant σ > 0.
123
A penalty method for a finite-dimensional obstacle problem
The solvability of (7) is given in the following theorem.
Theorem 3 For any λ ≥ 1, Problem 3 has a unique solution.
Proof From Lemma 1 we see that G is monotone. It is easy to see that λΦ(yλ) is alsomonotone. Therefore, from Theorem 2.3.5 of [7] we see that there exists a solution toProblem 3.
Let (x�λ , y�
λ )� and (u�λ , v�
λ )� be two solutions to Problem 3. Then, it is trivial toshow using the monotonicity of the mapping in (7) that xλ = uλ.
From the first m equations of (7) we have A�yλ = f (xλ) = A�vλ. This impliesyλ = vλ since A is of rank n by Assumption A3.
We now consider the convergence of the solution to Problem 3 to that of Problem 2.We start this discussion with the following lemma.
Lemma 2 Let zλ = (x�λ , y�
λ )� be a solution to (7) for any λ ≥ 1. Then, there existsa positive constant M, independent of λ and zλ, such that
||xλ||2 ≤ M and ||yλ||2 ≤ M. (8)
Proof Left-multiplying both sides of (7) by z�λ gives
z�λ [G(xλ, yλ) + λΦ(yλ)] = 0.
From this and the definitions of G and Φ in (4) and (7) respectively, we have
z�λ [G(xλ, yλ) − G(0, 0)] = −λy�
λ [yλ]1/k+ − z�
λ G(0, 0)
Using Lemma 1 we get from the above
α||xλ||ξ2 ≤ −x�λ f (0) − λy�
λ [yλ]1/k+ . (9)
But yλ = [yλ]+−[yλ]−, where [u]− = max{−u, 0} for any given function u satisfying[u]+[u]− = 0. Thus,
− y�λ [yλ]1/k
+ = −([yλ]+ − [yλ]−)�[yλ]1/k+ = −[yλ]�+[yλ]1/k
+ ≤ 0. (10)
Therefore, using (10) and Cauchy–Schwarz inequality, we have from (9)
α||xλ||ξ2 ≤ −x�λ f (0) ≤ ||xλ||2|| f (0)||2, or ||xλ||2 ≤ (α−1|| f (0)||2)1/(ξ−1).
Thus, we have proved the first inequality in (8).Now, left-multiplying both sides of (7) by (A, 0) and solving the resulting system
for y gives yλ = (AA�)−1 A f (xλ). Therefore,
||yλ||2 ≤ ||A||2||(AA�)−1||2|| f (xλ)||2.
123
S. Wang
Since f is continuous by Assumption A1 and ||xλ||2 is bounded above by a positiveconstant for all λ ≥ 1, we have that ||yλ||2 is also bounded above by a positive constant.This finishes the proof of (8).
Remark 1 We comment that from Lemma 2 we see that there exists a bounded setD ⊂ R
m such that xλ ∈ D for all λ ≥ 1. Using this lemma, we show in the followinglemma that [yλ]+ converges to zero as λ → ∞.
Lemma 3 For any λ ≥ 1, let (x�λ , y�
λ )� be the solution to (7). Then, there exists apositive constant C, independent of (x�
λ , y�λ )� and λ, such that
||[yλ]+||2 ≤ Cλ−ke , (11)
where λe = λ/ supz∈D ||z||q with q = 1 + k and D defined in Remark 1.
Proof Left-multiplying (7) by (0�, [yλ]�+) yields
[yλ]�+ Axλ + λ[yλ]�+[yλ]1/k+ = 0.
Let p = 1 + 1/k and q = 1 + k. Using Holder’s inequality we have from the above
λ[yλ]�+[yλ]1/k+ ≡ λ||[yλ]+||p
p = −[yλ]�+ Axλ ≤ C ||[yλ]+||p||xλ||q ,
or
||[yλ]+||p−1p ≤ C ||xλ||q · λ−1,
where C is a generic positive constant, independent of λ and (x�λ , y�
λ )�. Taking the(p − 1)-th root on both sides of the above gives
||[yλ]+||p ≤ C ||xλ||1
p−1q · λ
−1p−1 ≤ Cλ−k
e .
Finally, since all norms in Rn are equivalent, (11) follows from the above.
Remark 2 We remark that λe defined in Lemma 3 can be regarded as an effectivepenalty constant. Also, though the constant C is independent of (x�
λ , y�λ )� and λ,
it does depend on n because the equivalence of norms on a finite-dimension spacedepends on the dimensions of the space.
We are now ready to prove the following main convergence result.
Theorem 4 For any λ ≥ 1, let z = (x�, y�)� and zλ = (x�λ , y�
λ )� be the solutionsto Problems 1 and 3 respectively. There exists a constant C > 0, independent of zλ
and λ, such that
||x − xλ||2 ≤ Cλ−k/(ξ−1)e , (12)
where λe is the effective penalty constant. Furthermore, limλ→∞(y − yλ) = 0.
123
A penalty method for a finite-dimensional obstacle problem
Proof Let C be a generic positive constant, independent of zλ and λ. We decomposez − zλ into
z − zλ = z −(
0[yλ]+
)+
( −xλ
[yλ]−
)= rλ −
(0
[yλ]+
),
where rλ := z + (−x�λ , [yλ]�−)�. Note z − rλ = (x�
λ ,−[yλ]�−)� ∈ K, since−[yλ]− ≤ 0. Replacing w in (4) with z − rλ gives
− r�λ G(x, y) ≥ 0. (13)
Left-multiplying both sides of (7) by r�λ , we have
r�λ G(xλ, yλ) + λr�
λ Φ(yλ) = 0. (14)
Adding up both sides of (13) and (14) gives
r�λ [G(xλ, yλ) − G(x, y)] + λr�
λ Φ(yλ) ≥ 0. (15)
From the definitions of rλ and Φ, we get
r�λ Φ(yλ) = (y + [yλ]−)�[yλ]1/k
+ = y�[yλ]1/k+ ≤ 0,
since [yλ]�−[yλ]1/k+ = 0, y ≤ 0 and [yλ]+ ≥ 0. Combining this with (15) leads to
r�λ
[G(x, y) − G(xλ, yλ)
] ≤ 0.
Since [yλ]− = −yλ + [yλ]+, we have rλ = z − zλ + (0�, [yλ]�+)�, and the aboveinequality becomes
When f satisfies (17), it is trivial to show that q also satisfies Holder continuitycondition. Thus, we have proved the theorem.
Now, we define the following penalty equation approximating (21):
q(zλ) + λ[zλ]1/k+ = 0, (23)
where λ > 1 and k ≥ 1 are constants. Then, we have the following theorem:
Theorem 6 Let Assumptions A1 and A2 be fulfilled, and let z and zλ be solutions toProblem 4 and (23) respectively. Then, there exists a positive constant C, independentof λ and zλ, such that
||z − zλ||2 ≤ Cλ−k/ξ .
Furthermore, if f satisfies (17), then
||z − zλ||2 ≤ Cλ−k/(ξ−γ ).
The proof of this theorem can be found in [8].
5 Numerical experiments
Let us consider the optimization problem: find u ∈ C2(0, 1) satisfying u(0) = u(1) =01 such that
u = arg minv′(s)≤g(s)
1∫0
(1
2(v′(s))2 + 1
4v4(s) − p(s)v(s)
)ds,
1 A more rigorous statement is: find u ∈ H10 (0, 1), where H1
0 (0, 1) denotes the usual Sobolev functionalspace on (0, 1) satisfying u(0) = u(1) = 0.
123
S. Wang
where C2(0, 1) denotes the set of functions which have continuous second derivativesin (0, 1) and g(s) and p(s) are given functions on (0, 1). To discretize this problem, wedivide [0, 1] uniformly into N sub-intervals with points si = ih for i = 0, 1, . . . , Nfor a positive integer N , where h = 1/N . On this mesh, we approximate the aboveproblem by
minAx≤b
hN−1∑i=0
[(xi+1 − xi )
2
2h2 + x4i
4− ci xi
]= min
Ax≤b
(x� Bx
2+
N−1∑i=0
[x4
i
4− ci xi
])h,
where bi = g(si ), ci = p(si ) and A and B are respectively (N − 2) × (N − 1)
bi-diagonal and (N − 1) × (N − 1) tri-diagonal matrices given by
A = 1
h
⎛⎜⎜⎜⎜⎜⎝
−1 1−1 1
. . .. . .
−1 1−1 1
⎞⎟⎟⎟⎟⎟⎠
, B = 1
h2
⎛⎜⎜⎜⎜⎜⎝
2 −1−1 2 −1
. . .. . .
. . .
−1 2 −1−1 2
⎞⎟⎟⎟⎟⎟⎠
Therefore, the KKT conditions corresponding to this optimization problem is of thesame form as that in Problem 1 with m = N −1, n = N −2 and f (x) = Bx + x3 −c.Clearly, f is strongly monotone as B is positive-definite.
We choose p(s) = −4π2 sin(2πs) + sin3(2πs) and g(s) = 2π(sin(2πs) + 0.5).The unconstrained problem has the exact solution uun = − sin(2πs). We also chooseN = 100. Since (7) is nonlinear, we use a damped Newton’s method to solve it. NoteΦ in (7) is non-smooth and in the computations, it is smoothed out locally using theformula in [8].
We first investigate the convergence rates in λ for a fixed k. Since the exact solutionto the above constrained problem is unknown, we use the numerical solution withk = 4 and λ = 1010/h2 as the ‘exact’ or reference solution (x∗, y∗). Table 1 is a listof the computed errors in the l2-norm (||x − x∗||22 +||y − y∗||22)1/2 for different values
Fig. 1 Computed u and u′ and multiplier y along with the bound g. a u and unconstrained solution,b u′, g and unconstrained derivative, c multiplier y = −μ
of λ and k and the ratios of the errors between two consecutive values of λ. From (19)it is easy to see that the theoretical ratio for two consecutive values of λ for a fixed kis equal to λk
i+1/λki = 2k . From Table 1, we see that our computed ratios match this
theoretical one well.
123
S. Wang
Let us consider the convergence of the numerical solutions in k for a given λ.Theoretically, from (12) and (18) we see the ratio of the errors for k and k + 1 is aconstant λk+1
e /λke = λe. We now list the computed errors and ratios for k = 1, 2, . . . , 5
and different values of λ in Table 2. From Table 2 we see that the ratios are almostconstants.
The computed solution u and uun are plotted in Fig. 1a. We also plot u′, u′un and
g(s) in Fig. 1b from which we see that u is bounded above by g(s). Figure 1c containsthe computed multiplier y which shows that y = 0 when Ax ≤ b is inactive and y < 0when it is active.
6 Conclusions
In this paper, we have proposed a power penalty method for a discretized obstacleproblem containing derivative constraints. We have shown that the solution to thepenalty equation converges to that of the original problem at the rate O(1/λk), whereλ > 1 is the penalty constant and k ≥ 1 is the power parameter in the penalty term.Numerical results for a non-trivial large-scale test problem showed that the computedrates of convergence coincide with the theoretical ones.
Acknowledgments Project 11001178 supported by National Natural Science Foundation of China.
References
1. Angermann, L., Wang, S.: Convergence of a fitted finite volume method for the penalized Black-Scholesequation governing European and American Option pricing. Numer. Math. 106, 1–40 (2007)
2. Chen, C.H., Mangasarian, O.L.: A class of smoothing functions for nonlinear and mixed complemen-tarity problems. Comput. Optim. Appl. 5, 97–138 (1996)
3. Choe, H.J., Shim, Y.: Degenerate variational inequalities with gradient constraints. Annali della ScuolaNormale Superiore di Pisa, Classes di Scienze 4e serie, 22, 25–53 (1995)
4. Damgaard, A.: Computation of reservation prices of options with proportional transaction costs.J. Econ. Dyn. Control 30, 415–444 (2006)
5. Daryina, A.N., Izmailov, A.F., Solodov, M.V.: A class of active-set Newton methods for mixed com-plementarity problems. SIAM J. Optim. 36, 409–429 (2004)
6. Davis, M.H.A., Zariphopoulou, T.: American options and transaction fees. In: Davis, M.H.A., et al.(eds.) Mathematical Finance, Springer-Verlag, New York (1995)
7. Facchinei, F., Pang, J.S.: Finite-Dimensional Variational Inequalities and Complementarity Problems,vols. I, II. Springer, New York (2003)
8. Huang, C.C., Wang, S.: A power penalty approach to a nonlinear complementarity problem. Oper.Res. Lett. 38, 72–76 (2010)
9. Huang, C.C., Wang, S.: A penalty method for a mixed nonlinear complementarity problem. NonlinearAnal. 75, 588–597 (2012)
10. Kanzow, C.: Global optimization techniques for mixed complementarity problems. J. Global Optim.16, 1–21 (2000)
11. Li, W., Wang, S.: Penalty approach to the HJB equation arising in European stock option pricing withproportional transaction costs. J. Optim. Theory Appl. 143, 279–293 (2009)
12. Li, W., Wang, S.: Pricing American options under proportional transaction costs using a penaltyapproach and a finite difference scheme. J. Ind. Manag. Optim. 9, 365–389 (2013)
13. Monteiro, R.D.C., Pang, J.-S.: Properties of an interior-point mapping for mixed complementarityproblems. Math. Oper. Res. 21, 629–654 (1996)
14. Wang, S.: A novel fitted finite volume method for the Black–Scholes equation governing option pricing.IMA J. Numer. Anal. 24, 699–720 (2004)
123
A penalty method for a finite-dimensional obstacle problem
15. Wang, S., Yang, X.Q.: A power penalty method for linear complementarity problems. Oper. Res. Lett.36, 211–214 (2008)
16. Wang, S., Yang, X.Q., Teo, K.L.: Power penalty method for a linear complementarity problem arisingfrom American option valuation. J. Optim. Theory Appl. 129, 227–254 (2006)
