Indefinite Symmetric System

8/2/2019 Indefinite Symmetric System

1/18

Direct Methods for Solving Symmetric Indefinite Systems of Linear Equations

Author(s): J. R. Bunch and B. N. ParlettReviewed work(s):Source: SIAM Journal on Numerical Analysis, Vol. 8, No. 4 (Dec., 1971), pp. 639-655Published by: Society for Industrial and Applied MathematicsStable URL: http://www.jstor.org/stable/2949596 .

Accessed: 13/03/2012 00:39

Your use of the JSTOR archive indicates your acceptance of the Terms & Conditions of Use, available at .http://www.jstor.org/page/info/about/policies/terms.jsp

JSTOR is a not-for-profit service that helps scholars, researchers, and students discover, use, and build upon a wide range of

content in a trusted digital archive. We use information technology and tools to increase productivity and facilitate new forms

of scholarship. For more information about JSTOR, please contact [email protected].

Society for Industrial and Applied Mathematics is collaborating with JSTOR to digitize, preserve and extend

access to SIAM Journal on Numerical Analysis.

http://www.jstor.org
http://www.jstor.org/action/showPublisher?publisherCode=siamhttp://www.jstor.org/stable/2949596?origin=JSTOR-pdfhttp://www.jstor.org/page/info/about/policies/terms.jsphttp://www.jstor.org/page/info/about/policies/terms.jsphttp://www.jstor.org/stable/2949596?origin=JSTOR-pdfhttp://www.jstor.org/action/showPublisher?publisherCode=siam


2/18

SIAM J. NUMER. ANAL.Vol. 8, No. 4, December 1971

DIRECT METHODS FOR SOLVING SYMMETRIC INDEFINITESYSTEMS OF LINEAR EQUATIONS*J. R. BUNCHt AND B. N. PARLETTt

Abstract. Methods for solving symmetric indefinite systems are surveyed including a new onewhich is stable and almost as fast as the Cholesky method.

1. Introduction.1.1. The problem. Consider direct as against iterative methods for solvingAx = b, where A is an n x n real symmetric matrix. If A is also positive definite,then Cholesky's method and triangular factorization are fast ((1/6)n3 multiplica-tions), stable, and preservesymmetry.If A is indefinite, these methods can producevery inaccurate results and fail to give warning of what has occurred. It is thereforeusual to recommend Gaussian elimination with partial or complete pivoting forindefinite systems, and the symmetry of A is of no advantage.Such systems do arise in practice,not so much in models of physical problemsbut in the intermediate stages which are produced by numerical methods. Therehave been attempts to devise algorithms which take advantage of symmetry andare also competitive with Gaussian elimination. These are reviewed and a newmethod is proposed which uses Kahan's generalization of a pivot to include 2 x 2principal submatrices. The bound on growth of elements in the reduced matricesis nearly as good as Wilkinson's bound for elimination with complete pivoting;the number of multiplications, additions, and comparisons is at most of the order(1/6)n3 for each. Unfortunately, bandwidth is not preserved. The results in ?? 3-6also hold for Hermitian systems, but for convenience we shall only consider realsymmetric systems.

1.2. Summary of methods. Table 1 summarizes properties of some methodsfor solving Ax = b, where A is an n x n matrix and b is an arbitrary n-vector.

Normalization: maxi,j lAijI= 1.Notation: f(n) = (2* 3 1/2 * 41/3 .*. n l(n/- 1))1/2 < 2n(l/4)lognf(100) - 330 (Wilkinson [15, p. 285]).In the rest of the article, the notation - n'will denote that lower order terms have been omitted. If x is an n-vector and A is ann x nmatrix, hen the max-norms fx andA aredefinedby X11 = max1


3/18

640 J. R. BUNCH AND B. N. PARLETT

iAll E~

< -1Cr 1m l l ~~~~~~~~~~~~~ll

z l 0 SVt2 v

,, 5 ; =,, | f | E - -. e X~~~~~~~V1I~

2~~~~~~~~~~~~~~~~~~~ -X=icoeJ>1). A symmetric matrix A is equilibrated if maxj IAijI= 1for each row index i. Parlett conjectured that the partial diagonal pivoting strategywould be stable when applied to equilibrated matrices.4. The decomposition for diagonal pivoting.4.1. Definitions. Let A be an n x n symmetric nonsingular matrix. We wantto decompose A into the "diagonal" form MDMt by congruences, where D is ablock diagonal matrix, each block being of order 1 or 2, and M is unit lowertriangular with Mi+,ni = 0 if Di+li = 0.Let 0 = maxiJAJ, -= maxilAiijand v- A11A22 - A2Let

A=[ X ] ,where P isj x j, C is (n - j) x j, B is (n - j) x (n - j), andj = 1 or 2.Let A(n)= A; let A(k) be the reduced matrix of order k in the eliminationprocess.

4.2. The 1 x 1pivot. Suppose P is of order 1. (We shall not make a distinctionbetween a matrix P of order 1 and its element, which we shall also call P.)Let us assume that we have already interchanged rows and columns so thatIPI= ml, i.e., P is the maximum diagonal element.If P` exists (i.e., pu1= 0), then A(n-1) B - CP-'C'. Since IP1= p, and,o = max, IAjI, we have the following lemma.


12/18

SYMMETRIC NDEFINITESYSTEMSOF LINEAR EQUATIONS 649LEMMA1. If P is of order 1 and IPI= t,u 0 , then

(i) maxI(CP- 1)iI ? [o/pl,(ii) nmaxA8nI- ? (1 + [0/yj)/oIi,Thus a 1 x 1 pivot P is useful if and only if [PI-,1 is large relative to g,ui.e., if and only if ,uJ,o is bounded away from zero.4.3. The 2 x 2 pivot. Suppose P is of order 2 and P-' exists (i.e., v # 0).

Then CP- is (n -2) x 2 and the (k - 2)nd row of CP1 is(Akl , Ak2)P = 1 - 2 (AkjA22-Ak2A2l,Ak2Alj - AkjA2).A1A2- A21(A12

Then An-2) = B - CP-1Ct, but Aln-1) does not exist.Since v = A11A22 - A 1, lAklt and 4Ak21 ito, and JA11j, JA22jwe have the following lemma.LEMMA . If P is of order 2 and IdetPI = v = 0, then(i) maxi,q I(CP- 1)iql< [LO(/LO,I1)/v,for q = 1, 2,(ii) maxi,j IA(ij->2)1 ? [1 + 2[o([o + 1)/v][0.Thus a 2 x 2 pivot is useful if and only if we can bound v away from zero.In particular,from Lemma 1 we need to have v bounded away from zero whenever

p1I/o is near zero.(Note that the use of the standard norm inequalities would give bounds inLemmas 1 and 2 which are too crude for our needs.)4.4. Bounding v. We can easily bound v from above, since v = JA11A22

-A 11 ? JA212 +AIl A22A _? 2 + y4.LEMMA . Idet PI = v _ ,pto+ [4.This upper bound is sharp for A defined by: A1l = p1, Aii =-p for i > 1;

Ai+[1,i = o = Ai,i+1, Aij = 0 otherwise; where 0 < [1i [A2 - 12. Assuming this lower bound, we have the following lemma.

LEMMA4. If |det PI = v > [12 - 2 > 0, then(i) maxj,qICP- 1)iql_ [o/(to p- 11) or q = 1, 2,(ii) maxi,j IA(y2)1?_[o[1 + 2/(1 - p1/po)].The lower bound on v is sharp for A defined by: Aii = [1l, Ai+1,j = [oAi,i+1, Aij = 0 otherwise; where 0 [p1 < [o.


13/18

650 J. R. BUNCH AND B. N. PARLETT

4.5. Criteria for choosing a 1 x 1 or 2 x 2 pivot. Define A(n) = A, p(On) ,uO,(n) (n) = V.Let A(k) be the reduced matrix of order k. Let ,u(k) = maxijJA(.1, p(bk)

- maxj JA..)J, and v(k) iA(k)A(k) - Agj. Note that if A(k) uses a 2 x 2 pivotthen A( k- 1) 1(k-1) I(k-1) and v(k 1)do not exist.All considerations in ?? 4.1-4.4 hold for A(k).If we made our criterion to be the minimization of the number of multiplica-tions (comparisons), then we would want a 1 x 1 (2 x 2) pivot at each step. Butthis would be unstable.Instead, let us aim to minimize the element growth that can take place in thetransformation from one reduced matrix to the next.Let F5k)be the growth factor permitted by choosing a j x j pivot for A(k),where j = 1 or 2.If the hypothesis of Lemma 4 holds (i.e., v(k) > 1(k)2 -_ (k)2 whenever A(k)uses a 2 x 2 pivot), then, by Lemmas 1 and 4,

F(k) = 1 + p(k)/I(k'), F(k) = 1 + 2/(1 - I(k)/I(k))Thus, we are led to the following strategy, Sa, 0 < OC 1: for each reducedmatrix A(k), choose a 1 x 1 pivot if and only if t(lk)y(ok) > a (and a 2 x 2 pivototherwise).With Sa we have F k) ? 1 + 1/o and F k) ? 1 + 2/(1 - oc) for all A(k). Notethat these bounds are sharp.But at any stage the choice of a 2 x 2 pivot carries us further towards thecomplete reduction than does the choice of a 1 x 1 pivot. We would like to com-pare F k+1) F(k) with F k), but in an a priori analysis we must be content with com-paring bounds on these quantities.Let G(a) = max {(1 + 1/a)2, 1 + 2/(1 - oc)}. Thus we seek mino<


14/18

SYMMETRIC INDEFINITE SYSTEMS OF LINEAR EQUATIONS 651The strategy Sa allows us to proceed in the following order:(i) calculate l(k) and l(k) (nomultiplications, (k + 1)/2comparisons);(ii) if = oXt40, hen we use a 1 x 1 pivot;(iii) otherwise we find a 2 x 2 by some strategy.Thus,we search or a 2 x 2 forA(k) if and only if l(k) < O,(k)5. The complete and partial pivotingstrategies.5.1. Complete pivoting. Let us consider the complete pivoting strategy("complete" in the sense that we search over all the principal 2 x 2 minors) sug-gested by W. Kahan (? 3.3).Let vc= maxijJA A Aj-I.The complete strategy involves:(i) finding ,u1 = maxi lAiij,1k = maxi,j lAijl;(ii) choosing a 1 x 1 or 2 x 2 pivot according to Sa(?4.5);(iii) for a 1 x 1, interchanging so that IP1= Ml(?4.2);(iv) for a 2 x 2, finding vc, and interchanging so that IdetPI = vc(?4.3).This would be repeated for each reduced matrix. Actually this process isslightly faster than the one originally considered by Kahan.The result of all this work is that we do obtain a lower bound for vcin termsof Mo nd 1,.

THEOREM 1. /0 - 12 < VC- /2 + ft2Proof. The upper bound follows from Lemma 3 of ?4.4. Let Mo= IArsI.ThenVC maxij lAiiA- Ajl ArrAss A I = ,o- ArrAss? p2 82 since Mi=maxi IAiil.Thus Lemma 4 in ?4.4 holds for vcwhen ,u1 0(usually we normalize by taking Mo= 1).Let vp = maxi IA11Aj - A2ilThe partial strategy involves:(i) equilibrating A (thus we know Mo);(ii) finding M, and choosing a 1 x 1 or 2 x 2 according to Sa ? 4.5);(iii) for a 1 x 1, interchanging so that IP1= M, (? 4.2);(iv) for a 2 x 2, finding vp,and interchanging so that IdetPI = vp(?4.3).Now let us find a lower bound for vp.


15/18

652 J. R. BUNCH AND B. N. PARLETTTHEOREM 2. If A is equilibrated (maxj IAijI= /lo for every i), then /l - y'

< V < , + 14.Proof. The upper bound follows from Lemma 3 of ?4.4. By equilibration,either (i) 1A111 1,k)or (ii) there exists an integer k ? 2 with lAkll = 1o. If(i), then/1o= [, and, trivially, vp > 2 -_2 = 0. If (ii), then vp > IlAkk -A211= -AllAkk _4 _142.This completes the proof.Thus, Lemma 4 in ? 4.4 holds for vp f ,u1 < ,o. According to Sa0,we choose a1 x 1 pivot for the equilibrated reduced matrix A(k) of order k if and only if

(k) ? got(- ).For A= A n), only 2(n -1) multiplications, n-1 additions, and n-1comparisons are required to calculate v(n), in contrast to n(n - 1) multiplications,n(n - 1)/2 additions, and n(n - 1)/2 comparisons to calculate v(n") n ? 5.1.5.3. Criticism of the partial pivoting strategy. The drawback to this methodand the criteria which we have found for the pivoting strategy is that the matrixmust be equilibrated at the beginning and then each reduced matrix should beequilibrated. But a fast algorithm for equilibrating symmetric matrices in themax-norm has never been exhibited, i.e., we seek a diagonal matrix D such thatDAD is equilibrated.However, Bunch [4] presents an algorithm which can equilibrate any sym-metric matrix in a very simple way. In ? 6, we shall exhibit another version of

diagonal pivoting which is applicable to unequilibrated matrices; we call thisunequilibrated diagonal pivoting. Equilibration is unnecessary for this strategyand this is the algorithm that we recommend.6. Unequilibrateddiagonal pivoting.6.1. Maximal off-diagonal element. In order to obtain a lower bound of

-2_2 for vpin Theorem 2 in ? 5.2, we needed the fact that, due to equilibration,there existed an element of maximal absolute value in the first column. However,if ,u < [to then there exist integers i, j with i > j, such that IAijI= ,uo.We needonly bring the element Aijup to the (2, 1) position and then we will have a 2 x 2pivot with a maximal off-diagonal element. We shall call this variation unequili-brated diagonal pivoting.Let pi = maxi Aiij = lAkkl, where k is the least such integer. Let ,o- maxij AijI= JArsI,where, in order to be specific, we take r to be the least suchrow integer, and s to be the least such column integer in the rth row. Let Vb

-AIrAss - A2I.Arr S-rslThis strategy involves:(i) finding p, and the least integer k with IAkkl Pl(ii) finding /lo and the least row integer r and then the least column integer sin the rth row such that lArsl = o0;(iii) choosing a 1 x 1 or 2 x 2 pivot according to Sa (?4.6);(iv) for a 1 x 1, interchanging row and column 1 with k so that IPI=,u(?4.2);(v) for a 2 x 2, interchanging rows and columns 1 with r and 2 with s sothat IdetPI = Vband 1A211= Mo ? 4.3).This procedure is repeated for each reduced matrix.


16/18

SYMMETRIC INDEFINITE SYSTEMS OF LINEAR EQUATIONS 653Note that calculating Vbrequiresonly 2 multiplications instead of n(n - 1) for

vCand 2(n - 1) for vp.Clearly, from the definitions of Vb, vp,and vcand from Lemma 3 of ? 4.4, wehave the following lemma.LEMMA 7. Vb?_VP,p _H + 81

6.2. BoundingVb. Let us now bound Vb from below.THEOREM3. IfIA21I = o,u then y- _,u12?y2 ? Ho+ 12.Proof. The upper bound follows from Lemma 7.Since 1A2 1 = to, vb = A11A22,AA 1l =-2 2 A > -2Here, as in ? 5.1 and ? 5.2, symmetry was used to get the lower bound on Vb.By symmetry, if g0 = 1A211,hen IA121=o.If A were not symmetric, then /t = IA211would not imply that 1A121 PO(in fact we could have A12 = 0). Thus, no such lower bound on the absolutevalue of the determinant of 2 x 2 submatricescan exist fornonsymmetric matrices.Thus, Theorem 3 implies that Lemma 4 in ?4.4 holds for Vb if 1


17/18

654 J. R. BUNCH AND B. N. PARLETTwhere 0 < ax< 1 and 0 < 3)denote summation over those indices i, 1 < i < n, such that, if A(')exists then it yields a j x j pivot, j = 1 or 2. Let p be the number of 1 x 1 pivotsused. Let Mults (Divs, Adds, Comps) mean the number of multiplications(divisions, additions, comparisons).Then the requirements of solving Ax = b appear in Table 2.

TABLE 2

Exact Upper bound

Mults kn3 +n 2 _ n?P+31(2)i 13 + n2-Divs 2-2 2n2Adds n3 + n2 _ 6n + p + n3 + n2 _ n + 4Comps n3 + 3n2 +?n + 8P +p (1)i2 n3 + n2 + ln

Recall that for Gaussian elimination with partial or complete pivoting thenumber of comparisons required is n2/2 - n/2 and n3/3 + n2/2 - 5n/6, respec-tively (Table 1).Backward error analysis reveals that the stability of this method depends onthe bounding of the growth of the elements of the reduced matrices (Bunch [3]).Thus we would like to bound B(n) = maxA (maxk maxi,j JAj)I/maxj,j IAijl),wherethe A are n x n nonsingular symmetricmatrices. Further,we would like the boundto be independent of ay,0 < cx< 1. But, as we recall from ?4.5, ac+ 0 (cx? 1)corresponds to the use of a 1 x 1 (2 x 2) pivot at each step, and either situation isunstable. Thus B(n) -+ oo as a -+* 0, 1.We saw in ?4.5 that maxi,j jAY9j (2.57)n k maxi,j AijI(i.e., B(n) < (2.57)n)-1for ox= cxo= (1 + /17)/8. But, by the use of Wilkinson's techniques for Gaussianelimination with complete pivoting (Wilkinson [15, pp. 281-285]), Bunch [3]obtains B(n) < /nf (n)c(cx)h(n,x),where

nf -(n)2 = H[r 1(r- 1) (recall ?1.2),

r = 2

c(cx)= ijmax {x- 1, (1 + c2)/(1-S1 if the last reduced matrix is 1 x 1,(1 -( x2)- 1/2 if the last reduced matrix is 2 x 2,

nh(n, x)2 = rl fll(r-1)r= 2


18/18

SYMMETRIC NDEFINITE SYSTEMSOF LINEAR EQUATIONS 655(a-2 if A(r) yields a 1 x 1pivot,

fr = (1-2)-l if A(r) yields a 2 x 2 pivot,if A(r+1) yieldsa2 x 2 pivot.

This bound on B(n) holds for each a, 0 < a < 1, but h(n,ca) oo as ax* 0 whenp = n (e.g., when A is positive definite) and as a -? 1 when p = 0 (? 5.1). Further-more, this is an a posteriori bound, since we cannot calculate c(a)h(n,aL)until weknow the position of the blocks of order 1 and 2 in the decomposition.Wilkinson obtained the factor lnf(n) maxi JIAijlas the bound on the ele-ments in the reduced matrices for solving Ax = b by Gaussian elimination withcomplete pivoting. We have the extra factor c(a)h(n,a) since our pivots are notnecessarily the maximal elements in the reduced matrices.We would like an a priori bound on the element growth for the diagonalpivoting method, i.e., a bound independent of the selection of a 1 x 1or a 2 x 2pivot at each stage.Bunch [3] shows that

inf c(ac)h(n,) > (2.029)n?3465,0

Date post:	06-Apr-2018
Category:	Documents
Upload:	salmansalsabila
View:	217 times
Download:	0 times

Indefinite Symmetric System

Documents