Journal of Computational Mathematics

Vol.34, No.3, 2016, 300–316.

http://www.global-sci.org/jcm

doi:10.4208/jcm.1511-m2015-0299

POSITIVE DEFINITE AND SEMI-DEFINITE SPLITTINGMETHODS FOR NON-HERMITIAN POSITIVE DEFINITE

LINEAR SYSTEMS*

Na Huang and Changfeng Ma

School of Mathematics and Computer Science, Fujian Normal University, Fuzhou 350117, China

Email: macf@fjnu.edu.cn

Abstract

In this paper, we further generalize the technique for constructing the normal (or pos-

itive definite) and skew-Hermitian splitting iteration method for solving large sparse non-

Hermitian positive definite system of linear equations. By introducing a new splitting, we

establish a class of efficient iteration methods, called positive definite and semi-definite

splitting (PPS) methods, and prove that the sequence produced by the PPS method con-

verges unconditionally to the unique solution of the system. Moreover, we propose two

kinds of typical practical choices of the PPS method and study the upper bound of the

spectral radius of the iteration matrix. In addition, we show the optimal parameters such

that the spectral radius achieves the minimum under certain conditions. Finally, some

numerical examples are given to demonstrate the effectiveness of the considered methods.

Mathematics subject classification: 65F10, 65F30, 65F50.

Key words: Linear systems, Splitting method, Non-Hermitian matrix, Positive definite

matrix, Positive semi-definite matrix, Convergence analysis.

1. Introduction

Many problems in scientific computing give rise to a system of linear equations

Ax = b, A ∈ Cn×n, and x, b ∈ Cn, (1.1)

with A being a large sparse non-Hermitian but positive definite matrix.

We call a matrix B positive definite (or positive semi-definite), if B + B∗ is Hermitian

positive definite (or positive semi-definite), i.e., for all 0 = x ∈ Cn, x∗(B + B∗)x > 0 (or

x∗(B + B∗)x ≥ 0), where B∗ denotes the complex conjugate transpose of the B. Let D =

diag(a11, a22, · · · , ann) be the diagonal part of A and ei = (0, · · · , 0, 1, 0, · · · , 0)T . Since the

coefficient matrix A is positive definite, we have e∗i (A+A∗)ei = aii + aii > 0. This shows that

D is positive definite.

The linear system has many important practical applications, such as diffuse optical tomog-

raphy, molecular scattering, lattice quantum chromodynamics (see, e.g., [1,7,8,22,24,38,39,41]).

Many researchers have been devoted themselves to the numerical solution of (1.1) (see e.g.,

[2–4,10,11,18,21,25,27,28,36,37,40,42,45–47] and the references therein) and proposed kinds

of available iteration methods for solving the system (1.1), in which splitting iteration methods

(see e.g., [9,13–17,19,29–31,35,44]) and Krylov subspace methods (see e.g., [5,20,23,26,32,43])

* Received May 25, 2015 / Revised version received October 13, 2015 / Accepted November 4, 2015 /

Published online May 3, 2016 /

Splitting Methods for Non-Hermitian Positive Definite Linear Systems 301

attract a lot of attention. In [16], the authors presented a Hermitian and skew-Hermitian s-

plitting (HSS) iteration method for solving (1.1) and showed that the HSS method converges

unconditionally to the unique solution of the system. Then many researchers focused on the

HSS method and proposed kinds of iterations method based on Hermitian and skew-Hermitian

splitting (see e.g., [6, 9, 12, 13, 17]). In recent years, other kinds of splitting iteration methods

have also been studied (see e.g., [14,15,33–35,44]). Normal and skew-Hermitian splitting (NSS)

iteration methods for solving large sparse non-Hermitian positive definite linear system was

studied in [15]. Based on block triangular and skew-Hermitian splitting, a class of iteration

methods for solving positive-definite linear systems was established in [14]. Krukier et. al pro-

posed the generalized skew-Hermitian triangular splitting iteration methods to solve (1.1) and

applied the methods to solve the saddle-point linear systems (see [34]). In this work, we further

generalize the technique for constructing the normal (or positive definite) and skew-Hermitian

splitting iteration method for solving (1.1).

Throughout this paper, we use the following notations: Cm×n is the set of m× n complex

matrices and Cm = Cm×1. We use C and R to denote the set of complex numbers and real

numbers, respectively. For any a ∈ C, we write Re(a) and Im(a) to denote the real and

imaginary parts of a. For B ∈ Cn×n, we write B−1, ∥B∥2, Λ(B) and ρ(B) to denote the the

inverse, 2-norm, the spectrum and the spectral radius of the matrix B, respectively. I denotes

the identity matrix of size implied by context. i =√−1 denotes the imaginary unit.

The organization of this paper is as follows. In Section 2, we present the positive definite

and semi-definite splitting methods for solving non-hermitian positive definite linear systems

and study the convergence properties of the PPS iteration. In Section 3, we establish two kinds

of typical practical choices of the PPS method and study the upper bound of the spectral radius

of iteration matrix. Numerical experiments are presented in Section 4 to show the effectiveness

of our methods.

2. The Positive Definite and Semi-definite Splitting Method

In this section, we study efficient iterative methods for solving (1.1) based on the positive

definite and semi-definite splitting (PPS for short) of the coefficient matrix A, and establish

the convergence analysis of the new methods. For this purpose, we split A into positive-definite

and positive semi-definite parts as follows:

A =M +N, (2.1)

where M is a positive-definite matrix and N is a positive semi-definite matrix. Then it is easy

to see that

A = αI +M − (αI −N) = αI +N − (αI −M).

This implies that the system (1.1) can be reformulated equivalently as:

(αI +M)x = (αI −N)x+ b,

or

(αI +N)x = (αI −M)x+ b.

By the two above fixed point equations, we can get the following iteration method:(αI +M)x(k+

12 ) = (αI −N)x(k) + b,

(αI +N)x(k+1) = (αI −M)x(k+12 ) + b.

(2.2)

302 N. HUANG AND C.F. MA

For convenience, we call the scheme (2.2) positive-definite and semi-definite splitting (PPS for

short) iteration method. It is worth mentioning that the normal and skew-Hermitian splitting

iteration method proposed in [15] and the positive definite and skew-Hermitian splitting itera-

tion method proposed in [14] are both the special cases of the PPS iteration method. Actually,

if A =M +N with N being a skew-Hermitian matrix, then we can see that

M +M∗ = A−N + (A−N)∗ = A+A∗,

which shows that M is positive definite. This, along with the fact that all the skew-Hermitian

matrix are positive semi-definite, follows that both of the two splitting mentioned above belong

to the positive-definite and semi-definite splitting.

In the following, we shall establish the convergence theorem of the PPS method. To this

end, we rewrite the scheme (2.2) equivalently. By using (2.2), we can obtain

x(k+12 ) = (αI +M)−1(αI −N)x(k) + (αI +M)−1b, (2.3)

and

x(k+1) = (αI +N)−1(αI −M)x(k+12 ) + (αI +N)−1b, (2.4)

which follows that

x(k+1) = S(α)x(k) + T (α)b, (2.5)

where

S(α) = (αI +N)−1(αI −M)(αI +M)−1(αI −N), (2.6a)

T (α) = 2α[(αI +M)(αI +N)]−1. (2.6b)

Therefore, the PPS method is convergent if and only if the spectral radius of the iterative

matrix S(α) is less than 1. To this purpose, we present the following lemma.

Lemma 2.1. For any positive semi-definite matrix P ∈ Cn×n, it holds that

∥(αI − P )(αI + P )−1∥2 ≤ 1, ∀α > 0.

Furthermore, if P is positive definite, then it holds that

∥(αI − P )(αI + P )−1∥2 < 1, ∀α > 0.

Proof. Let σ(α) =: ∥(αI − P )(αI + P )−1∥2. By the definition of 2-norm and the similarity

invariance of the matrix spectrum, we have

σ(α)2 = ∥(αI − P )(αI + P )−1∥22= ρ

((αI + P ∗)−1(αI − P ∗)(αI − P )(αI + P )−1

)= ρ

((αI − P ∗)(αI − P )(αI + P )−1(αI + P ∗)−1

).

Note that the matrix (αI + P ∗)−1(αI − P ∗)(αI − P )(αI + P )−1 is Hermitian positive semi-

definite. It follows that all the eigenvalues of (αI − P ∗)(αI − P )(αI + P )−1(αI + P ∗)−1 are

Splitting Methods for Non-Hermitian Positive Definite Linear Systems 303

nonnegative. This, together with the fact

(αI − P ∗)(αI − P )(αI + P )−1(αI + P ∗)−1

= [α2I + P ∗P − α(P + P ∗)][α2I + P ∗P + α(P + P ∗)]−1

= [α2I + P ∗P + α(P + P ∗)− 2α(P + P ∗)][α2I + P ∗P + α(P + P ∗)]−1

= I − 2α(P + P ∗)[α2I + P ∗P + α(P + P ∗)]−1

= I − 2α(P + P ∗)(αI + P )−1(αI + P ∗)−1

=: I − L(α),

yields that all the eigenvalues of the matrix I − L(α) are nonnegative and

σ(α)2 = ρ(I − L(α)). (2.7)

For any λ ∈ Λ(L(α)), it is easy to see that 1 − λ ∈ Λ(I − L(α)), which shows that 1 − λ ≥ 0.

Then for any positive semi-definite matrix P and α > 0, we can know that 2α(P + P ∗) is

Hermitian positive semi-definite. This, along with the fact that L(α) is similar to 2α(αI +

P ∗)−1(P + P ∗)(αI + P )−1, leads to λ ≥ 0. Then we have

0 ≤ 1− λ ≤ 1.

Combining this with (2.7), it leads to

σ(α)2 = maxλ∈Λ(L(α))

1− λ ≤ 1.

Furthermore, if P is positive definite, then it is easy to see that 2α(αI+P ∗)−1(P+P ∗)(αI+P )−1

is Hermitian positive definite, which shows that λ > 0. At this case, we have 0 ≤ 1− λ < 1. It

leads to σ(α)2 < 1 immediately. Then the result follows.

Now we establish the convergence theorem for the PPS iteration method.

Theorem 2.1. Let A = M + N be a positive definite and semi-definite splitting of A. Then

for any positive constant α, the spectral radius ρ(S(α)) of the iteration matrix S(α) satisfied

ρ(S(α)) < 1, where S(α) is defined as in (2.6a). Therefore, the PPS iteration method is

convergent to the exact solution x∗ ∈ Cn of the linear system (1.1).

Proof. Note that N is a positive semi-definite matrix and M is a positive definite matrix.

Then by using Lemma 2.1, we can get

ρ(S(α)) = ρ((αI +N)−1(αI −M)(αI +M)−1(αI −N))

= ρ((αI −M)(αI +M)−1(αI −N)(αI +N)−1)

≤ ∥(αI −M)(αI +M)−1∥2∥(αI −N)(αI +N)−1∥2≤ ∥(αI −M)(αI +M)−1∥2 < 1,

which completes the proof. We shall emphasize that the scheme (2.5) can induce a class of preconditioners for solving

the system (1.1). Actually, let

R(α) =1

2α(αI −M)(αI −N),

304 N. HUANG AND C.F. MA

where M, N are defined as in (2.1). Then by using the definition of T (α), we can know that

A = T (α)−1 −R(α) (2.8)

defines a new splitting of the coefficient matrix A in (1.1). It is easy to see that the PPS iteration

method can also be induced by the matrix splitting (2.8). Moreover, the matrix T (α)−1 can be

alternatively used to precondition the Krylov subspace iteration methods for solving the system

(1.1).

3. Two Typical Practical Choices of the PP-splitting

In this section, we give two kinds of typical practical choices of the PPS method. One is

based on Hermitian and skew-Hermitian splitting, the other one is based on triangular splitting.

To this end, let

H(A) =1

2(A+A∗), S(A) =

1

2(A−A∗).

Then H(A), S(A) are the Hermitian and skew-Hermitian parts of the matrix A, respectively.

Let η be any real constant, let

M = H(A) + iηI, N = S(A)− iηI. (3.1)

It is easy to verify that M +N = A, M +M∗ = H(A) + iηI +H(A)− iηI = 2H(A) is positive

definite, and N +N∗ = S(A)− iηI − S(A) + iηI = 0 is positive semi-definite. Thereby, we can

get the following PPS iteration method: (αI +H(A) + iηI)x(k+12 ) = (αI − S(A) + iηI)x(k) + b,

(αI + S(A)− iηI)x(k+1) = (αI −H(A)− iηI)x(k+12 ) + b.

(3.2)

If η = 0, the iteration scheme (3.2) reduces to the Hermitian and skew-Hermitian splitting

(HSS) iteration method proposed in [16]. This implies that the HSS iteration method is a

special case of the PPS iteration method.

In the following, we shall establish the convergence analysis of (3.2). For this purpose, we

introduce the quantities

V (α, η) = (αI −H(A)− iηI)(αI +H(A) + iηI)−1,

W (α, η) = (αI − S(A) + iηI)(αI + S(A)− iηI)−1,

S(α, η) = (αI + S(A)− iηI)−1V (α, η)(αI − S(A) + iηI), (3.3)

T (α, η) = 2α[(αI +H(A) + iηI)(αI + S(A)− iηI)]−1. (3.4)

Then it is easy to verify that the iteration scheme (3.2) can be equivalent to

x(k+1) = S(α, η)x(k) + T (α, η)b. (3.5)

This shows that the method (3.2) is convergent if and only if the spectral radius of the iterative

matrix S(α, η) is less than 1.

Theorem 3.1. Let A ∈ Cn×n be a positive definite matrix, M and N be defined as in (3.1),

λ1 and λn be the maximum and minimum eigenvalues of H(A). Then for any η ∈ R and α > 0,

Splitting Methods for Non-Hermitian Positive Definite Linear Systems 305

(1) the sequence generated by the iteration scheme (3.2) converges to the exact solution x∗ ∈Cn of the system (1.1);

(2) the spectral radius ρ(S(α, η)) of the iteration matrix S(α, η) is bounded by

ρ(S(α, η)) ≤√1− φ(α, η), (3.6a)

where

φ(α, η) = min 4αλ1(α+ λ1)2 + η2

,4αλn

(α+ λn)2 + η2

. (3.6b)

Proof. Noticing that A and M are positive definite, and N is positive semi-definite, then by

Theorem 2.1, we can get the first result. Now we estimate the bound of ρ(S(α, ω)).

It is easy to see that (αI − S(A) + iηI)−1S(A) = S(A)(αI − S(A) + iηI)−1. Then we can

know that

W (α, η)∗W (α, η)

= (αI − S(A) + iηI)−1(αI + S(A)− iηI)(αI − S(A) + iηI)(αI + S(A)− iηI)−1

= (αI + S(A)− iηI)(αI − S(A) + iηI)−1(αI − S(A) + iηI)(αI + S(A)− iηI)−1

= I.

Similarly, we can get W (α, η)W (α, η)∗ = I. This shows that W (α, η) is a unitary matrix and

∥W (α, η)∥2 = 1. This, along with the fact that S(α, η) is similar to V (α, η)W (α, η), leads to

ρ(S(α, η)) = ρ(V (α, η)W (α, η)) ≤ ∥V (α, η)∥2∥W (α, η)∥2 = ∥V (α, η)∥2. (3.7)

On the other hand, by Rayleigh-Ritz theorem, it follows that

∥V (α, η)∥22 = ρ(V (α, η)∗V (α, η))

= ρ((αI +H(A)− iηI)−1(αI −H(A) + iηI)(αI −H(A)− iηI)(αI +H(A) + iηI)−1

)= max

λ∈Λ(H(A))

(α− λ+ iη)(α− λ− iη)

(α+ λ− iη)(α+ λ+ iη)= max

λ∈Λ(H(A))

(α− λ)2 + η2

(α+ λ)2 + η2. (3.8)

Noticing that for any λ ∈ Λ(H(A)), we can get

(α− λ)2 + η2

(α+ λ)2 + η2=α2 − 2λα+ λ2 + η2

α2 + 2λα+ λ2 + η2=α2 + 2λα+ λ2 + η2 − 4λα

α2 + 2λα+ λ2 + η2

= 1− 4λα

α2 + 2λα+ λ2 + η2= 1− 4α

(α2 + η2)/λ+ λ+ 2α. (3.9)

Since α > 0 and α2 + η2 > 0, we have

maxλ∈Λ(H(A))

α2 + η2

λ+ λ = max

α2 + η2

λ1+ λ1,

α2 + η2

λn+ λn

.

306 N. HUANG AND C.F. MA

This, together with α > 0, (3.8) and (3.9), yields that

∥V (α, η)∥22 = maxλ∈Λ(H(A))

[1− 4α

(α2 + η2)/λ+ λ+ 2α

]= 1− min

λ∈Λ(H(A))

4α

(α2 + η2)/λ+ λ+ 2α

= 1−min 4α

(α2 + η2)/λ1 + λ1 + 2α,

4α

(α2 + η2)/λn + λn + 2α

= 1−min

4αλ1(α+ λ1)2 + η2

,4αλn

(α+ λn)2 + η2

.

Then the second result follows.

From the above theorem, we can see that the optimal point (α∗, η∗) of ρ(S(α, η)) make

φ(α, η) maximum. This implies that we just need to solve the following optimization problem

maxα>0,η∈R

φ(α, η)

if we need to find the optimal parameters.

Theorem 3.2. Under the same settings and conditions as in Theorem 3.1, for any η ∈ R and

α > 0, it holds that

(α∗, η∗) ≡ arg maxα>0,η∈R

φ(α, η) = (√λ1λn, 0), (3.10)

φ(α∗, η∗) =4√λ1λn

(√λ1 +

√λn)2

. (3.11)

Proof. From (3.6b), we can see that the function φ(α, η) decreases monotonically with

respect to η2 for any α > 0. This shows that

η∗ ≡ argmaxα>0

φ(α, η) = 0, (3.12a)

φ(α, 0) = min 4αλ1(α+ λ1)2

,4αλn

(α+ λn)2

. (3.12b)

In order to find the maximum point of φ(α, 0), in the following, we derive α∗ such that minimize

the function 1/φ(α, 0). It follows from (3.12b) that

1

φ(α, 0)= max

(α+ λ1)2

4αλ1,(α+ λn)

2

4αλn

= max

α

4λ1+λ14α

+1

2,α

4λn+λn4α

+1

2

. (3.13)

Since H(A) is Hermitian positive definite, we have λ1 > 0 and λn > 0. Then it is easy to

see that α∗ =√λ1λn. This, along with the independence of α and η, leads to (3.10) which

immediately yields (3.11). This completes the proof.

Remark 3.1. We shall emphasize that (√λ1λn, 0) is only the minimum point of

√1− φ(α, η),

rather than that of ρ(S(α, η)). Therefore, from Theorem 3.3, we can see that the optimal point

(α∗, η∗) is the quasi-optimal parameter of ρ(S(α, η)), and the quasi-optimal spectral radius is

ρ(S(α∗, η∗)) ≤√1− φ(α∗, η∗) =

√λ1 −

√λn√

λ1 +√λn. (3.14)

Splitting Methods for Non-Hermitian Positive Definite Linear Systems 307

In the following, we shall study another typical practical choice of the PPS method. Let

A = D + L+ U, (3.15)

where D, L and U are the diagonal, the strictly lower-triangular, and the strictly upper-

triangular matrices of the matrix A. For any ω ∈ R, let

M = D + L+ U∗ + iωI, N = U − U∗ − iωI, (3.16a)

M = D + L∗ + U + iωI, N = L− L∗ − iωI, (3.16b)

it is easy to verify that A =M +N . Noticing that

M +M∗ = D + L+ U∗ + iωI +D∗ + L∗ + U − iωI = A+A∗, (3.17a)

M +M∗ = D + L∗ + U + iωI +D∗ + L+ U∗ − iωI = A+A∗, (3.17b)

we can know that the matrix M defined above is positive definite. On the other hand, from the

fact N +N∗ = 0, it follows that N is positive semi-definite. Then we can get the PPS iteration

method as follows: (αI +D + L+ U∗ + iωI)x(k+12 ) = (αI − U + U∗ + iωI)x(k) + b,

(αI + U − U∗ − iωI)x(k+1) = (αI −D − L− U∗ − iωI)x(k+12 ) + b,

(3.18)

or (αI +D + L∗ + U + iωI)x(k+12 ) = (αI − L+ L∗ + iωI)x(k) + b,

(αI + L− L∗ − iωI)x(k+1) = (αI −D − L∗ − U − iωI)x(k+12 ) + b.

(3.19)

Particularly, if ω = 0, then the above iteration formulas reduce to the triangular and skew-

Hermitian splitting splitting (TSS) iteration method proposed in [14]. This shows that the TSS

iteration method is a special case of the PPS iteration method. In addition, it is easy to see

that the iterative formula (3.18) and (3.19) have same structure and property. Hence we just

need to analyze the convergence of (3.18) in the following. Let

V (α, ω) = (αI −D − L− U∗ − iωI)(αI +D + L+ U∗ + iωI)−1,

W (α, ω) = (αI − U + U∗ + iωI)(αI + U − U∗ − iωI)−1,

S(α, ω) = (αI + U − U∗ − iωI)−1V (α, ω)(αI − U + U∗ + iωI), (3.20)

T (α, ω) = 2α[(αI +D + L+ U∗ + iωI)(αI + U − U∗ − iωI)]−1. (3.21)

Then the iteration scheme (3.18) can be reformulated equivalently as

x(k+1) = S(α, ω)x(k) + T (α, ω)b. (3.22)

Firstly, we propose the following lemmas.

Lemma 3.1. Let D, U, L be defined as in (3.15), and let t1 = α− iω, t2 = α+iω, Q = L+U∗.

Then it holds that

V (α, ω) = (t1I −D)(t2I +D)−1 + 2α(t2I +D)−1n−1∑j=1

(−1)j [Q(t2I +D)−1]j . (3.23)

308 N. HUANG AND C.F. MA

Proof. By the definition of t1, t2 and Q, we can get

V (α, ω) = (t1I −D −Q)(t2I +D +Q)−1. (3.24)

It is easy to verify that t2I +D +Q = [I +Q(t2I +D)−1](t2I +D). Consequently,

(t2I +D +Q)−1 = (t2I +D)−1[I +Q(t2I +D)−1]−1

= (t2I +D)−1∞∑j=0

(−1)j [Q(t2I +D)−1]j .

Noticing thatQ(t2I+D)−1 is a strictly lower triangular matrix, it follows that all the eigenvalues

of Q(t2I + D)−1 are 0. Thus its characteristic polynomial is f(λ) = λn. This, together with

the Hamilton-Cayley theorem, yields [Q(t2I +D)−1]n = 0. Then we have

(t2I +D +Q)−1 = (t2I +D)−1n−1∑j=0

(−1)j [Q(t2I +D)−1]j . (3.25)

Using the definition of t1, t2 and (3.24), we can obtain

V (α, ω) = I − [(t2 − t1)I+2(D +Q)](t2I+D +Q)−1 = I − 2(iωI+D +Q)(t2I+D +Q)−1

= I − 2(iωI +D +Q)(t2I +D)−1n−1∑j=0

(−1)j [Q(t2I +D)−1]j

= I − 2(iωI +D)(t2I +D)−1n−1∑j=0

(−1)j [Q(t2I +D)−1]j

− 2Q(t2I +D)−1n−1∑j=0

(−1)j [Q(t2I +D)−1]j

= I − 2(iωI +D)(t2I +D)−1 − 2(iωI +D)(t2I +D)−1n−1∑j=1

(−1)j [Q(t2I +D)−1]j

+ 2

n−1∑j=1

(−1)j [Q(t2I +D)−1]j

= (t1I −D)(t2I +D)−1 + 2α(t2I +D)−1n−1∑j=1

(−1)j [Q(t2I +D)−1]j , (3.26)

where the sixth equality holds by [Q(t2I +D)−1]n = 0. This completes the proof.

By Lemma 3.1, we can take (t1I − D)(t2I + D)−1 as a first-order approximation of the

matrix V (α, ω), then we get

∥V (α, ω)∥2 ≈ ∥(t1I −D)(t2I +D)−1∥2.

Lemma 3.2. For any α > 0 and ω ∈ R, the matrix W (α, ω) is a unitary matrix, which follows

that ∥W (α, ω)∥2 = 1

By the above analysis, we are now in a position to present the following theorem.

Theorem 3.3. Assume A ∈ Cn×n is a positive definite matrix. Let D = diaga11, a22, · · · , ann,L and U be the diagonal, the strictly lower-triangular, and the strictly upper-triangular matrices

of the matrix A. Then for any α > 0 and ω ∈ R,

Splitting Methods for Non-Hermitian Positive Definite Linear Systems 309

(1) the sequence generated by the iteration scheme (3.18) converges to the exact solution

x∗ ∈ Cn of the system (1.1);

(2) the spectral radius ρ(S(α, ω)) of the iteration matrix S(α, ω) is bounded by

ρ(S(α, ω)) ≤ max1≤i≤n

|α− iω − aii||α+ iω + aii|

, (3.27)

approximatively;

(3) If aii, i = 1, · · · , n are all real with amin and amax being the minimum and the maximum

elements of aii, i = 1, · · · , n, respectively, then it holds that

(α∗, ω∗) ≡ arg minα>0,ω∈R

ψ(α, ω) = (√aminamax, 0), (3.28a)

ψ(α∗, ω∗) =

√amax −

√amin√

amax +√amin

, (3.28b)

where

ψ(α, ω) = max1≤i≤n

|α− iω − aii||α+ iω + aii|

. (3.28c)

Proof. As the iteration scheme (3.18) is a special case of the PPS iteration method, from

Theorem 2.1, we get the first result.

Since A is a positive definite matrix, we have Re(aii) > 0, i = 1, · · · , n. It follows from

Lemmas 3.1 and 3.2 that

ρ(S(α, ω)) = ρ(V (α, ω)W (α, ω)) ≤ ∥V (α, ω)∥2∥W (α, ω)∥2≈ ∥(t1I −D)(t2I +D)−1∥2

= max1≤i≤n

|t1 − aii||t2 + aii|

= max1≤i≤n

|α− iω − aii||α+ iω + aii|

< 1, (3.29)

where the last equality holds by the definition of t1 and t2. Then the second result follows.

Furthermore, if aii, i = 1, · · · , n are all real, then aii > 0. This yields that

ψ(α, ω) = max1≤i≤n

|α− iω − aii||α+ iω + aii|

= max1≤i≤n

[ (α− aii)2 + ω2

(α+ aii)2 + ω2

] 12

. (3.30)

For any i = 1, · · · , n, since

(α− aii)2 + ω2

(α+ aii)2 + ω2=

(α+ aii)2 + ω2 − 4αaii

(α+ aii)2 + ω2= 1− 4αaii

(α+ aii)2 + ω2,

we can know that ψ(α, ω) increases monotonically with respect ω2 for any α > 0. Then

ω∗ ≡ argminω∈R

ψ(α, ω) = 0, (3.31)

which immediately follows that

ψ(α, ω∗) = max1≤i≤n

|α− aii||α+ aii|

= max |α− amin||α+ amin|

,|α− amax||α+ amax|

. (3.32)

If α∗ is a minimum point of ψ(α, ω∗), then it must satisfy that α−amin > 0, α−amax < 0, and

α− amin

α+ amin=amax − α

α+ amax.

310 N. HUANG AND C.F. MA

By some simple calculation, we can get

α∗ ≡ argminα>0

ψ(α, ω∗) =√aminamax.

This, along with the independence of α and ω, leads to (3.28a) and (3.28b).

Remark 3.2. We shall emphasize that the condition of the third result on the diagonal ele-

ments is necessary. Otherwise, we can not derive the specific formula of ψ(α, ω). For example,

let

A =

1 + i 0 0

0 2 0

0 0 1− 1

2i

.

Evidently, A is positive definite and a11 = 1 + i, a22 = 2, a33 = 1 − 12 i. Then for α = 1 and

ω = 0, we have

|1− a11||1 + a11|

=1√5,

|1− a22||1 + a22|

=1

3,

|1− a33||1 + a33|

=1√17.

This show that

ψ(1, 0) =|1− a11||1 + a11|

.

However, a11 does not have apparent characteristic point, such as maximum modulus or mini-

mum modulus. Therefore, for general problems, we can not determine the expression of ψ(α, ω).

And then we can not calculate the minimum of ψ(α, ω).

4. Numerical Experiments

In this section, we use two examples to numerically examine feasibility and effectiveness of

our new methods. All our tests are started from zero vector, and terminated when the current

iteration satisfies∥rk∥2∥r0∥2

< 10−5, (4.1)

where rk is the residual of the current, say k-th, iteration. We report the number of required

iterations (IT), the norm of the relative residual (4.1) (Res) when the process is stopped, the

required CPU time (CPU). All algorithms were coded by MATLAB 2011b and were run on a

PC with 2.20 GHz Pentium(R) Dual-Core CPU and 2.00 GB RAM.

Example 4.1. Consider the system of linear equations (1.1), for which

A = tridiag(−1 + i, 10, 1− i), b = (1, 1, · · · , 1)T .

Since A+A∗ = tridiag(2i, 20,−2i), A is non-Hermitian positive definite. We split A intoM+N

with

M = tridiag(0, 5, 1− i), and N = tridiag(−1 + i, 5, 0).

This shows that

M +M∗ = tridiag(1 + i, 10, 1− i), and N +N∗ = tridiag(−1 + i, 10,−1− i).

Splitting Methods for Non-Hermitian Positive Definite Linear Systems 311

Table 4.1: Numerical results of Example 4.1.

HSS TSS PPS

α IT CPU Res IT CPU Res IT CPU Res

3 19 14.0936 4.2169e-6 20 9.6555 4.3508e-6 5 0.9476 1.1832e-7

4 14 10.7445 3.0389e-6 15 8.0475 2.7010e-6 4 0.8178 1.6163e-8

5 11 8.8524 1.8959e-6 12 6.0794 1.5284e-6 3 0.6128 1.5982e-7

6 9 7.1883 9.6551e-7 9 4.9430 2.7557e-6 3 0.6843 1.2306e-7

7 7 5.6605 9.5964e-7 8 4.4507 8.3373e-7 4 0.8672 3.5326e-8

8 6 4.9500 2.2618e-7 7 3.9645 3.5884e-7 4 0.8936 5.0542e-7

9 4 3.4438 5.4925e-7 6 3.1593 3.0152e-7 5 0.9906 3.2121e-7

10 3 2.8982 6.9605e-7 5 2.9549 9.6901e-7 6 1.1582 2.2039e-7

11 4 3.5298 5.3756e-7 6 3.3052 1.5220e-7 6 1.1204 1.1160e-6

12 5 4.2154 6.6345e-7 6 3.1742 5.7696e-7 7 1.2235 6.9808e-7

13 6 4.7301 6.9013e-7 7 3.6064 3.6556e-7 8 1.4491 4.6709e-7

Fig. 4.1. ρ(M(α)) versus α for the PPS, the HSS and the TSS iteration matrices for Example 4.1.

Hence both M and N are positive definite. In our test, we compare the PPS method based on

this splitting with the HSS method and the TSS method. We take the dimension n = 1024.

The test results are listed in Table 4.1. And we depict the curves of ρ(M(α)) with respect to

α for all PPS, HSS and TSS iteration methods to intuitively show this functional relationship

in Figure 4.1.

From Table 4.1, we can see that all the methods stopped regularly and the optimal parameter

αopt of the PPS method is in the interval of [5, 6]. The optimal parameters of the HSS method

and the TSS method are both near by 10. This fact is further confirmed by Figure 4.1. As

shown in Table 4.1, whatever the method, the number of iterations and the actual computing

time vary seriously for the different parameter α. Furthermore, whatever the parameter α we

choose, the CPU time of the PPS method is always the least.

312 N. HUANG AND C.F. MA

Table 4.2: Numerical results of Example 4.2.

HPPS TPPS

η/ω αopt IT CPU Res αopt IT CPU Res

-8 11.7742 73 4.6751 7.7659e-006 11.2158 72 3.1349 8.3495e-006

-6 12.9086 66 4.1304 8.2780e-006 12.4248 65 2.9226 8.1309e-006

-4 13.6613 63 3.9267 7.2506e-006 13.2241 61 2.9510 8.0710e-006

-2 14.0936 61 3.6657 7.2436e-006 13.7097 59 2.4537 7.7419e-006

0 14.2683 60 0.5630 7.5525e-006 13.8493 59 0.5602 6.8548e-006

2 14.0936 61 3.6096 7.2436e-006 13.7097 59 2.6133 7.7419e-006

4 13.6613 63 3.8884 7.2506e-006 13.2241 61 3.0034 8.0710e-006

6 12.9086 66 4.2420 8.2780e-006 12.4248 65 2.9795 8.1309e-006

8 11.7742 73 4.6436 7.7659e-006 11.2158 72 3.1213 8.3495e-006

Example 4.2. (see [14]) Consider the system of linear equations (1.1), for which

A =

(W FΩ

−FT N

),

where W ∈ Rq×q, and N, Ω ∈ R(n−q)×(n−q), with 2q > n, b = Ae, e = (1, · · · , 1)T .In particular, the matrices W , N , F and Ω = diagν1, · · · , νn−q are defined as follows:

wkj =

k + 1, for j = k,

1, for |k − j| = 1,

0, otherwise,

k, j = 1, · · · , q,

nkj =

k + 1, for j = k,

1, for |k − j| = 1,

0, otherwise,

k, j = 1, · · · , n− q,

fkj =

j, for k = j + 2q − n,

0, otherwise,k = 1, · · · , q, j = 1, · · · , n− q,

νk =1

k, k = 1, · · · , n− q.

In this example, we compare the iteration scheme (3.2) (HPPS for short) with the iteration

scheme (3.19) (TPPS for short) with different parameter η or ω. In our computations, we

choose n = 200 and q = 910n. The test results are listed in Table 4.2. And we depict the

surfaces of ρ(S(α, η)) (or ρ(S(α, ω))) with respect to α and η (or ω) for HPPS (or TPPS)

iteration method to intuitively show this functional relationship in Figure 4.2 (or Figure 4.3).

Moreover, we depict the eigenvalue distributions of the HPPS (or TPPS) iteration matrices for

different parameter η (or ω) and the corresponding optimal parameter αopt in Figure 4.4 (or

Figure 4.5).

As can be seen from Table 4.2, when η = 0, the number of iterations and the CPU time

of the HPPS method are the best. And the HPPS method may be slowed by increasing the

module of η. Moreover, we can see that the numerical results of the HPPS method are much

the same when the module of η is equal. And the optimal parameter αopt is equal when the

module of η is equal. This fact is further confirmed by Figure 4.2. For TPPS method, we have

the same results as the HPPS method, which can be seen from Table 4.2 and Figure 4.3. As is

shown in Figure 4.4, the distribution domain of the eigenvalues of the HPPS iteration matrix

Splitting Methods for Non-Hermitian Positive Definite Linear Systems 313

510

1520

2530

−5

0

5

0.85

0.9

0.95

αη

ρ(S(

α,η)

)

Fig. 4.2. Surfaces of ρ(S(α, η)) versus α and η for the HPPS iteration matrix for Example 4.2.

510

1520

2530

−5

0

5

0.85

0.9

0.95

αω

ρ(S(

α,ω)

)

Fig. 4.3. Surfaces of ρ(S(α, ω)) versus α and ω for the TPPS iteration matrix for Example 4.2.

Fig. 4.4. The eigenvalue distributions of the HPPS iteration matrices when η = −4, 0, 4 for Example

4.2.

314 N. HUANG AND C.F. MA

Fig. 4.5. The eigenvalue distributions of the TPPS iteration matrices when ω = −4, 0, 4 for Example

4.2.

0 5 10 15 20 25 300.82

0.84

0.86

0.88

0.9

0.92

0.94

0.96

0.98

1

1.02

Parameter α

Spec

tral r

adiu

s

Take (n,η)=(200,0.25)

HPSSTPSS

Fig. 4.6. Curves of ρ(S(α)) versus α for the HPPS and the TPPS iteration matrices for Example 4.2

by η = ω = 0.25.

when η = 0 is considerably smaller than that when η = 0, in particular, along the direction of

the imaginary axis. From Figure 4.5, we can see the same result for the TPPS iteration matrix.

In addition, as you will see in Table 4.2, the numerical results of the TPPS method are a little

better than that of the HPPS method, which also can be seen from Figure 4.6.

Acknowledgments. The authors thank the anonymous reviewers for their valuable comments

and suggestions that helped improve the quality of this paper. The project was supported by

National Natural Science Foundation of China (Grant Nos. 11071041, 11201074) and Fujian

Natural Science Foundation (Grant No. 2016J01005).

References

[1] S.R. Arridge, Optical tomography in medical imaging, Inverse Probl., 15 (1999) 41-93.

[2] O. Axelsson, A generalized SSOR method, BIT, 12 (1972) 443-467.

Splitting Methods for Non-Hermitian Positive Definite Linear Systems 315

[3] O. Axelsson, Z.-Z. Bai and S.-X. Qiu, A class of nested iteration schemes for linear systems with

a coefficient matrix with a dominant positive definite symmetric part, Numer. Algorithms, 35

(2004) 351-372.

[4] K. Arrow, L. Hurwicz and H. Uzawa, Studies in Nonlinear Programming, Stanford University

Press, Stanford, CA, 1958.

[5] Z.-Z. Bai, Sharp error bounds of some Krylov subspace methods for non-Hermitian linear systems,

Appl. Math. Comput., 109 (2000) 273-285.

[6] Z.-Z. Bai, Optimal parameters in the HSS-like methods for saddle point problems, Numer. Linear

Algebra Appl., 16 (2009) 447-479.

[7] A. Bendali, Numerical analysis of the exterior boundary value problem for the time-harmonic

Maxwell equations by a boundary finite element method, Math. Comp., 43 (1984) 29-68.

[8] M. Benzi and D. Bertaccini, Block preconditioning of real-valued iterative algorithms for complex

linear systems, IMA J. Numer. Anal., 28 (2008) 598-618.

[9] Z.-Z. Bai, M. Benzi and F. Chen, Modified HSS iteration methods for a class of complex symmetric

linear systems, Computing, 87 (2010) 93-111.

[10] Z.-Z. Bai, M. Benzi and F. Chen, On preconditioned MHSS iteration methods for complex sym-

metric linear systems, Numer. Algorithms, 56 (2011) 297-317.

[11] Z.-Z. Bai, I. Duff and A.J. Wathen, A class of incomplete orthogonal factorization methods I:

Methods and theories, BIT, 41 (2001) 53-70.

[12] Z.-Z. Bai and G.H. Golub, Accelerated Hermitian and skew-Hermitian splitting iteration methods

for saddle point problems, IMA J. Numer. Anal., 27 (2007) 1-23.

[13] Z.-Z. Bai, G.H. Golub and C.-K. Li, Convergence properties of preconditioned Hermitian and skew-

Hermitian splitting methods for non-Hermitian positive semidefinite matrices, Math. Comp., 76

(2007) 287-298.

[14] Z.-Z. Bai, G.H. Golub, L.-Z. Lu and J.-F. Yin, Block triangular and skew-Hermitian splitting

method for positive-definite linear systems, SIAM J. Sci. Comput., 26 (2005) 844-863.

[15] Z.-Z. Bai, G.H. Golub and M.K. Ng, On successive-overrelaxation acceleration of the Hermitian

and skew-Hermitian splitting iterations, Numer. Linear Algebra Appl., 14 (2007) 319-335.

[16] Z.-Z. Bai, G.H. Golub and M.K. Ng, Hermitian and skew-Hermitian splitting methods for non-

Hermitian positive definite linear systems, SIAM J. Matrix Anal. Appl., 24 (2003) 603-626.

[17] Z.-Z. Bai, G.H. Golub and J.-Y. Pan, Preconditioned Hermitian and skew-Hermitian splitting

methods for non-Hermitian positive semidefinite linear systems, Numer. Math., 98 (2004) 1-32.

[18] M.A. Bochev and L.A. Krukier, On an iteration solution for strongly nonsymmetric systems of

linear algebraic equations, Comp. Math. Math. Phys., 37 (1997) 1241-1251.

[19] M. Benzi and D. Szyld, Existence and uniqueness of splittings of stationary iterative methods

with applications to alternating methods, Numer. Math., 76 (1997) 309-321.

[20] A. Bunse-Gerstner and R. Stover, On a conjugate gradient-type method for solving complex

symmetric linear systems, Linear Algebra Appl., 287 (1999) 105-123.

[21] Z.-Z. Bai and Z.-Q. Wang, On parameterized inexact Uzawa methods for generalized saddle point

problems, Linear Algebra Appl., 428 (2008) 2900-2932.

[22] S.H. Christiansen, Discrete Fredholm properties and convergence estimates for the electric field

integral equation, Math. Comp., 73 (2004) 143-167.

[23] M. Clemens and T. Weiland, Comparison of Krylov-type methods for complex linear systems

applied to high-voltage problems, IEEE Trans. Mag., 34 (1998) 3335-3338.

[24] D.D. Day and M.A. Heroux, Solving complex-valued linear systems via equivalent real formula-

tions, SIAM J. Sci. Comput., 23 (2001) 480-498.

[25] M. Eiermann, W. Niethammer and R.S. Varga, Acceleration of relaxation methods for non-

Hermitian linear systems, SIAM J. Matrix Anal. Appl., 13 (1992) 979-991.

[26] R.W. Freund, Conjugate gradient-type methods for linear systems with complex symmetric coef-

ficient matrices, SIAM J. Sci. Stat. Comput., 13 (1992) 425-448.

316 N. HUANG AND C.F. MA

[27] A. Hadjidimos, Accelerated overrelaxation method, Math. Comp., 32 (1978) 149-157.

[28] N. Huang and C.-F. Ma, Parallel multisplitting iteration methods based on M-splitting for the

PageRank problem, Appl. Math. Comput., 271 (2015) 337-343.

[29] N. Huang and C.-F. Ma, The BGS-Uzawa and BJ-Uzawa iterative methods for solving the saddle

point problem, Appl. Math. Comput., 256 (2015) 94-108.

[30] N. Huang and C.-F. Ma, The nonlinear inexact Uzawa hybrid algorithms based on one-step Newton

method for solving nonlinear saddle-point problems, Appl. Math. Comput., 270 (2015) 291-311.

[31] N. Huang and C.-F. Ma, Y.-J. Xie, An inexact relaxed DPSS preconditioner for saddle point

problem, Appl. Math. Comput., 265 (2015) 431-447.

[32] M.R. Hestenes and E. Stiefel, Methods of conjugate gradients for solving linear systems, J. Res.

Nat. Bur. Standards, 49 (1952) 409-436.

[33] L.A. Krukier, L.G. Chikina and T.V. Belokon, Triangular skew-symmetric iterative solvers for

strongly nonsymmetric positive real linear system of equations, Appl. Numer. Math., 41 (2002)

89-105.

[34] L.A. Krukier, B.L. Krukier and Z.-R. Ren, Generalized skew-Hermitian triangular splitting iter-

ation methods for saddle-point linear systems, Numer. Linear Algebra Appl., 21 (2014) 152-170.

[35] L.A. Krukier, T. Martynova and Z.-Z. Bai, Product-type skew-Hermitian triangular splitting

iteration methods for strongly non-Hermitian positive definite linear systems, J. Comput. Appl.

Math., 232 (2009) 3-16.

[36] J. Lu and Z. Zhang, A modified nonlinear inexact Uzawa algorithm with a variable relaxation

parameter for the stabilized saddle point problem, SIAM J. Matrix Anal. Appl., 31 (2010) 1934-

1957.

[37] T. Manteuffel, An incomplete factorization technique for positive definite linear systems, Math.

Comp., 34 (1980) 473-497.

[38] G. Moro and J.H. Freed, Calculation of ESR spectra and related Fokker-Planck forms by the use

of the Lanczos algorithm, J. Chem. Phys., 74 (1981) 3757-3773.

[39] B. Poirier, Effcient preconditioning scheme for block partitioned matrices with structured sparsity,

Numer. Linear Algebra Appl., 7 (2000) 715-726.

[40] Y. Saad, Iterative methods for sparse linear systems, (2nd edn), SIAM: Philadelphia, 2003.

[41] D. Schmitt, B. Steffen and T. Weiland, 2D and 3D computations of lossy eigenvalue problems,

IEEE Trans. Magnetics, 30 (1994) 3578-3581.

[42] T. Sogabe and S.-L. Zhang, A COCR method for solving complex symmetric linear systems, J.

Comput. Appl. Math., 199 (2007) 297-303.

[43] H. Van der Vorst, Iterative Krylov methods for large linear systems, Cambridge University Press,

Cambridge, 2003.

[44] L. Wang and Z.-Z. Bai, Skew-Hermitian triangular splitting iteration methods for non-Hermitian

positive definite linear systems of strong skew-Hermitian parts, BIT, 44 (2004) 363-386.

[45] W.-W. Li and X. Wang, A modified GPSS method for non-Hermitian positive definite linear

systems, Appl. Math. Comput., 234 (2014) 253-259.

[46] X. Wang, Y. Li and L. Dai, On Hermitian and skew-Hermitian splitting iteration methods for the

linear matrix equation AXB = C, Comput. Math. Appl., 65 (2013) 657-664.

[47] X. Wang, W.-W. Li and L.-Z. Miao, On positive-definite and skew-Hermitian splitting iteration

methods for continuous Sylvester equation AX + XB = C, Comput. Math. Appl., 66 (2013)

2352-2361.