Date post: | 19-Jun-2015 |
Category: |
Education |
Upload: | thomas-mach |
View: | 833 times |
Download: | 4 times |
MAX PLANCK INSTITUTE
FOR DYNAMICS OF COMPLEX
TECHNICAL SYSTEMS
MAGDEBURG
82nd GAMM Annual Scientific ConferenceGraz, April 19, 2011
Preconditioned Inverse Iteration forHierarchical Matrices
Peter Benner and Thomas Mach
Max Planck Institute for Dynamics of Complex Technical SystemsComputational Methods in Systems and Control Theory
Magdeburg
Max Planck Institute Magdeburg Thomas Mach, Preconditioned Inverse Iteration for Hierarchical Matrices 1/28
Hierarchical (H-)Matrices PINVIT Numerical Results
Outline
1 Hierarchical (H-)Matrices
2 PINVIT
3 Numerical Results
Max Planck Institute Magdeburg Thomas Mach, Preconditioned Inverse Iteration for Hierarchical Matrices 2/28
Hierarchical (H-)Matrices PINVIT Numerical Results
H-Matrices [Hackbusch 1998]
Some dense matrices, e.g. BEM or FEM, can be approximated byH-matrices in a data-sparse manner.
hierarchical tree TI block H-tree TI× I
I = {1, 2, 3, 4, 5, 6, 7, 8}
{1, 2, 3, 4} {5, 6, 7, 8}
{1, 2} {3, 4} {5, 6} {7, 8}
{1}{2}{3}{4}{5}{6}{7}{8}
1 1 1 12 2 2 23 3 3 34 4 4 45 5 5 56 6 6 67 7 7 78 8 8 8
1 1 1 12 2 2 23 3 3 34 4 4 45 5 5 56 6 6 67 7 7 78 8 8 8
dense matrices, rank-k-matrices
rank-k-matrix: Ms×t = ABT , A ∈ Rn×k ,B ∈ Rm×k (k � n,m)
Max Planck Institute Magdeburg Thomas Mach, Preconditioned Inverse Iteration for Hierarchical Matrices 3/28
Hierarchical (H-)Matrices PINVIT Numerical Results
H-Matrices [Hackbusch 1998]
Hierarchical matrices
H(TI× I, k) ={M ∈ RI× I
∣∣ rank (Ma×b) ≤ k ∀a× b admissible}
22
3 3
7 10
3 7
3 10
19 10
10 31
14 8
11 11
14 118 11
19 10
10 31 11
11 31
11 9
9 16 12
11 1611 8
9 16 11
11 16
619 73 3
8
11 11
9 3
7 3
11
8 11
25 10
10 19 11
11 31
8 5
11 8
8 15 8 12
116 515 6
13
13
7 5
13 8
8 11 8 11
116 5
15 6
8
11 8
8 15
5 8 11
126 15
5 6 13
137
13 8
8 11
5 8 11
116 15
5 6
6110 103
6 14
10 3 6
10 14
25 10 6
10
6
19 10
10 31
15 9
9 11 10
10 1615 9
8 11 10
10 16
5110 107 9 7
3 3
10 7
109 3
7 3
25 11
1125 10
10 19
11 811 8
8 15 9
8 15 12 13
1310 7
13 8
8 11 9
9 15 1119
2010 7
13 8
9
11 8
8 15 11 12
139 7
13 8
9
13 8
8 1111
1011 8
8 15 8
9 15
7 12 13
1310
13 8
8 11 8
9 15
7 1120
199
13 9
8
11 8
8 15
7 11 12
129
13 9
813 8
8 11
7 11
39 10
10 25
3 7
3 10 107 10 6
3 3
7 10 7
1010
6
22
3 3
7 10
3 7
3 10
19 10
10 31
15 10
11
11 9
9 16
15 11
1011 8
9 16
34 10
10 25
13 10
7 1113 7
10 11 61
6 513 6 11
12
8 5
11 8
8 15 812
136 5
13 6 11
11 236 13
5 6 12
118
11 8
8 15
5 813
12
6 13
5 6 10
11 23
20 9
9 39
9 7
103 7
3 10
9 107
3 3
7 10 61
15 10
1015 9
9 11
15 10
1015 9
8 11
20 9
9 34
9 7
10 13
9 107 13 51
adaptive rank k(ε)
storage NSt,H(T , k) = O(n log n k(ε))
complexity of approximate arithmetic
MHv O(n log n k(ε))+H,−H O(n log n k(ε)2)
∗H,HLU(·), (·)−1H O(n (log n)2 k(ε)2)
Max Planck Institute Magdeburg Thomas Mach, Preconditioned Inverse Iteration for Hierarchical Matrices 4/28
Hierarchical (H-)Matrices PINVIT Numerical Results
H-Matrices [Hackbusch 1998]
Hierarchical matrices
H(TI× I, k) ={M ∈ RI× I
∣∣ rank (Ma×b) ≤ k ∀a× b admissible}
22
3 3
7 10
3 7
3 10
19 10
10 31
14 8
11 11
14 118 11
19 10
10 31 11
11 31
11 9
9 16 12
11 1611 8
9 16 11
11 16
619 73 3
8
11 11
9 3
7 3
11
8 11
25 10
10 19 11
11 31
8 5
11 8
8 15 8 12
116 515 6
13
13
7 5
13 8
8 11 8 11
116 5
15 6
8
11 8
8 15
5 8 11
126 15
5 6 13
137
13 8
8 11
5 8 11
116 15
5 6
6110 103
6 14
10 3 6
10 14
25 10 6
10
6
19 10
10 31
15 9
9 11 10
10 1615 9
8 11 10
10 16
5110 107 9 7
3 3
10 7
109 3
7 3
25 11
1125 10
10 19
11 811 8
8 15 9
8 15 12 13
1310 7
13 8
8 11 9
9 15 1119
2010 7
13 8
9
11 8
8 15 11 12
139 7
13 8
9
13 8
8 1111
1011 8
8 15 8
9 15
7 12 13
1310
13 8
8 11 8
9 15
7 1120
199
13 9
8
11 8
8 15
7 11 12
129
13 9
813 8
8 11
7 11
39 10
10 25
3 7
3 10 107 10 6
3 3
7 10 7
1010
6
22
3 3
7 10
3 7
3 10
19 10
10 31
15 10
11
11 9
9 16
15 11
1011 8
9 16
34 10
10 25
13 10
7 1113 7
10 11 61
6 513 6 11
12
8 5
11 8
8 15 812
136 5
13 6 11
11 236 13
5 6 12
118
11 8
8 15
5 813
12
6 13
5 6 10
11 23
20 9
9 39
9 7
103 7
3 10
9 107
3 3
7 10 61
15 10
1015 9
9 11
15 10
1015 9
8 11
20 9
9 34
9 7
10 13
9 107 13 51
adaptive rank k(ε)
storage NSt,H(T , k) = O(n log n k(ε))
complexity of approximate arithmetic
MHv O(n log n k(ε))+H,−H O(n log n k(ε)2)
∗H,HLU(·), (·)−1H O(n (log n)2 k(ε)2)
A1
BT1
+ A2
BT2
= A1 A2
BT1
BT2
Max Planck Institute Magdeburg Thomas Mach, Preconditioned Inverse Iteration for Hierarchical Matrices 4/28
Hierarchical (H-)Matrices PINVIT Numerical Results
Goal: Eigenvalues of symmetric H-Matrices
M = MT ∈ H(T , k),
Λ(M) = {λ1, λ2, . . . , λn},λ1 ≥ λ2 ≥ · · · ≥ λn > 0
⇓How to find λi in O(n (log n)α kβ)?
X
How to find λi in O(n (log n)α kβ)?
X
Max Planck Institute Magdeburg Thomas Mach, Preconditioned Inverse Iteration for Hierarchical Matrices 5/28
Hierarchical (H-)Matrices PINVIT Numerical Results
Goal: Eigenvalues of symmetric H-Matrices
M = MT ∈ H(T , k),
Λ(M) = {λ1, λ2, . . . , λn},λ1 ≥ λ2 ≥ · · · ≥ λn > 0
⇓How to find λn in O(n (log n)α kβ)?
X
How to find λi in O(n (log n)α kβ)?
X
Max Planck Institute Magdeburg Thomas Mach, Preconditioned Inverse Iteration for Hierarchical Matrices 5/28
Hierarchical (H-)Matrices PINVIT Numerical Results
Preconditioned Inverse Iteration[Knyazev, Neymeyr, et al.]
Definition
The function
µ(x) = µ(x ,M) =xTMx
xT x
is called the Rayleigh quotient.
Minimize the Rayleigh quotient by a gradient method:
xi+1 := xi − α∇µ(xi ), ∇µ(x) =2
xT x(Mx − xµ(x)) ,
+ preconditioning ⇒ update equation:
xi+1 := xi − B−1 (Mxi − xiµ(xi )) .
Residual r(x) = Mx − xµ(x).
Preconditioned residual B−1r(x) = B−1 (Mx − xµ(x)).
Max Planck Institute Magdeburg Thomas Mach, Preconditioned Inverse Iteration for Hierarchical Matrices 6/28
Hierarchical (H-)Matrices PINVIT Numerical Results
Preconditioned Inverse Iteration[Knyazev, Neymeyr, et al.]
Definition
The function
µ(x) = µ(x ,M) =xTMx
xT x
is called the Rayleigh quotient.
Minimize the Rayleigh quotient by a gradient method:
xi+1 := xi − α∇µ(xi ), ∇µ(x) =2
xT x(Mx − xµ(x)) ,
+ preconditioning ⇒ update equation:
xi+1 := xi − B−1 (Mxi − xiµ(xi )) .
Residual r(x) = Mx − xµ(x).
Preconditioned residual B−1r(x) = B−1 (Mx − xµ(x)).
Max Planck Institute Magdeburg Thomas Mach, Preconditioned Inverse Iteration for Hierarchical Matrices 6/28
Hierarchical (H-)Matrices PINVIT Numerical Results
Preconditioned Inverse Iteration[Knyazev, Neymeyr, et al.]
Definition
The function
µ(x) = µ(x ,M) =xTMx
xT x
is called the Rayleigh quotient.
Minimize the Rayleigh quotient by a gradient method:
xi+1 := xi − α∇µ(xi ), ∇µ(x) =2
xT x(Mx − xµ(x)) ,
+ preconditioning ⇒ update equation:
xi+1 := xi − B−1 (Mxi − xiµ(xi )) .
Residual r(x) = Mx − xµ(x).
Preconditioned residual B−1r(x) = B−1 (Mx − xµ(x)).
Max Planck Institute Magdeburg Thomas Mach, Preconditioned Inverse Iteration for Hierarchical Matrices 6/28
Hierarchical (H-)Matrices PINVIT Numerical Results
Preconditioned Inverse Iteration[Knyazev, Neymeyr, et al.]
Definition
The function
µ(x) = µ(x ,M) =xTMx
xT x
is called the Rayleigh quotient.
Minimize the Rayleigh quotient by a gradient method:
xi+1 := xi − α∇µ(xi ), ∇µ(x) =2
xT x(Mx − xµ(x)) ,
+ preconditioning ⇒ update equation:
xi+1 := xi − B−1 (Mxi − xiµ(xi )) .
Residual r(x) = Mx − xµ(x).
Preconditioned residual B−1r(x) = B−1 (Mx − xµ(x)).
Max Planck Institute Magdeburg Thomas Mach, Preconditioned Inverse Iteration for Hierarchical Matrices 6/28
Hierarchical (H-)Matrices PINVIT Numerical Results
Preconditioned Inverse Iteration[Knyazev, Neymeyr, et al.]
Definition
The function
µ(x) = µ(x ,M) =xTMx
xT x
is called the Rayleigh quotient.
Minimize the Rayleigh quotient by a gradient method:
xi+1 := xi − α∇µ(xi ), ∇µ(x) =2
xT x(Mx − xµ(x)) ,
+ preconditioning ⇒ update equation:
xi+1 := xi − B−1 (Mxi − xiµ(xi )) .
Residual r(x) = Mx − xµ(x).
Preconditioned residual B−1r(x) = B−1 (Mx − xµ(x)).
Max Planck Institute Magdeburg Thomas Mach, Preconditioned Inverse Iteration for Hierarchical Matrices 6/28
Hierarchical (H-)Matrices PINVIT Numerical Results
Preconditioned Inverse Iteration[Knyazev, Neymeyr 2009]
xi+1 := xi − B−1 (Mxi − xiµ(xi ))
If
M ∈ Rn×n symmetric positive definite and
B−1 approximates the inverse of M, so that∥∥I − B−1M∥∥M≤ c < 1,
then Preconditioned INVerse ITeration (PINVIT) converges andthe number of iterations is independent of n.
Max Planck Institute Magdeburg Thomas Mach, Preconditioned Inverse Iteration for Hierarchical Matrices 7/28
Hierarchical (H-)Matrices PINVIT Numerical Results
Preconditioned Inverse Iteration[Knyazev, Neymeyr, et al.]
The residual
ri = Mxi − xiµ(xi )
converges to 0, so that
‖ri‖2 < ε
is a useful termination criterion.
Max Planck Institute Magdeburg Thomas Mach, Preconditioned Inverse Iteration for Hierarchical Matrices 8/28
Hierarchical (H-)Matrices PINVIT Numerical Results
Variants of PINVIT[Neymeyr 2001: A Hierarchy of Precond. Eigens. for Ellipt. Diff. Op.]
Classification by Neymeyr:
PINVIT(1): xi+1 := xi − B−1ri .
PINVIT(2): xi+1 := arg minv∈span{xi ,B−1ri} µ(v).
PINVIT(3): xi+1 := arg minv∈span{xi−1,xi ,B−1ri} µ(v).
PINVIT(n): Analogously.
PINVIT(·,d): Replacing x by a rectangular full rank matrixX ∈ Rn×d one gets the subspace version of PINVIT(·).
Max Planck Institute Magdeburg Thomas Mach, Preconditioned Inverse Iteration for Hierarchical Matrices 9/28
Hierarchical (H-)Matrices PINVIT Numerical Results
Algorithm and Complexity
The number of iterations is independent of matrix size n.
H-PINVIT(1,d)
Input: M ∈ Rn×n, X0 ∈ Rn×d (XT0 X0 = I , e.g. randomly chosen)
Output: Xp ∈ Rn×d , µ ∈ Rd×d , with ‖MXp − Xpµ‖ ≤ εB−1 = (M)−1H or B−1 = L−TH L−1H
O(n (log n)2 k (c)2)
R := MX0 − X0µ, µ = XT0 MX0
for (i := 1;‖R‖F > ε; i + +) do
Xi := Orthogonalize(Xi−1 − B−1R
)
O(n (log n) k (c)2)
R := MXi − Xiµ, µ = XTi MXi
O(n (log n) k (c))
end
The complexity of the algorithm is determined by the H-matrixinversion/Cholesky decomposition: ⇒ O(n (log n)2 k (c)2).
Competitive to MATLAB® eigs.
Expensive.
Max Planck Institute Magdeburg Thomas Mach, Preconditioned Inverse Iteration for Hierarchical Matrices 10/28
Hierarchical (H-)Matrices PINVIT Numerical Results
Algorithm and Complexity
The number of iterations is independent of matrix size n.
H-PINVIT(1,d)
Input: M ∈ Rn×n, X0 ∈ Rn×d (XT0 X0 = I , e.g. randomly chosen)
Output: Xp ∈ Rn×d , µ ∈ Rd×d , with ‖MXp − Xpµ‖ ≤ εB−1 = (M)−1H or B−1 = L−TH L−1H O(n (log n)2 k (c)2)R := MX0 − X0µ, µ = XT
0 MX0
for (i := 1;‖R‖F > ε; i + +) do
Xi := Orthogonalize(Xi−1 − B−1R
)O(n (log n) k (c)2)
R := MXi − Xiµ, µ = XTi MXi O(n (log n) k (c))
end
The complexity of the algorithm is determined by the H-matrixinversion/Cholesky decomposition: ⇒ O(n (log n)2 k (c)2).
Competitive to MATLAB eigs.
Expensive.
Max Planck Institute Magdeburg Thomas Mach, Preconditioned Inverse Iteration for Hierarchical Matrices 10/28
Hierarchical (H-)Matrices PINVIT Numerical Results
Goal: Eigenvalues of symmetric H-Matrices
M = MT ∈ H(T , k),
Λ(M) = {λ1, λ2, . . . , λn},λ1 ≥ λ2 ≥ · · · ≥ λn > 0
⇓How to find λn in O(n (log n)α kβ)?
X
How to find λi in O(n (log n)α kβ)?
X
Max Planck Institute Magdeburg Thomas Mach, Preconditioned Inverse Iteration for Hierarchical Matrices 11/28
Hierarchical (H-)Matrices PINVIT Numerical Results
Goal: Eigenvalues of symmetric H-Matrices
M = MT ∈ H(T , k),
Λ(M) = {λ1, λ2, . . . , λn},λ1 ≥ λ2 ≥ · · · ≥ λn > 0
⇓How to find λn in O(n (log n)α kβ)? X
How to find λi in O(n (log n)α kβ)?
X
Max Planck Institute Magdeburg Thomas Mach, Preconditioned Inverse Iteration for Hierarchical Matrices 11/28
Hierarchical (H-)Matrices PINVIT Numerical Results
Goal: Eigenvalues of symmetric H-Matrices
M = MT ∈ H(T , k),
Λ(M) = {λ1, λ2, . . . , λn},λ1 ≥ λ2 ≥ · · · ≥ λn > 0
⇓How to find λn in O(n (log n)α kβ)? X
How to find λi in O(n (log n)α kβ)?
X
Max Planck Institute Magdeburg Thomas Mach, Preconditioned Inverse Iteration for Hierarchical Matrices 12/28
Hierarchical (H-)Matrices PINVIT Numerical Results
Folded Spectrum Method
How to find λi in O(n (log n)α kβ)?
If i = n − d with d < O(log n),
use subspace version PINVIT(·,d).
0λn λn−1λn−2. . .
λ1
But (M − σI) is not positive definite.
Max Planck Institute Magdeburg Thomas Mach, Preconditioned Inverse Iteration for Hierarchical Matrices 13/28
Hierarchical (H-)Matrices PINVIT Numerical Results
Folded Spectrum Method
How to find λi in O(n (log n)α kβ)?
If i = n − d with d � log n?
shift with σ near λi .
0λn. . .
λi+1 λi λi−1. . .
λ1
But (M − σI) is not positive definite.
Max Planck Institute Magdeburg Thomas Mach, Preconditioned Inverse Iteration for Hierarchical Matrices 13/28
Hierarchical (H-)Matrices PINVIT Numerical Results
Folded Spectrum Method
How to find λi in O(n (log n)α kβ)?
If i = n − d with d � log n,
shift with σ near λi .
0λn. . .
λi+1 λi λi−1. . .
λ1σ
But (M − σI) is not positive definite.
Max Planck Institute Magdeburg Thomas Mach, Preconditioned Inverse Iteration for Hierarchical Matrices 13/28
Hierarchical (H-)Matrices PINVIT Numerical Results
Folded Spectrum Method
How to find λi in O(n (log n)α kβ)?
If i = n − d with d � log n,
shift with σ near λi .
0λn. . .
λi+1 λi λi−1. . .
λ1σ
But (M − σI) is not positive definite.
Max Planck Institute Magdeburg Thomas Mach, Preconditioned Inverse Iteration for Hierarchical Matrices 13/28
Hierarchical (H-)Matrices PINVIT Numerical Results
Folded Spectrum Method
Folded Spectrum Method [Wang, Zunger 1994]
Mσ = (M − σI)2
Mσ is s.p.d., if M is s.p.d. and σ 6= λi .
Assume all eigenvalues of Mσ are simple.
Mv = λv ⇔ Mσv = (M − σI)2 v
= M2v − 2σMv + σ2v
= λ2v − 2σλv + σ2v
= (λ− σ)2v
Max Planck Institute Magdeburg Thomas Mach, Preconditioned Inverse Iteration for Hierarchical Matrices 14/28
Hierarchical (H-)Matrices PINVIT Numerical Results
Folded Spectrum Method
1 Choose σ.2 Compute
a) Mσ = (M − σI)2 andb) M−1
σ or LLT = Mσ.
3 Use PINVIT to find the smallest eigenpair (µσ, v) of Mσ.4 Compute µ = vTMv/vTv .
(µ, v) is the nearest eigenpair to σ.
H-arithmetic enables us to shift, square and invert M resp.Mσ in linear-polylogarithmic complexity.
If M is sparse, then shifting, squaring and inverting isprohibitive.
Max Planck Institute Magdeburg Thomas Mach, Preconditioned Inverse Iteration for Hierarchical Matrices 15/28
Hierarchical (H-)Matrices PINVIT Numerical Results
Folded Spectrum Method
1 Choose σ.2 Compute
a) Mσ = (M − σI)2 andb) M−1
σ or LLT = Mσ.
3 Use PINVIT to find the smallest eigenpair (µσ, v) of Mσ.4 Compute µ = vTMv/vTv .
(µ, v) is the nearest eigenpair to σ.
H-arithmetic enables us to shift, square and invert M resp.Mσ in linear-polylogarithmic complexity.
If M is sparse, then shifting, squaring and inverting isprohibitive.
Max Planck Institute Magdeburg Thomas Mach, Preconditioned Inverse Iteration for Hierarchical Matrices 15/28
Hierarchical (H-)Matrices PINVIT Numerical Results
Folded Spectrum Method
1 Choose σ.2 Compute
a) Mσ = (M − σI)2 andb) M−1
σ or LLT = Mσ.
3 Use PINVIT to find the smallest eigenpair (µσ, v) of Mσ.4 Compute µ = vTMv/vTv .
(µ, v) is the nearest eigenpair to σ.
H-arithmetic enables us to shift, square and invert M resp.Mσ in linear-polylogarithmic complexity.
If M is sparse, then shifting, squaring and inverting isprohibitive.
Max Planck Institute Magdeburg Thomas Mach, Preconditioned Inverse Iteration for Hierarchical Matrices 15/28
Hierarchical (H-)Matrices PINVIT Numerical Results
Folded Spectrum Method
1 Choose σ.2 Compute
a) Mσ = (M − σI)2 andb) M−1
σ or LLT = Mσ.
3 Use PINVIT to find the smallest eigenpair (µσ, v) of Mσ.4 Compute µ = vTMv/vTv .
(µ, v) is the nearest eigenpair to σ.
H-arithmetic enables us to shift, square and invert M resp.Mσ in linear-polylogarithmic complexity.
If M is sparse, then shifting, squaring and inverting isprohibitive.
Max Planck Institute Magdeburg Thomas Mach, Preconditioned Inverse Iteration for Hierarchical Matrices 15/28
Hierarchical (H-)Matrices PINVIT Numerical Results
Folded Spectrum Method
1 Choose σ.2 Compute
a) Mσ = (M − σI)2 andb) M−1
σ or LLT = Mσ.
3 Use PINVIT to find the smallest eigenpair (µσ, v) of Mσ.4 Compute µ = vTMv/vTv .
(µ, v) is the nearest eigenpair to σ.
H-arithmetic enables us to shift, square and invert M resp.Mσ in linear-polylogarithmic complexity.
If M is sparse, then shifting, squaring and inverting isprohibitive.
Max Planck Institute Magdeburg Thomas Mach, Preconditioned Inverse Iteration for Hierarchical Matrices 15/28
Hierarchical (H-)Matrices PINVIT Numerical Results
Goal: Eigenvalues of symmetric H-Matrices
M = MT ∈ H(T , k),
Λ(M) = {λ1, λ2, . . . , λn},λ1 ≥ λ2 ≥ · · · ≥ λn > 0
⇓How to find λn in O(n (log n)α kβ)? X
How to find λi in O(n (log n)α kβ)?
X
Max Planck Institute Magdeburg Thomas Mach, Preconditioned Inverse Iteration for Hierarchical Matrices 16/28
Hierarchical (H-)Matrices PINVIT Numerical Results
Goal: Eigenvalues of symmetric H-Matrices
M = MT ∈ H(T , k),
Λ(M) = {λ1, λ2, . . . , λn},λ1 ≥ λ2 ≥ · · · ≥ λn > 0
⇓How to find λn in O(n (log n)α kβ)? X
How to find λi in O(n (log n)α kβ)? X
Max Planck Institute Magdeburg Thomas Mach, Preconditioned Inverse Iteration for Hierarchical Matrices 16/28
Hierarchical (H-)Matrices PINVIT Numerical Results
Numerical Results
Numerical Results
Max Planck Institute Magdeburg Thomas Mach, Preconditioned Inverse Iteration for Hierarchical Matrices 17/28
Hierarchical (H-)Matrices PINVIT Numerical Results
Hlib
Hlib [Borm, Grasedyck, et al.]
We use the Hlib1.3 (www.hlib.org) for theH-arithmetic operations and some examples out ofthe library for testing the eigenvalue algorithm.
Max Planck Institute Magdeburg Thomas Mach, Preconditioned Inverse Iteration for Hierarchical Matrices 18/28
Hierarchical (H-)Matrices PINVIT Numerical Results
2D Laplace, smallest eigenvalues
2D Laplace over [−1, 1]× [−1, 1], precond.: H-inversion
Name ni error titi
ti−1
N(ni )N(ni−1)
FEM8 64 5.6146E-010 0.01
FEM16 256 4.5918E-010 0.02 2.00 106.67
FEM32 1 024 3.7550E-010 0.12 6.17 27.08
FEM64 4 096 3.8009E-010 0.82 6.68 8.06
FEM128 16 384 4.4099E-010 5.84 7.09 5.44
FEM256 65 536 3.9651E-010 34.47 5.91 5.22
FEM512 262 144 3.7877E-010 194.00 5.63 6.51
d = 4, c = 0.2, εeig = 10−4, N(ni ) = ni (log2 ni )CspCid
Time t only H-PINVIT (without H-inversion)
∥∥∥∥∥∥∥∥∥
λ1 − λ1λ2 − λ2λ3 − λ3λ4 − λ4
∥∥∥∥∥∥∥∥∥2
Max Planck Institute Magdeburg Thomas Mach, Preconditioned Inverse Iteration for Hierarchical Matrices 19/28
Hierarchical (H-)Matrices PINVIT Numerical Results
2D Laplace, smallest eigenvalues
2D Laplace over [−1, 1]× [−1, 1], precond.: H-inversion
Name ni error titi
ti−1
N(ni )N(ni−1)
FEM8 64 5.6146E-010 0.01
FEM16 256 4.5918E-010 0.02 2.00 106.67
FEM32 1 024 3.7550E-010 0.12 6.17 27.08
FEM64 4 096 3.8009E-010 0.82 6.68 8.06
FEM128 16 384 4.4099E-010 5.84 7.09 5.44
FEM256 65 536 3.9651E-010 34.47 5.91 5.22
FEM512 262 144 3.7877E-010 194.00 5.63 6.51
d = 4, c = 0.2, εeig = 10−4, N(ni ) = ni (log2 ni )CspCid
Time t only H-PINVIT (without H-inversion)
∥∥∥∥∥∥∥∥∥
λ1 − λ1λ2 − λ2λ3 − λ3λ4 − λ4
∥∥∥∥∥∥∥∥∥2
Max Planck Institute Magdeburg Thomas Mach, Preconditioned Inverse Iteration for Hierarchical Matrices 19/28
Hierarchical (H-)Matrices PINVIT Numerical Results
2D Laplace, smallest eigenvalues
FEM16FEM32
FEM64FEM128
FEM256FEM512
10−3
10−2
10−1
100
101
102
103
104
CP
Uti
me
ins
H-PINVIT(1,4) H-inversion
H-PINVIT(3,4) O(N(ni ) log ni )
O(N(ni ))
Max Planck Institute Magdeburg Thomas Mach, Preconditioned Inverse Iteration for Hierarchical Matrices 20/28
Hierarchical (H-)Matrices PINVIT Numerical Results
2D Laplace, smallest eigenvalues
FEM16FEM32
FEM64FEM128
FEM256FEM512
10−3
10−2
10−1
100
101
102
103
104
CP
Uti
me
ins
H-PINVIT(1,4) H-inversion
H-PINVIT(3,4) O(N(ni ) log ni )
O(N(ni )) MATLAB eigs
Max Planck Institute Magdeburg Thomas Mach, Preconditioned Inverse Iteration for Hierarchical Matrices 20/28
Hierarchical (H-)Matrices PINVIT Numerical Results
Algorithm and Complexity
The number of iterations is independent of matrix size n.
H-PINVIT(1,d)
Input: M ∈ Rn×n, X0 ∈ Rn×d (XT0 X0 = I , e.g. randomly chosen)
Output: Xp ∈ Rn×d , µ ∈ Rd×d , with ‖MXp − Xpµ‖ ≤ εB−1 = (M)−1H or B−1 = L−TH L−1H O(n (log n)2 k (c)2)R := MX0 − X0µ, µ = XT
0 MX0
for (i := 1;‖R‖F > ε; i + +) do
Xi := Orthogonalize(Xi−1 − B−1R
)O(n (log n) k (c)2)
R := MXi − Xiµ, µ = XTi MXi O(n (log n) k (c))
end
The complexity of the algorithm is determined by the H-matrixinversion/Cholesky decomposition: ⇒ O(n (log n)2 k (c)2).
Competitive to MATLAB eigs.
Expensive.
Max Planck Institute Magdeburg Thomas Mach, Preconditioned Inverse Iteration for Hierarchical Matrices 21/28
Hierarchical (H-)Matrices PINVIT Numerical Results
Algorithm and Complexity
The number of iterations is independent of matrix size n.
H-PINVIT(1,d)
Input: M ∈ Rn×n, X0 ∈ Rn×d (XT0 X0 = I , e.g. randomly chosen)
Output: Xp ∈ Rn×d , µ ∈ Rd×d , with ‖MXp − Xpµ‖ ≤ εB−1 = (M)−1H or B−1 = L−TH L−1H O(n (log n)2 k (c)2)R := MX0 − X0µ, µ = XT
0 MX0
for (i := 1;‖R‖F > ε; i + +) do
Xi := Orthogonalize(Xi−1 − B−1R
)O(n (log n) k (c)2)
R := MXi − Xiµ, µ = XTi MXi O(n (log n) k (c))
end
The complexity of the algorithm is determined by the H-matrixinversion/Cholesky decomposition: ⇒ O(n (log n)2 k (c)2).
Competitive to MATLAB eigs.
Expensive.
Max Planck Institute Magdeburg Thomas Mach, Preconditioned Inverse Iteration for Hierarchical Matrices 21/28
Hierarchical (H-)Matrices PINVIT Numerical Results
Algorithm and Complexity
The number of iterations is independent of matrix size n.
H-PINVIT(1,d)
Input: M ∈ Rn×n, X0 ∈ Rn×d (XT0 X0 = I , e.g. randomly chosen)
Output: Xp ∈ Rn×d , µ ∈ Rd×d , with ‖MXp − Xpµ‖ ≤ εB−1 = (M)−1H or B−1 = L−TH L−1H O(n (log n)2 k (c)2)R := MX0 − X0µ, µ = XT
0 MX0
for (i := 1;‖R‖F > ε; i + +) do
Xi := Orthogonalize(Xi−1 − B−1R
)O(n (log n) k (c)2)
R := MXi − Xiµ, µ = XTi MXi O(n (log n) k (c))
end
The complexity of the algorithm is determined by the H-matrixinversion/Cholesky decomposition: ⇒ O(n (log n)2 k (c)2).
Competitive to MATLAB eigs.
Expensive.
Max Planck Institute Magdeburg Thomas Mach, Preconditioned Inverse Iteration for Hierarchical Matrices 21/28
Hierarchical (H-)Matrices PINVIT Numerical Results
2D Laplace, smallest eigenvalues
2D Laplace over [−1, 1]× [−1, 1], precond.: H-Cholesky decomp.
Name ni error titi
ti−1
N(ni )N(ni−1)
FEM8 64 4.6920E-010 0.01
FEM16 256 4.7963E-010 0.02 2.00 106.67
FEM32 1 024 3.4696E-010 0.08 4.00 27.08
FEM64 4 096 4.6414E-010 0.48 6.00 8.06
FEM128 16 384 3.3206E-010 3.20 6.67 5.44
FEM256 65 536 3.8468E-010 13.90 4.34 5.22
FEM512 262 144 3.1353E-010 62.40 4.49 6.51
d = 4, c = 0.2, εeig = 10−4, N(ni ) = ni (log2 ni )CspCid
Time t only H-PINVIT (without H-Cholesky decomposition)
∥∥∥∥∥∥∥∥∥
λ1 − λ1λ2 − λ2λ3 − λ3λ4 − λ4
∥∥∥∥∥∥∥∥∥2
Max Planck Institute Magdeburg Thomas Mach, Preconditioned Inverse Iteration for Hierarchical Matrices 22/28
Hierarchical (H-)Matrices PINVIT Numerical Results
2D Laplace, smallest eigenvalues
FEM16FEM32
FEM64FEM128
FEM256FEM512
10−3
10−2
10−1
100
101
102
103
104
CP
Uti
me
ins
H-PINVIT(1,4) H-Chol. decomp.
H-PINVIT(3,4) O(N(ni ) log ni )
O(N(ni ))
Max Planck Institute Magdeburg Thomas Mach, Preconditioned Inverse Iteration for Hierarchical Matrices 23/28
Hierarchical (H-)Matrices PINVIT Numerical Results
2D Laplace, smallest eigenvalues
FEM16FEM32
FEM64FEM128
FEM256FEM512
10−3
10−2
10−1
100
101
102
103
104
CP
Uti
me
ins
H-PINVIT(1,4) H-Chol. decomp.
H-PINVIT(3,4) O(N(ni ) log ni )
O(N(ni )) MATLAB eigs
Max Planck Institute Magdeburg Thomas Mach, Preconditioned Inverse Iteration for Hierarchical Matrices 23/28
Hierarchical (H-)Matrices PINVIT Numerical Results
2D Laplace, vn, FEM64
−1 −0.5 00.5 1−1
0
1−5
0
5
·10−2
−6
−4
−2
0
2
4
6·10−2
Max Planck Institute Magdeburg Thomas Mach, Preconditioned Inverse Iteration for Hierarchical Matrices 24/28
Hierarchical (H-)Matrices PINVIT Numerical Results
2D Laplace, vn−1, FEM64
−1 −0.5 00.5 1−1
0
1−5
0
5
·10−2
−6
−4
−2
0
2
4
6·10−2
Max Planck Institute Magdeburg Thomas Mach, Preconditioned Inverse Iteration for Hierarchical Matrices 24/28
Hierarchical (H-)Matrices PINVIT Numerical Results
2D Laplace, vn−2, FEM64
−1 −0.5 00.5 1−1
0
1−5
0
5
·10−2
−6
−4
−2
0
2
4
6·10−2
Max Planck Institute Magdeburg Thomas Mach, Preconditioned Inverse Iteration for Hierarchical Matrices 24/28
Hierarchical (H-)Matrices PINVIT Numerical Results
2D Laplace, vn−3, FEM64
−1 −0.5 00.5 1−1
0
1−5
0
5
·10−2
−6
−4
−2
0
2
4
6·10−2
Max Planck Institute Magdeburg Thomas Mach, Preconditioned Inverse Iteration for Hierarchical Matrices 24/28
Hierarchical (H-)Matrices PINVIT Numerical Results
2D Laplace, inner eigenvalues
2D Laplace over [−1, 1]× [−1, 1], precond.: H-inversion
Name ni error titi
ti−1
N(ni )N(ni−1)
FEM8 64 1.7219E-013 <0.01
FEM16 256 7.2388E-012 0.03 106.67
FEM32 1 024 6.0045E-013 0.12 3.57 27.08
FEM64 4 096 1.0915E-012 0.93 7.48 8.06
FEM128 16 384 1.5655E-012 6.08 6.50 5.44
FEM256 65 536 1.1976E-011 52.40 8.62 5.22
d = 4, c = 0.2, εeig = 10−4, N(ni ) = ni (log2 ni )CspCid
Time t only H-PINVIT (without H-inversion)
Max Planck Institute Magdeburg Thomas Mach, Preconditioned Inverse Iteration for Hierarchical Matrices 25/28
Hierarchical (H-)Matrices PINVIT Numerical Results
2D Laplace, inner eigenvalues
2D Laplace over [−1, 1]× [−1, 1], precond.: H-Cholesky decomp.
Name ni error titi
ti−1
N(ni )N(ni−1)
FEM8 64 1.5306E-013 <0.01
FEM16 256 9.8071E-012 0.02 106.67
FEM32 1 024 1.0577E-012 0.05 2.50 27.08
FEM64 4 096 1.5776E-012 0.31 6.20 8.06
FEM128 16 384 2.1416E-012 1.64 5.29 5.44
FEM256 65 536 6.7400E-012 9.73 5.93 5.22
FEM512 262 144 1.5002E-010 545.00 56.01 6.51
d = 4, c = 0.2, εeig = 10−4, N(ni ) = ni (log2 ni )CspCid
Time t only H-PINVIT (without H-Cholesky decomposition)
Max Planck Institute Magdeburg Thomas Mach, Preconditioned Inverse Iteration for Hierarchical Matrices 25/28
Hierarchical (H-)Matrices PINVIT Numerical Results
BEM example, smallest eigenvalues
dense matrix, but approximable by an H-matrix ⇒ MATLAB eigsis expensive, precond.: H-inversion
Name ni error titi
ti−1
N(ni )N(ni−1)
BEM8 258 7.5498E-08 0.02
BEM16 1 026 2.0455E-07 0.16 8.00 24.18
BEM32 4 098 5.4044E-07 1.33 8.31 24.22
BEM64 16 386 1.8159E-06 11.90 8.95 23.99
BEM128 65 538 — 146.00 12.27 14.66
d = 6, c = 0.2, εeig = 10−4, N(ni ) = ni (log2 ni )CspCid
Time t only H-PINVIT (without H-inversion)
Max Planck Institute Magdeburg Thomas Mach, Preconditioned Inverse Iteration for Hierarchical Matrices 26/28
Hierarchical (H-)Matrices PINVIT Numerical Results
BEM example, smallest eigenvalues
dense matrix, but approximable by an H-matrix ⇒ MATLAB eigsis expensive, precond.: H-Cholesky decomp.
Name ni error titi
ti−1
N(ni )N(ni−1)
BEM8 258 8.2758E-08 0.02
BEM16 1 026 1.9546E-07 0.18 9.00 24.18
BEM32 4 098 5.2752E-07 0.95 5.28 24.22
BEM64 16 386 2.1935E-06 7.66 8.06 23.99
BEM128 65 538 — 33.20 4.33 14.66
d = 6, c = 0.2, εeig = 10−4, N(ni ) = ni (log2 ni )CspCid
Time t only H-PINVIT (without H-Cholesky decomposition)
Max Planck Institute Magdeburg Thomas Mach, Preconditioned Inverse Iteration for Hierarchical Matrices 26/28
Hierarchical (H-)Matrices PINVIT Numerical Results
BEM example, smallest eigenvalues
BEM8BEM16
BEM32BEM64
BEM12810−3
10−2
10−1
100
101
102
103
104
105
CP
Uti
me
ins
H-PINVIT(1,6) O(N(ni ))H-inversion O(N(ni ) log ni )
Max Planck Institute Magdeburg Thomas Mach, Preconditioned Inverse Iteration for Hierarchical Matrices 27/28
Hierarchical (H-)Matrices PINVIT Numerical Results
BEM example, smallest eigenvalues
BEM8BEM16
BEM32BEM64
BEM12810−3
10−2
10−1
100
101
102
103
104
105
CP
Uti
me
ins
H-PINVIT(1,6) O(N(ni ))H-inversion O(N(ni ) log ni )
MATLAB eigs
Max Planck Institute Magdeburg Thomas Mach, Preconditioned Inverse Iteration for Hierarchical Matrices 27/28
Hierarchical (H-)Matrices PINVIT Numerical Results
BEM example, smallest eigenvalues
BEM8BEM16
BEM32BEM64
BEM12810−3
10−2
10−1
100
101
102
103
104
105
CP
Uti
me
ins
H-PINVIT(1,6) O(N(ni ))H-Chol. decomp. O(N(ni ) log ni )
Max Planck Institute Magdeburg Thomas Mach, Preconditioned Inverse Iteration for Hierarchical Matrices 27/28
Hierarchical (H-)Matrices PINVIT Numerical Results
BEM example, smallest eigenvalues
BEM8BEM16
BEM32BEM64
BEM12810−3
10−2
10−1
100
101
102
103
104
105
CP
Uti
me
ins
H-PINVIT(1,6) O(N(ni ))H-Chol. decomp. O(N(ni ) log ni )
MATLAB eigs
Max Planck Institute Magdeburg Thomas Mach, Preconditioned Inverse Iteration for Hierarchical Matrices 27/28
Hierarchical (H-)Matrices PINVIT Numerical Results
Conclusions
Once we have inverted M ∈ H(T , k) finding the eigenvaluesby H-PINVIT is cheap and storage efficient.
For dense, data-sparse matrices, H-PINVIT can solveproblems where the MATLAB function eigs (implicitly,restarted Arnoldi/Lanczos iteration) is not applicable.
The folded spectrum method enables us to compute innereigenvalues, too.
Further details in our preprint MPIMD/11-01:http://www.mpi-magdeburg.mpg.de/preprints/.
Thank you for your attention.
Max Planck Institute Magdeburg Thomas Mach, Preconditioned Inverse Iteration for Hierarchical Matrices 28/28
Hierarchical (H-)Matrices PINVIT Numerical Results
Conclusions
Once we have inverted M ∈ H(T , k) finding the eigenvaluesby H-PINVIT is cheap and storage efficient.
For dense, data-sparse matrices, H-PINVIT can solveproblems where the MATLAB function eigs (implicitly,restarted Arnoldi/Lanczos iteration) is not applicable.
The folded spectrum method enables us to compute innereigenvalues, too.
Further details in our preprint MPIMD/11-01:http://www.mpi-magdeburg.mpg.de/preprints/.
Thank you for your attention.
Max Planck Institute Magdeburg Thomas Mach, Preconditioned Inverse Iteration for Hierarchical Matrices 28/28
Hierarchical (H-)Matrices PINVIT Numerical Results
Conclusions
Once we have inverted M ∈ H(T , k) finding the eigenvaluesby H-PINVIT is cheap and storage efficient.
For dense, data-sparse matrices, H-PINVIT can solveproblems where the MATLAB function eigs (implicitly,restarted Arnoldi/Lanczos iteration) is not applicable.
The folded spectrum method enables us to compute innereigenvalues, too.
Further details in our preprint MPIMD/11-01:http://www.mpi-magdeburg.mpg.de/preprints/.
Thank you for your attention.
Max Planck Institute Magdeburg Thomas Mach, Preconditioned Inverse Iteration for Hierarchical Matrices 28/28
Hierarchical (H-)Matrices PINVIT Numerical Results
Conclusions
Once we have inverted M ∈ H(T , k) finding the eigenvaluesby H-PINVIT is cheap and storage efficient.
For dense, data-sparse matrices, H-PINVIT can solveproblems where the MATLAB function eigs (implicitly,restarted Arnoldi/Lanczos iteration) is not applicable.
The folded spectrum method enables us to compute innereigenvalues, too.
Further details in our preprint MPIMD/11-01:http://www.mpi-magdeburg.mpg.de/preprints/.
Thank you for your attention.
Max Planck Institute Magdeburg Thomas Mach, Preconditioned Inverse Iteration for Hierarchical Matrices 28/28
Hierarchical (H-)Matrices PINVIT Numerical Results
Conclusions
Once we have inverted M ∈ H(T , k) finding the eigenvaluesby H-PINVIT is cheap and storage efficient.
For dense, data-sparse matrices, H-PINVIT can solveproblems where the MATLAB function eigs (implicitly,restarted Arnoldi/Lanczos iteration) is not applicable.
The folded spectrum method enables us to compute innereigenvalues, too.
Further details in our preprint MPIMD/11-01:http://www.mpi-magdeburg.mpg.de/preprints/.
Thank you for your attention.
Max Planck Institute Magdeburg Thomas Mach, Preconditioned Inverse Iteration for Hierarchical Matrices 28/28
Appendix
Adaptive H-Inversion [Grasedyck 2001]
Adaptive H-Inversion
Input: M ∈ H (TI×I ), c = c√‖M‖2
∈ R
Output: M−1H , with∥∥I −M−1H M
∥∥2< c
Compute M−1H with εlocal = c/(Csp(M) ‖M‖H2 )
δ0M :=∥∥I −M−1H M
∥∥H2/c
while δiM > 1 doCompute M−1H with εlocal = εlocal/δ
iM
δiM :=∥∥I −M−1H M
∥∥H2/c
end∥∥I −M−1H M∥∥2< c = c√
‖M‖2⇔
∥∥I −M−1H M∥∥M≤ c
Max Planck Institute Magdeburg Thomas Mach, Preconditioned Inverse Iteration for Hierarchical Matrices 27/28
Appendix
Adaptive H-Cholesky Decomposition
Adaptive H-Cholesky Decomposition
Input: M ∈ H (TI×I ), c = c√‖M‖2
∈ R
Output: LHLTH, with
∥∥∥I − L−TH L−1H M∥∥∥2< c
Compute LHLTH = M with εlocal = c/(Csp(M) ‖M‖H2 )
δ0M :=∥∥∥I − L−TH L−1H M
∥∥∥H2/c
while δiM > 1 doCompute LHL
TH = M with εlocal = εlocal/δ
iM
δiM :=∥∥∥I − L−TH L−1H M
∥∥∥H2/c
end∥∥∥I − L−TH L−1H M∥∥∥2< c = c√
‖M‖2⇔
∥∥∥I − L−TH L−1H M∥∥∥M≤ c
Max Planck Institute Magdeburg Thomas Mach, Preconditioned Inverse Iteration for Hierarchical Matrices 27/28
Appendix
Why is it called Preconditioned Inverse Iteration?
Inverse Iteration:
xi+1 = µ(xi )A−1xi ,
xi+1 = xi − A−1Axi︸ ︷︷ ︸=0
+µ(xi )A−1xi ,
xi+1 = xi − A−1 (Axi + µ(xi )xi ) .
Replace A−1 with the inexact solution by the preconditioner B:
xi+1 = xi − B−1 (Axi + µ(xi )xi ) .
Max Planck Institute Magdeburg Thomas Mach, Preconditioned Inverse Iteration for Hierarchical Matrices 28/28