Optimizing Least-Squares Rational Filters for Solving...

Optimizing Least-Squares Rational Filters for SolvingInterior Eigenvalue Problems

Jan Winkelmann, Edoardo Di Napoli{winkelmann,dinapoli}@aices.rwth-aachen.de

Aachen Institute for Advanced Studies in Computational Engineering Science

July 6, 2016

Jan Winkelmann (AICES) Optimizing Least-Squares Filters July 6, 2016 1 / 29

1 Introduction and Motivation

2 Least-Squares Filters

3 Comparison to State of the Art

4 Optimization of Least-Squares Filters

5 Conclusion

6 Constrained LSOP


Filtered Subspace Iteration

Consider the interior hermitian eigenvalue problem

Ax = λx , with A = AH ∈ Cn×n sparse and λ ∈ I = [λmin, λmax ]

Can be extended to generalized interior (non-hermitian) eigenproblems

Algorithm Sketch1 Filter: X̂n+1 := ρ(A)Xn

2 Rayleigh-Ritz:[Q,Λ] := eig(X̂H

n+1 · A · X̂n+1)

Xn+1 := X̂n+1 · Q

X0 initial guess of eigenvectorsρ(A) spectral projector

Example (Gauss filter)

−2 −1 0 1 2

0

0.2

0.4

0.6

0.8

1

Real axis

Filte

rva

lue

Π(x)Gauss


Rational filter methods

Classic rational filter methods use numerical integration of a contour

ρ(A)X =12πı

∮C

(zI − A)−1Xdz ≈N∑i=1

ωi (zi I − A)−1X

We can use any function that yieldscheap system solves. For Example:

ρ(A) =N∑i=1

αi (zi I − A)−1

Previous WorkSS-Method 1

FEAST 2

Zolotarev Filters 3

Least-Squares 4,5


Rational filter methods

Classic rational filter methods use numerical integration of a contour

ρ(A)X =12πı

∮C

(zI − A)−1Xdz ≈N∑i=1

ωi (zi I − A)−1X

We can use any function that yieldscheap system solves. For Example:

ρ(A) =N∑i=1

αi (zi I − A)−1

Previous WorkSS-Method 1

FEAST 2

Zolotarev Filters 3

Least-Squares 4,5

1Sakurai, Sugiura. Journal of Computational and Applied Mathematics; 20032Polizzi. Physical Review B; 20093Güttel, et.al. SIAM Journal on Scientific Computing; 20154Barel. Linear Algebra and its Applications; 20155Xi, Saad, Y. Preprint; 2015


Least-Squares based filters

The idea:

minαi ,zi 1≤i≤N

∫ ∞−∞

∣∣∣∣∣∏(x)−N∑i=1

αi

zi − x

∣∣∣∣∣2

dx

Note that:∏(x) is the the indicator

function on [−1, 1].Coefficients αi , and poles zi arecomplex-valued.We optimize coefficients andpoles of the rational function.

Potential Benefits:Faster convergenceIncreased flexibilityProblem specific filters


First Results

0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2−0.2

0

0.2

0.4

0.6

0.8

1

1.2

Real axis

Filte

rva

lue

GaussLSOPΠ


First Results

0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 210−7

10−6

10−5

10−4

10−3

10−2

10−1

100

Real axis

Filte

rva

lue

GaussLSOP


Our Goals

Filtered Subspace Iteration is sensitive to the spectrumAppropriate choice of subspace size and degree requiredDifferent filter types have different strengthsEven with some prior knowledge:Optimal choice of parameters is difficult.This is a difficult problem ⇒ We will not change this

The filter should be "smart"Take advantage of the spectrumIdeally, this would work automatically


State of the Art Filters

Gauss FilterFrom Gauss(-Legendre) QuadratureBenefits from larger search subspace (≈ 1.5)

Zolotarev FilterFrom Cauer, Elliptical, or Zolotarev FiltersDoes not benefit from larger subspacesMay have better convergence or load-balancing

Choosing optimal filter and parameters is difficultFEAST 3.0 as Benchmark toolAll Timings are for single threaded execution

Maximum of 40 iterationsTermination criterion: Residual ≤ 10−14

Filter–type, –degree and search subspace as indicated


Filter "Comparison", 4 Poles per Quadrant

0 0.5 1 1.5 2 2.5 310−9

10−8

10−7

10−6

10−5

10−4

10−3

10−2

10−1

100

Real axis

Filte

rva

lue

ZolotarevLSOPGauss


2D FEM: CNT Spectrum Visualization

10 20 30 40 50 60 70 80 90 100

−60

−40

−20

0

Arbitrary Index

Eige

nval

ue

Eigenvaluesλ minλ max


2D FEM: CNT, 4 Poles per Quadrant

100 120 140 1600

10

20

30

% of target subspace

Iter

atio

ns

GaussZolotarev

LSOP

100 120 140 1600

10

20

30


Tim

e(s)

GaussZolotarev

LSOP

Gauss filter benefits from larger subspacesLeast-Squares filter requires less iterations than ZolotarevSomewhat benefits from larger subspace


2D FEM: CNT, 1 Pole per Quadrant

100 120 140 1600

10

20

30

40


Iter

atio

ns

GaussLSOP

100 120 140 1600

10

20

30


Tim

e(s)

GaussLSOP

Zolotarev does not converge within 40 iterations


2D FEM: CNT, Complete Timings

100 120 140 160

10

20

30


Tim

e(s)

Gauss-8Zolotarev-8LSOP-8Gauss-4

Zolotarev-4LSOP-4Gauss-1LSOP-1



100 120 140 160

10

20

30


Tim

e(s)





100 120 140 160

10

20

30


Tim

e(s)




3D FEM: Caffeinep2 Spectrum Visualization

20 40 60 80 100 120 140 160 180 200−1.2

−1

−0.8

−0.6

−0.4

−0.2

0

·10−16

Arbitrary Index

Eige

nval

ue

Eigenvaluesλ minλ max


3D FEM: Caffeinep2, 4 Poles per Quadrant

100 120 140 160

10

15

20


Iter

atio

ns

ZolotarevLSOP

100 120 140 160600

800

1,000

1,200


Tim

e(s)

ZolotarevLSOP

Gauss does not converge(recall the large cluster at the edge of the interval)Zolotarev outperforms Least-Squares filters for any number of poles


Recall: Zolotarev vs LSOP Filters

0.9 0.95 1 1.05 1.1 1.15 1.210−7

10−6

10−5

10−4

10−3

10−2

10−1

100

Real axis

Filte

rva

lue

ZolotarevLSOP


Wrap-up: Non Problem-Specific Filters

Results for unconstrained Least-Squares filtersWorks well even with low degreesThis is not the take away today

The take-away should be:1 Gather spectrum information (separate topic)2 Use appropriate filter and parameters3 Facilitated via custom Least-Squares filters

Least-Squares filters are very flexibleConstraints for asymmetry, steepness, decay possibleFast generation of filters possible


1 Introduction and Motivation

2 Least-Squares Filters

3 Comparison to State of the Art

4 Optimization of Least-Squares Filters

5 Conclusion

6 Constrained LSOP


Approximating Π(x)

minγi ,zi 1≤i≤N/4

∫ ∞−∞

∣∣∣∏(x)− f (x , z , γ)∣∣∣2 dx , with

f (x , z , γ) =

N/4∑i=1

γizi − x

+γi

zi − x− γi

zi + x− γi

zi + x

Reflection Symmetry: f (−x , z , γ) = f (x , z , γ)

Complex Conjugation: f (x , z , γ) = f (x , z , γ)

Symmetries force evenness and real-nessAnalogous to existing integration rulesFor simplicity we disallow poles on the axesAsymmetric filters lack reflection symmetry

We require cheap computation of:Least-Squares ResidualGradients w.r.t. αi , zi


Approximating Π(x)


∫ ∞−∞

∣∣∣∏(x)− f (x , z , γ)∣∣∣2 dx , with

f (x , z , γ) =

N/4∑i=1

γizi − x

+γi

zi − x− γi

zi + x− γi

zi + x

Reflection Symmetry: f (−x , z , γ) = f (x , z , γ)

Complex Conjugation: f (x , z , γ) = f (x , z , γ)

Symmetries force evenness and real-nessAnalogous to existing integration rulesFor simplicity we disallow poles on the axesAsymmetric filters lack reflection symmetry

We require cheap computation of:Least-Squares ResidualGradients w.r.t. αi , zi


Calculating Least-Squares Residuals

∫ ∞−∞

w(x)

∣∣∣∣∣∣∏

(x)−N/4∑i=1

γizi − x

+γi

zi − x− γi

zi + x− γi

zi + x

∣∣∣∣∣∣2

dx

An appropriate w(x) simplifies the calculation:

w(x) =

0, if |x | > a

β, if |x | ≤ 11, else

With β ≈ 0.4, and a ≈ 5.Exact 1.0 in [−1, 1] not required⇒ Small β increases sharpness at -1,1’Cut-off’ after a ⇒ Definite integrals

We can provide a (matrix) form for the residual


Optimization

We can derive Gradients w.r.t. zi , and αi

Use of symmetries ⇒ Cheaper GradientsFast filter generation via Levenberg-MarquardtDetails in forthcoming publication

Descent methods require a starting positionVery relevant for non-convex optimization


Local Minima for 4 Poles per Quadrant

0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2

0

0.5

1

Real axis

Filte

rva

lue

The filter is non-convex in the poles ziResulting filter depends on the starting positionMany approaches do not yield good results


Local Minima for 4 Poles per Quadrant

0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2

0

0.5

1

Real axis

Filte

rva

lue

The filter is non-convex in the poles ziResulting filter depends on the starting positionMany approaches do not yield good results


Use Existing Methods as Starting Points

Existing filters already have good residuals

1 2 3 4 5 6 7 8 9 10

100

10−2

10−4

10−6

Poles per Quadrant

Leas

t-Sq

uare

sRes

idua

l

GaussZolotarev

⇒ Good results as starting positionsBoth produce the same results for ≤ 7 poles per quadrantSo far, these are the best residuals we know of


Use Existing Methods as Starting Points

Existing filters already have good residuals

1 2 3 4 5 6 7 8 9 10

100

10−2

10−4

10−6

Poles per Quadrant

Leas

t-Sq

uare

sRes

idua

l

GaussZolotarev

LSOP (Gauss)LSOP (Zolotarev)

⇒ Good results as starting positionsBoth produce the same results for ≤ 7 poles per quadrantSo far, these are the best residuals we know of


Conclusion

Filters are important for contour-based eigensolversNo single filter type is best

We present Least-Squares based filters:Rich framework for filter generationPotential for custom filtersCompetetive results for unconstrained Least-Squares filtersFast generation of filters

No theoretical insights on generated filters(i.e. worst-case convergence ratio)Systematic comparisons of filters is tricky


Future Work

Parallelism and convergence of linear system solves

Further details in forthcoming publication:

More constrained optimization of filters

Systematic comparisons of filters


The End

Thank you for your attentionQuestions?

Financial support from the DeutscheForschungsgemeinschaft (GermanResearch Association) through grantGSC 111 is gratefully acknowledged.


Constrained LSOP

Poles near the real axis ⇒ Almost real shift zi of MatrixN/4∑i=1

γiIzi − A

+γi

I zi − A− γi

Izi + A− γi

I zi + A

X = B

The shifted (zi I − A) may become (almost) singular

Larger imaginary part on the poles can prevent this "problem"


F (x), such that

=(zi ) ≥ 0.1, 1 ≤ i ≤ N/4

F (x) =

∫ ∞−∞

w(x)

∣∣∣∣∣∣∏

(x)−N/4∑i=1

γizi − x

+γi

zi − x− γi

zi + x− γi

zi + x

∣∣∣∣∣∣2

dx


Constrained LSOP

Poles near the real axis ⇒ Almost real shift zi of MatrixN/4∑i=1

γiIzi − A

+γi

I zi − A− γi

Izi + A− γi

I zi + A

X = B

The shifted (zi I − A) may become (almost) singular

Larger imaginary part on the poles can prevent this "problem"


F (x), such that

=(zi ) ≥ 0.1, 1 ≤ i ≤ N/4

F (x) =

∫ ∞−∞

w(x)

∣∣∣∣∣∣∏

(x)−N/4∑i=1

γizi − x

+γi

zi − x− γi

zi + x− γi

zi + x

∣∣∣∣∣∣2

dx


Zolotarev vs constrained LSOP, 4 Poles per Quadrant

0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2−0.2

0

0.2

0.4

0.6

0.8

1

1.2Zolotarev

LSOP



0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 210−7

10−6

10−5

10−4

10−3

10−2

10−1

100

ZolotarevLSOP



0.2 0.4 0.6 0.8 1

10−3

10−2

10−1

100

Zolotarev PolesLSOP Poles


CAFF timings

100 120 140 160

600

800

1,000

1,200

1,400

1,600


Tim

e(s)

Zolotarev-3LSOP-3

Zolotarev-4LSOP-4

Zolotarev-8LSOP-8


Date post:	05-Aug-2019
Category:	Documents
Upload:	phungnhan
View:	212 times
Download:	0 times

Optimizing Least-Squares Rational Filters for Solving...

Documents