RBF{QR Radial basis function approximationRadial basis function approximation PhD student course in...

transcript

RBF–QROutline

Global RBFsRBF limits

Stable methods

Convergence theory

RBF-PUMTheoretical resultsNumerical results

RBF-FD

Radial basis function approximationPhD student course in Approximation Theory

Elisabeth Larsson

2017-09-18

E. Larsson, 2017-09-18 (1 : 77)

RBF–QROutline

Stable methods

Convergence theory

RBF-FD

Outline

Global RBF approximationRBF limits

Stable evaluation methods

Convergence theory

RBF partition of unity methodsTheoretical resultsNumerical results

RBF-FD

E. Larsson, 2017-09-18 (2 : 77)

RBF–QROutline

Stable methods

Convergence theory

RBF-FD

Short introduction to (global) RBF methods

Basis functions: φj(x) = φ(‖x − x j‖). Translates of onesingle function rotated around a center point.

Example: Gaussiansφ(εr) = exp(−ε2r2)

Approximation:sε(x ) =

∑Nj=1 λjφj(x )

Collocation:sε(x i ) = fi ⇒ Aλ = f ε=3ε=1/3ε=1

Advantages:

• Flexibility with respect to geometry.

• As easy in d dimensions.

• Spectral accuracy / exponential convergence.

• Continuosly differentiable approximation.

E. Larsson, 2017-09-18 (3 : 77)

RBF–QROutline

Stable methods

Convergence theory

RBF-FD

Commonly used RBFs

Global infinitely smoothGaussian exp(−ε2r2), ε > 0

(Inverse) multiquadric (1 + ε2r2)β/2, ε > 0, |β| ∈ N

Global piecewise smoothPolyharmonic spline (odd) |r |2m−1, m ∈ NPolyharmonic spline (even) r2m log(r), m ∈ NMatern/Sobolev rνKν(r), ν > 0C 2 Matern (1 + r) exp(−r), v = 3/2

Compactly supported Wendland functionsC 2 and pos def for d ≤ 3, (1− r

ρ)4+(4 r

ρ + 1), ρ > 0

C 2 and pos def for d ≤ 5, (1− rρ)4

+(5 rρ + 1), ρ > 0

E. Larsson, 2017-09-18 (4 : 77)

RBF–QROutline

Stable methods

Convergence theory

RBF-FD

demo1.m(RBF interpolation in 1-D)

E. Larsson, 2017-09-18 (5 : 77)

RBF–QROutline

Stable methods

Convergence theory

RBF-FD

Observations from the results of demo1.m

I As N grows for fixed ε, convergence stagnates.

I As ε decreases for fixed N, the error blows up.

I λmin = −λmax means cancellation.

I Coefficients λ→∞ means that cond(A)→∞.

I For small ε, the RBFs are nearly flat, and almostlinearly dependent. That is, they form a bad basis.

10 20 30

10−5

rε = 1

10 20 30

ε = 1

10−5

N = 30

E. Larsson, 2017-09-18 (6 : 77)

RBF–QROutline

Stable methods

Convergence theory

RBF-FD

Why is it interesting to use small values of ε?

Driscoll & Fornberg [DF02]

Somewhat surprisingly, in 1-D for small ε

s(x , ε) = PN−1(x) + ε2PN+1(x) + ε4PN+3(x) + · · · ,

where Pj is a polynomial of degree j and PN−1(x) is theLagrange interpolant.

Implications

I It can be shown that cond(A) ∼ O(Nε−2(N−1)), butthe limit interpolant is well behaved.

I It is the intermediate step of computing λ that isill-conditioned.

I By choosing the corresponding nodes, the flat RBFlimit reproduces pseudo-spectral methods.

I This is a good approximation space.E. Larsson, 2017-09-18 (7 : 77)

RBF–QROutline

Stable methods

Convergence theory

RBF-FD

The multivariate flat RBF limit

Larsson & Fornberg [LF05], Schaback [Sch05]In n-D the flat limit can either be

s(x , ε) = PK (x) + ε2PK+2(x) + ε4PK+4(x) + · · · ,

((K − 1) + d

)< N ≤

(K + d

)and PK is a

polynomial interpolant or

s(x , ε) = ε−2qPM−2q(x) + ε−2q+2PM−2q+2(x) + · · ·+ PM(x) + ε2PM+2(x) + ε4PM+4(x) + · · · .

The questions of uniqueness and existence are connectedwith multivariate polynomial uni-solvency.

Schaback [Sch05]

Gaussian RBF limit interpolants always converge to thede Boor/Ron least polynomial interpolant.

E. Larsson, 2017-09-18 (8 : 77)

RBF–QROutline

Stable methods

Convergence theory

RBF-FD

The multivariate flat RBF limit: Divergence

Necessary condition: ∃ Q(x) of degree N0 such thatQ(x j) = 0, j = 1, . . . ,N.Then divergence as ε−2q may occur, whereq = b(M − N0)/2c and M = min non-degenerate degree.

Points Q N0 Basis M qx − y 1 1, x , x2,

x3, x4, x55 2

x2 − y − 1 2 1, x , y , xy ,y2xy2

x2 + y2 − 1 2 1, x , y , x2, xy ,x3, x2y , x4

Divergence actually only occurs for the first case as ε−2.

E. Larsson, 2017-09-18 (9 : 77)

RBF–QROutline

Stable methods

Convergence theory

RBF-FD

The multivariate flat RBF limit, contd

Schaback [Sch05], Fornberg & Larsson [LF05]

Example: In two dimensions, the eigenvalues of A followa pattern: µ1 ∼ O(ε0), µ2,3 ∼ O(ε2), µ4,5,6 ∼ O(ε4),. . .

In general, there are

(k + n − 1n − 1

)= (k+1)···(k+n−1)

(n−1)!

eigenvalues µj ∼ O(ε2k) in n dimensions.

Implications

I There is an opportunity for pseudo-spectral-likemethods in n-D.

I There is no amount of variable precision that willsave us.

I For “smooth” functions, a small ε can lead to veryhigh accuracy.

E. Larsson, 2017-09-18 (10 : 77)

RBF–QROutline

Stable methods

Convergence theory

RBF-FD

Multivariate interpolation

Theorem (Mairhuber–Curtis)

For a domain Ω ⊂ Rd , d ≥ 2 that has an interior point,there is no basis of continuous functions f1(x ), . . . fN(x ),N ≥ 2 such that an interpolation matrix A = fj(x i )Ni ,j=1

is guaranteed to be non-singular (no Haar basis).

Proof.Let two of the points x i and xk change places along aclosed continuous path in Ω. When the two points havechanged places, two rows in A are interchanged, anddet(A) has changed sign. Then det(A) = 0 somewherealong the path.

I For RBF approximation, A = φ(‖x i − x j‖)Ni ,j=1. Iftwo points change place, two rows and two columnsare swapped. Determinant does not change sign.

E. Larsson, 2017-09-18 (11 : 77)

RBF–QROutline

Stable methods

Convergence theory

RBF-FD

Positive definite functions

Definition (Positive definite function)

A real valued continuous function Φ is positive definite onRd ⇔ it is even and

N∑j=1

N∑k=1

cjckΦ(x j − xk) ≥ 0

for any parwise distinct points x1, . . . , xN ∈ Rd , cj ∈ R.

Theorem (Bochner 1933)

A function Φ ∈ C (Rd) is positive definite on Rd ⇔ it isthe Fourier transform of a finite non-negative Borelmeasure µ on Rd

Φ(x ) =1√

(2π)d

e−ix ·ωdµ(ω).

E. Larsson, 2017-09-18 (12 : 77)

RBF–QROutline

Stable methods

Convergence theory

RBF-FD

Bochner Theorem contd

Partial proof

N∑j=1

N∑k=1

cjckΦ(x j − xk) =

(2π)d

N∑j=1

N∑k=1

e−i(x j−xk )·ωdµ(ω)

(2π)d

N∑j=1

cje−ix j ·ω

N∑k=1

ckeixk ·ω

dµ(ω)

(2π)d

∣∣∣∣∣∣N∑j=1

cje−ix j ·ω

∣∣∣∣∣∣2

dµ(ω) ≥ 0.

E. Larsson, 2017-09-18 (13 : 77)

RBF–QROutline

Stable methods

Convergence theory

RBF-FD

Example

The Gaussian is positive definite in any dimension

e−ε2‖x‖2

(2π)d

2ε)de−‖ω‖

2/(4ε2)e ix ·ωdω

Theorem (Schoenberg 1938)

A cont function ϕ : [0,∞)→ R is strictly pos def andradial on Rd for all d ⇔

ϕ(r) =

∫ ∞0

e−r2t2

dµ(t),

where µ is a finite non-negative Borel measure notconcentrated at the origin.

E. Larsson, 2017-09-18 (14 : 77)

RBF–QROutline

Stable methods

Convergence theory

RBF-FD

Results and consequences for RBFapproximation

I Non-singularity of RBF interpolation is guaranteedfor distinct node points and strictly pos def functionssuch as the Gaussian and the inverse multiquadric.

I There are no oscillatory or compactly supportedRBFs that are strictly pos def for all d .Because φ(r0) = 0 breaks theorem, cf. Bessel and Wendland.

I Non-singularity/positive definiteness of interpolationmatrix holds also for conditionally positive definiteRBFs augmented with polynomials.Micchelli [Mic86], cf. multiquadric RBFs

E. Larsson, 2017-09-18 (15 : 77)

RBF–QROutline

Stable methods

Convergence theory

RBF-FD

Tensor product vs multivariate basis

Tensor product basis

s(x ) =n∑

n∑j=0

cijpi (x1)pj(x2)

Number of unknowns NT = (n + 1)d .

Multivariate basisThinking in terms of polynomials, a multivariatepolynomial space of degree n has dimension

(n + dd

(n + 1) · · · (n + d)

Degrees of freedom for n = 8:

d 1 2 3

NT 9 81 729

NM 9 45 165E. Larsson, 2017-09-18 (16 : 77)

RBF–QROutline

Stable methods

Convergence theory

RBF-FD

demo2.m(Conditioning and errors)

E. Larsson, 2017-09-18 (17 : 77)

RBF–QROutline

Stable methods

Convergence theory

RBF-FD

Comments on the results of demo2

I Error is small where condition is high and vice versa.

I Interesting region only reachable with stable method.

I Best results for small ε.

171615141312111098765432118

RBF−Direct: log10

( cond2(A) )

181234567891011121314151617

−2.5 −2 −1.5 −1 −0.5 0 0.5 110

RBF−QR: log10

|s(x)−f(x)|

−2.5 −2 −1.5 −1 −0.5 0 0.5 110

Teaser: Conditioning for RBF–QR is perfect...

E. Larsson, 2017-09-18 (18 : 77)

RBF–QROutline

Stable methods

Convergence theory

RBF-FD

The Contour-Pade method

Fornberg & Wright [FW04]

I Think of ε as a complex variable.

I The limit ε = 0 is a removable singularity.

I Complex ε for which A is singular lead to poles.

I Pole location only depend on the location of nodes.

Example

I Evaluate f (ε) = 1−cos(ε)ε2

I Numerically unstable.

I Removable singularity at 0.

I Compute f (0) as average off (ε) around “safe path”.

Bad region

Safe path

Target point

E. Larsson, 2017-09-18 (19 : 77)

RBF–QROutline

Stable methods

Convergence theory

RBF-FD

The Contour-Pade method: Algorithm

I Compute s(x , ε) = AeA−1f at M points around a

“safe path” (circle).I Inverse FFT of the M values gives a Laurent

expansion

u(x) = . . .+ s−2(x)ε−4 + s−1(x)ε−2︸︷︷︸Needs to be converted

+s0(x)+s1(x)ε2+s2(x)ε4+. . .

I Convert the negative power expansion into Padeform and find the correct number of poles and theirlocations

s−1ε−2 + s−2ε

−4 + . . . =p1ε−2 + · · ·+ pmε

1 + q1ε−2 + · · ·+ qnε−2n.

I Evaluate u(x) using Taylor + Pade for any ε insidethe circle.

E. Larsson, 2017-09-18 (20 : 77)

RBF–QROutline

Stable methods

Convergence theory

RBF-FD

The Contour-Pade method: Results

10−3

10−2

10−1

10−10

10−5

)−f(

Usual method 1e−9

New method 1e−9

10−3

10−2

10−1

10−10

10−5

)−f(

Quad precision 9e−12Usual method 1e− 9New method 1e−10

I Stable computation for all ε with Contour-Pade.

I Limited number of nodes, otherwise general.

I Expensive to compute A−1 at M points.

I Tricky to find poles.

I Modern efficient version RBF-RA [WF17].

E. Larsson, 2017-09-18 (21 : 77)

RBF–QROutline

Stable methods

Convergence theory

RBF-FD

Expansions of (Gaussian) RBFs

On the surface of the sphere

Hubbert & Baxter [BH01]

For different RBFs there are expansions

φ(‖x − xk‖) =∞∑j=0

ε2jj∑

m=−jcj ,mY

mj (x )

Cartesian space, polynomial expansion

For Gaussians

φ(‖x − xk‖) = e−ε2(x−xk )·(x−xk )

= e−ε2(x ·x )e−ε

2(xk ·xk )e2ε2(x ·xk )

= e−ε2(x ·x )e−ε

2(xk ·xk )∞∑j=0

ε2j 2j

j!(x · xk)j

E. Larsson, 2017-09-18 (22 : 77)

RBF–QROutline

Stable methods

Convergence theory

RBF-FD

Expansions of (Gaussian) RBFs contd

Mercer expansion (Mercer 1909)

For a positive definite kernel K (x , xk) = φ(‖x − xk‖),there is an expansion

φ(‖x − xk‖) =∞∑j=0

λjϕj(x )ϕj(xk),

where λj are positive eigenvalues, and ϕj(x ) areeigenfunctions of an associated compact integraloperator.

E. Larsson, 2017-09-18 (23 : 77)

RBF–QROutline

Stable methods

Convergence theory

RBF-FD

The RBF-QR method on the sphere

Fornberg & Piret [FP07]

φ(‖x − xk‖) =∞∑j=0

ε2jj∑

m=−jcj ,mY

mj (x )

The number of SPH functions/power matches the RBFeigenvalue pattern on the sphere.

If we collect RBFs and expansion functions in vectors,and coefficients in the matrix B, we have a relation

Φ(x ) = B · Y = Q · E · R · Y (x )

The new basis Ψ(x ) = R · Y (x ) spans the same space asΦ(x ), but the ill-conditioning has been absorbed in E .

E. Larsson, 2017-09-18 (24 : 77)

RBF–QROutline

Stable methods

Convergence theory

RBF-FD

The RBF-QR method in Cartesian space

Fornberg, Larsson, Flyer [FLF11]

The expansion of the Gaussian

φ(‖x − xk‖) = e−ε2(x ·x )e−ε

2(xk ·xk )∞∑j=0

ε2j 2j

j!(x · xk)j

+ The number of expansion functions for each powerof ε matches the eigenvalue pattern in A.

− The expansion functions are the monomials.

Better expansion functions in 2-D

I Change to polar coordinates.

I Trigs in the angular direction are perfect.

I Necessary to preserve powers of ε ⇒Partial conversion to Chebyshev polynomials.

E. Larsson, 2017-09-18 (25 : 77)

RBF–QROutline

Stable methods

Convergence theory

RBF-FD

The RBF-QR method in Cartesian space contd

New expansion functions

T cj ,m(x) = e−ε

2r2r2mTj−2m(r) cos((2m + p)θ),

T sj ,m(x) = e−ε

2r2r2mTj−2m(r) sin((2m + p)θ),

Matrix form of factorized expansion

Express Φ(x ) = (φ(‖x − x1‖), . . . , φ(‖x − xN‖))T interms of expansion functions T (x ) = (T c

0,0,Tc1,0, . . .)

Φ(x ) = C · D · T (x ),

where cij is O(1) and D = diag(O(ε0, ε2, ε2, ε4, . . .)).

Note that C has an infinite number of columns etc.

E. Larsson, 2017-09-18 (26 : 77)

RBF–QROutline

Stable methods

Convergence theory

RBF-FD

The RBF-QR method in Cartesian space contd

The QR part

The coefficient matrix C is QR-factorized so that

Φ(x ) = Q ·[R1 R2

] [ D1 00 D2

]·T (x ), where R1 and

D1 are of size (N × N).

The change of basis

Make the new basis (same space) close to T

Ψ(x ) = D−11 R−1

1 QHΦ(x ) =[I R

]· T (x ).

Analytical scaling of R = D−11 R−1

1 R2D2

Any power of ε in D1 ≤ any power of ε in D2 ⇒Scaling factors O(ε0) or smaller, truncation is possible.

E. Larsson, 2017-09-18 (27 : 77)

RBF–QROutline

Stable methods

Convergence theory

RBF-FD

demo3.m(RBF interpolation in 2-D with and without RBF–QR)

E. Larsson, 2017-09-18 (28 : 77)

RBF–QROutline

Stable methods

Convergence theory

RBF-FD

Stable computation as ε → 0 and N →∞The RBF-QR method allows stable computations forsmall ε. (Fornberg, Larsson, Flyer [FLF11])

Consider a finite non-periodic domain.

Theorem (Platte, Trefethen, and Kuijlaars [PTK11]):

Exponential convergence on equispaced nodes ⇒exponential ill-conditioning.

Solution #1:

Cluster nodes towards the domain boundaries.

E. Larsson, 2017-09-18 (29 : 77)

RBF–QROutline

Stable methods

Convergence theory

RBF-FD

An RBF-QR example with clustered nodes in anon-trivial domain

f (x , y) = exp(−(x − 0.1)2 − 0.5y2)N=793 node pointsCosine-stretching towards each boundaryMaximum error 2.2e-10

E. Larsson, 2017-09-18 (30 : 77)

RBF–QROutline

Stable methods

Convergence theory

RBF-FD

demo4.m(RBF interpolation in 2-D with clustered nodes)

E. Larsson, 2017-09-18 (31 : 77)

RBF–QROutline

Stable methods

Convergence theory

RBF-FD

Brief survey of Mercer based methods

Fasshauer & McCourt [FM12]

Eigenvalues and eigenfunctions in 1-D can be chosen as

√α2

α2 + δ2 + ε2

)n−1

φn = γne−δ2x2

Hn−1(αβx),

where β =(

, γn =√

β2n−1Γ(n) , δ2 = α2

2 (β2 − 1).

I Eigenfunctions are orthogonal in a weighted norm.

I The QR-step is similar to that of previous methods.

I Tensor product form is used in higher dimensions ⇒The powers of ε do not match the eigenvalues of A.

I New parameter α to tune.

E. Larsson, 2017-09-18 (32 : 77)

RBF–QROutline

Stable methods

Convergence theory

RBF-FD

Brief survey of Mercer based methods contd

De Marchi & Santin [DMS13]

I Discrete numerical approximation of eigenfunctions.

I W diagonal matrix with cubature weights.Perform SVD

√W · A ·

√W = Q · Σ2 · QT .

The eigenbasis is given by√W−1 · Q · Σ.

I Rapid decay of singular values ⇒ Basis can betruncated ⇒ Low rank approximation of A.

De Marchi & Santin [DMS15]

I Faster: Lanczos algorithm on Krylov space K(A, f ).

I Eigenfunctions through SVD of Hm from Lanczos.

I Computationally efficient.

I Basis depends on f . Potential trouble for f 6∈ NK (X )

For details it is a good idea to ask the authors :-)E. Larsson, 2017-09-18 (33 : 77)

RBF–QROutline

Stable methods

Convergence theory

RBF-FD

Differentiation matrices and RBF-QR

Larsson, Lehto, Heryudono, Fornberg [LLHF13]

Let uX be an RBF approximation evaluated at the nodes.

To compute uY evaluated at the set of points Y , we useAλ = uX ⇒ λ = A−1uX to get

uY = AYλ = AYA−1uX

where AY (i , j) = φj(yi ).

To instead evaluate a differential operator applied to u,

uY = ALYA−1uX ,

where ALY (i , j) = Lφj(yi ).

To do the same thing using RBF–QR, replace φj with ψj .

E. Larsson, 2017-09-18 (34 : 77)

RBF–QROutline

Stable methods

Convergence theory

RBF-FD

Solving PDEs with RBFs/RBF-QR

Domain defined by: rb(θ) = 1 + 110 (sin(6θ) + sin(3θ)).

∆u=f (x ), x ∈ Ω,u=g(x ), x on ∂Ω,

Solution: u(x ) = sin(x21 + 2x2

2 )− sin(2x21 + (x2 − 0.5)2).

Collocation:(A∆X iA

)(uiXubX

(f iXgbX

)Evaluation:uY = AYA

−1X uX

Domain + nodesE. Larsson, 2017-09-18 (35 : 77)

RBF–QROutline

Stable methods

Convergence theory

RBF-FD

demo5.m(Solving the Poisson problem in 2-D using RBFs)

E. Larsson, 2017-09-18 (36 : 77)

RBF–QROutline

Stable methods

Convergence theory

RBF-FD

Reproducing Kernel Hilbert spaces andoptimality

Let N (Ω) be a real Hilbert space of functions u : Ω→ R,where Ω ⊆ Rd with inner product (·, ·)N (Ω).

Consider an RBF as a kernel K (x , y). The followingholds

(i) K (·, x ) ∈ N (Ω) for all x ∈ Ω.

(ii) (u,K (·, x ))N (Ω) = u(x ).

Let I (u) be the interpolant of u ∈ N (Ω). Then

‖I (u)‖N (Ω) ≤ ‖u‖N (Ω)

Consider a finite dimensional subspace N (X ) of thenative space N (Ω). Then

(I (u)− u, v)N (Ω) = 0 for all v ∈ N (X ).E. Larsson, 2017-09-18 (37 : 77)

RBF–QROutline

Stable methods

Convergence theory

RBF-FD

Ingredients for exponential convergenceestimates

I Dependence on geometry through interior coneconditions.Approximation quality depends on boundary shape.

I General sampling inequalities based on polynomialapproximation.These tell us how much a smooth error can grow between

nodes.

I Embedding constants relating Native spaces toSobolev spaces.These are needed to go from algebraic to exponential

estimates.

E. Larsson, 2017-09-18 (38 : 77)

RBF–QROutline

Stable methods

Convergence theory

RBF-FD

Interior cone conditions

Definition (Interior cone condition)

A domain Ω ⊂ Rd satisfies an interior cone conditionwith radius r and angle θ if every x ∈ Ω is the vertex ofsuch a cone that is contained entirely within Ω.

Definition (Star shaped)

A domain Ω ⊂ Rd is star shaped with respect to B(xc , r)if for every x ∈ Ω, the convex hull of x and B(xc , r) isentirely enclosed in Ω.

Example

A star shaped domain wrt B(xc , r),enclosed by B(xc ,R) satisfiesan interior cone condition withradius r and angle θ = 2 arcsin( r

Narcowich, Ward, Wendland [NWW05]E. Larsson, 2017-09-18 (39 : 77)

RBF–QROutline

Stable methods

Convergence theory

RBF-FD

General sampling inequalitites

Rieger & Zwicknagl [RZ10]

Bound derivatives of u through polynomial bounds

Detailed computations with averaged Taylor polys

‖Dαu‖Lq(Ω) ≤C kS δ

k−d( 1p− 1

(k − |α|)!(δ−|α|Ω + h−|α|)|u|W k

p (Ω)

+ 2δdq

Ωh−|α|‖u‖`∞(X ),

where δΩ is the diameter of Ω, h is the fill distance(largest ball empty of nodes from X ), and the constantCS depends on d , p, and θ, 1 ≤ p <∞, 1 ≤ q ≤ ∞.

Fill distance must be small enough and k large enough.E. Larsson, 2017-09-18 (40 : 77)

RBF–QROutline

Stable methods

Convergence theory

RBF-FD

Embedding constants

Assume there are embedding constants, for all k such that

‖u‖W kp (Ω) ≤ E (k)‖u‖H(Ω)

for some space H(Ω) of smooth functions. Furtherassume that E (k) ≤ C k

Ek(1−ε)k , for ε,CE > 0.

Then, the general sampling inequality can be rewritten as

‖Dαu‖Lq(Ω) ≤ eC log(h)√

h ‖u‖H(Ω) + 2δdq

Ωh−|α|‖u‖`∞(X ),

where C = ε√c0/4 and c0 = min1, r sin θ

4(1+sinθ).We are still considering star shaped domains.

E. Larsson, 2017-09-18 (41 : 77)

RBF–QROutline

Stable methods

Convergence theory

RBF-FD

Lipschitz domains

For a general Lipschitz domain that satisfies a uniforminterior cone condition, we create a cover of Ω consistingof star shaped subdomains.

This affects the terms in front of the norms, but not theessentials.

For E (k) ≤ C kEk

(1−ε)k

‖Dαu‖Lq(Ω) ≤ eC log(h)√

h ‖u‖H(Ω) + C2h−|α|‖u‖`∞(X ).

For E (k) ≤ C kEk

sk , s ≥ 1

‖Dαu‖Lq(Ω) ≤ eC

h1/(1+s) ‖u‖H(Ω) + C2h−|α|‖u‖`∞(X ).

E. Larsson, 2017-09-18 (42 : 77)

RBF–QROutline

Stable methods

Convergence theory

RBF-FD

Embedding constants for kernel spaces

Fourier characterization of spaces

NK (Rd) =

u ∈ C (Rd) ∩ L2(Rd) : ‖u‖2

|u(ω)|2

|K (ω)|dω <∞

W k2 (Rd) =

u ∈ L2(Rd) :

|u(ω)|2(1 + ‖ω‖22)kdω <∞

Finding a specific embedding constant

For a particular kernel function K find E (k) such that

(1 + y)k ≤ E (k)2

K (y),

where y = ‖ω‖22.

E. Larsson, 2017-09-18 (43 : 77)

RBF–QROutline

Stable methods

Convergence theory

RBF-FD

Embedding constants for kernel spaces contd

For the Gaussian K (y) = (2ε2)−d2 e−

4ε2 and

E (k) = C kkk2 .

For the inverse multiquadric

K (y) = 21−β

Γ(β)

(√yε

)β(ε√y)−d/2Kd/2−β(

√yε ),

where K is a modified Bessel function of the third kind,leading to E (k) = C kkk .

Using the embedding constants

We finally assume that there is an extension operator Esuch that ‖Eu‖N (Rd ) ≤ ‖u‖N (Ω). Then

‖u‖W k2 (Ω) ≤ ‖Eu‖W k

2 (Rd ) ≤ E (k)‖Eu‖N (Rd ) ≤ E (k)‖u‖N (Ω)

Wendland [Wen05, Theorem 10.46]

E. Larsson, 2017-09-18 (44 : 77)

RBF–QROutline

Stable methods

Convergence theory

RBF-FD

Implications for interpolation errors

The interpolant I (u) is zero at the node set X (discreteterm goes away). Together with the optimality property‖I (u)‖N (Ω) ≤ ‖u‖N (Ω), we get for the Gaussian

‖Dα(I (u)− u)‖Lq(Ω) ≤ eC log(h)√

h ‖u‖N (Ω),

and for the inverse multiquadric

‖Dα(I (u)− u)‖Lq(Ω) ≤ eC√h ‖u‖N (Ω).

These estimates can be improved, e.g., for a compact cube.

E. Larsson, 2017-09-18 (45 : 77)

RBF–QROutline

Stable methods

Convergence theory

RBF-FD

Cost of global method

Global RBF approximations of smooth functions are veryefficent.

A small number of node points per dimension are needed.

However N = 15 in 1-D becomes N = 50 625 in 4-D.

Up to three dimensions can be handled on a laptop, butnot more.

Furthermore, for less smooth functions, the number ofnodes per dimension grows quickly.

For a dense linear system: Direct solution O(N3), storageO(N2).

⇒ Move to localized methods.

E. Larsson, 2017-09-18 (46 : 77)

RBF–QROutline

Stable methods

Convergence theory

RBF-FD

Motivation for RBF-PUM

Global RBF approximation

+ Ease of implementation in any dimension.

+ Flexibility with respect to geometry.

+ Potentially spectral convergence rates.

− Computationally expensive for large problems.

RBF partition of unity methods

I Local RBF approximations on patches are blendedinto a global solution using a partition of unity.

I Provides spectral or high-order convergence.

I Solves the computational cost issues.

I Allows for local adaptivity.[Wen02, Fas07, HL12, Cav12, CDR14, CDR15, CDRP16, CRP16],

[SVHL15, HLRvS16, SL16, LSH17]

E. Larsson, 2017-09-18 (47 : 77)

RBF–QROutline

Stable methods

Convergence theory

RBF-FD

The RBF partition of unity method

Global approximation

u(x) =P∑j=1

wj(x)uj(x)

PU weight functions

Generate weight functions fromcompactly supported C 2 Wendland functions

ψ(ρ) = (4ρ+ 1)(1− ρ)4+

using Shepard’s method wi (x) = ψi (x)∑Mj=1 ψj (x)

CoverEach x ∈ Ω must be in the interior of at least one Ωj .Patches that do not contain unique points are pruned.

E. Larsson, 2017-09-18 (48 : 77)

RBF–QROutline

Stable methods

Convergence theory

RBF-FD

Differentiating RBF-PUM approximations

0 100 200 300

nz = 10801

Applying an operator globally

∆u =M∑i=1

∆wi ui + 2∇wi · ∇ui + wi∆ui

Local differentiation matricesLet ui be the vector of nodal values in patch Ωi , then

ui = Aλi , where Aij = φj(x i ) ⇒

Lui = ALA−1ui , where ALij = Lφj(x i ).

The global differentiation matrix

Local contributions are addedinto the global matrix.

E. Larsson, 2017-09-18 (49 : 77)

RBF–QROutline

Stable methods

Convergence theory

RBF-FD

demo6.m(Solving a Poisson problem in 2-D with RBF–PUM)

E. Larsson, 2017-09-18 (50 : 77)

RBF–QROutline

Stable methods

Convergence theory

RBF-FD

An RBF-PUM collocation method

Choices & Implications

I Nodes and evaluation points coincide.Square matrix, iterative solver available (Heryudono, Larsson,

Ramage, von Sydow [HLRvS16]).

I Global node set.Solutions ui (xk) = uj(xk) for xk in overlap regions.

I Patches are cut by the domain boundary.Potentially strange shapes and lowered local order.

E. Larsson, 2017-09-18 (51 : 77)

RBF–QROutline

Stable methods

Convergence theory

RBF-FD

An RBF-PUM least squares method

Choices & Implications

I Each patch has an identical node layout.Computational cost for setup is drastically reduced.

I Evaluation nodes are uniform.Easy to generate both local and global high quality node sets.

I Patches have nodes outside the domain.Good for local order, but requires denser evaluation points.

E. Larsson, 2017-09-18 (52 : 77)

RBF–QROutline

Stable methods

Convergence theory

RBF-FD

The RBF-PUM interpolation error

Eα = Dα(I (u)− u) =M∑j=1

∑|β|≤|α|

)DβwjD

α−β(I (uj)− uj)

The weight functions

For C k weight functions and |α| ≤ k

‖Dαwj‖L∞(Ωj ) ≤Cα

H|α|j

, Hj = diam(Ωj).

The local RBF interpolants (Gaussians)

Define the local fill distance hj (Rieger, Zwicknagl [RZ10])

‖Dα(I (uj)− uj)‖L∞(Ωj )≤ cα,jh

mj− d2−|α|

j ‖uj‖N (Ωj ),

‖Dα(I (uj)− uj)‖L∞(Ωj )≤ eγα,j log(hj )/

√hj‖uj‖N (Ωj )

E. Larsson, 2017-09-18 (53 : 77)

RBF–QROutline

Stable methods

Convergence theory

RBF-FD

RBF-PUM interpolation error estimates

Algebraic estimate for Hj/hj = c

‖Eα‖L∞(Ω) ≤ K max1≤j≤M CjHmj− d

2−|α|

j ‖u‖N (Ωj )

K — Maximum # of Ωj overlapping at one pointmj — Related to the local # of pointsΩj — Ωj ∩ Ω

Spectral estimate for fixed partitions

‖Eα‖L∞(Ω) ≤ K max1≤j≤M Ceγj log(hj )/√

hj‖u‖N (Ωj )

Implications

I Bad patch reduces global order.

I Two refinement modes.

I Guidelines for adaptive refinement.

E. Larsson, 2017-09-18 (54 : 77)

RBF–QROutline

Stable methods

Convergence theory

RBF-FD

Error estimate for PDE approximation

Larsson, Shcherbakov, Heryudono [LSH17]

The PDE estimate‖u − u‖L∞(Ω) ≤ CPEL + CP‖L·,XL+

Y ,X‖∞ (CMδM + EL),

where CP is a well-posedness constant and CMδM is asmall multiple of the machine precision.

Implications

I Interpolation error EL provides convergence rate.

I Norm of inverse/pseudoinverse can be large.

I Matrix norm better with oversampling.

I Finite precision accuracy limit involves matrix norm.

Follows strategies from Schaback [Sch07, Sch16]

E. Larsson, 2017-09-18 (55 : 77)

RBF–QROutline

Stable methods

Convergence theory

RBF-FD

Does RBF-PUM require stable methods?

In order to achieve convergence we have two options

I Refine patches such that diameter H decreases.

I Increase node numbers such that Nj increases.

I In both cases, theory assumes ε fixed.

The effect of patch refinement

H = 1, ε = 4 H = 0.5, ε = 4 H = 0.25, ε = 4

The RBF–QR method: Stable as ε → 0 for N 1Effectively a change to a stable basis.Fornberg, Piret [FP07], Fornberg, Larsson, Flyer [FLF11], Larsson,

Lehto, Heryudono, Fornberg [LLHF13]

E. Larsson, 2017-09-18 (56 : 77)

RBF–QROutline

Stable methods

Convergence theory

RBF-FD

Effects on the local matrices

10−2

10−1

10−10

10−5

N=10 N=20 N=40

Relative error in A∆j A−1j

without RBF-QR

Local contribution to a global Laplacian

Lj = (W∆j Aj + 2W∇

j A∇j + WjA∆j )A−1

Typically: Aj ill-conditioned, Lj better conditioned.

RBF-QR for accuracy

I Stable for small RBFshape parameters ε

I Change of basisA = AQR−T1 D−T1

I Same result in theoryALA−1 = ALA−1

I More accurate in practice

E. Larsson, 2017-09-18 (57 : 77)

RBF–QROutline

Stable methods

Convergence theory

RBF-FD

Poisson test problems in 2-D

Domain Ω = [−2, 2]2.

Uniform nodes in the collocation case.

uR(x , y) = 125x2+25y2+1

uT (x , y) = sin(2(x−0.1)2) cos((x−0.3)2)+sin2((y−0.5)2)

E. Larsson, 2017-09-18 (58 : 77)

RBF–QROutline

Stable methods

Convergence theory

RBF-FD

Error results with and without RBF–QR

I Least squares RBF-PUM

I Fixed shape ε = 0.5 or scaled such that εh = c

I Left: 5× 5 patches Right: 55 points per patch

Spectral mode

20 40 60 80 100 120

10−6

10−4

10−2

Points per dimension

RBF−QR

Direct

Scaled

Algebraic mode

4 8 16 32

10−8

10−6

10−4

10−2

Patches per dimensionE

p=−7

RBF−QR

Direct

Scaled

I With RBF–QR better results for H/h large.

I Scaled approach good until saturation.E. Larsson, 2017-09-18 (59 : 77)

RBF–QROutline

Stable methods

Convergence theory

RBF-FD

Convergence as a function of patch size

Runge Trig

0.2 0.3 0.4 0.510

10−4

10−2

0.2 0.3 0.4 0.5

10−8

10−6

10−4

10−2

Collocation (dashed lines) and Least Squares (solid lines).

I Points per patch n = 28, 55, 91.

I Theoretical rates p = 4, 7, 10.

I Numerical rates p ≈ 3.9, 6.9, 9.8.E. Larsson, 2017-09-18 (60 : 77)

RBF–QROutline

Stable methods

Convergence theory

RBF-FD

Spectral convergence for fixed patches

Runge Trig

−50 −40 −30 −2010

10−6

10−4

10−2

−1/h

−40 −30 −2010

10−8

10−6

10−4

10−2

−1/h

Collocation (dashed lines) and Least Squares (solid lines).

LS-RBF-PU is significantly more accurate due to theconstant number of nodes per patch.

E. Larsson, 2017-09-18 (61 : 77)

RBF–QROutline

Stable methods

Convergence theory

RBF-FD

Robustness and large scale problems

The global error estimate

‖u − u‖L∞(Ω) ≤ CPEL + CP‖L·,XL+Y ,X‖∞ (CMδM + EL)

The dark horse is the ’stability matrix norm’

I The stability norm is related to conditioning.

I In the collocation case, ‖L−1X ,X‖ grows with N.

I How does it behave with least squares?

E. Larsson, 2017-09-18 (62 : 77)

RBF–QROutline

Stable methods

Convergence theory

RBF-FD

Stability norm: Patch size

I Fixed number of points per patch n = 28, 55, 91

I Results as a function of patch diameter H

Stability(H) Error(H)

0.2 0.3 0.4 0.6

0.2 0.3 0.4 0.610

10−8

10−6

10−4

10−2

Collocation (dashed) and LS (solid)

I The norm does not grow for LS-RBF-PUM (!)E. Larsson, 2017-09-18 (63 : 77)

RBF–QROutline

Stable methods

Convergence theory

RBF-FD

RBF-generated finite differences RBF–FD

Flyer et al. [FW09, FLB+12]

I Approximate Lu(xc)using the n nearest nodes by

Lu(xc) ≈n∑

wku(xk)

I Find weights wk by askingexactness for RBF-interpolants.

φ1(x1) φ1(x2) · · · φ1(xn)φ2(x1) φ2(x2) · · · φ2(xn)

......

. . ....

φn(x1) φn(x2) · · · φn(xn)

w2...wn

Lφ1(xc)Lφ2(xc)

...Lφn(xc)

E. Larsson, 2017-09-18 (64 : 77)

RBF–QROutline

Stable methods

Convergence theory

RBF-FD

Is RBF-QR needed with RBF–FD?Approximation of ∆u with n = 56. Magenta lines arewith added polynomial terms p = 0, . . . , 3.

10−2

10−1

10−8

10−6

10−4

10−2

ℓ ∞error

ε = 1.5, direct

ε = 1.5, RBF-QR

ε = 0 (polynomial)

εh= 0.3, direct

I Scaled ε: No ill-conditioning, butsaturation/stagnation. [LLHF13]

I Fixed ε: RBF-QR is needed.

I Added terms: Compromise with partial recovery.E. Larsson, 2017-09-18 (65 : 77)

RBF–QROutline

Stable methods

Convergence theory

RBF-FD

RBF-FD with PHS and polynomials

Combine polyharmonic splines, e.g, φ(r) = |r |7 withpolynomial terms 1, x , y , . . . , x2, . . . such that the numberof polynomial terms ≈ the number of nodes.

I Contains both smooth and piecewise smoothcomponents, that have different roles in theapproximation.

I No shape parameter to tune.

I Heuristically, skewed stencils seem to behave wellnear boundaries.

Bayona, Flyer, Fornberg, Barnett [FFBB16, BFFB17]

E. Larsson, 2017-09-18 (66 : 77)

RBF–QROutline

Stable methods

Convergence theory

RBF-FD

References I

Victor Bayona, Natasha Flyer, Bengt Fornberg, andGregory A. Barnett, On the role of polynomials inRBF-FD approximations: II. Numerical solution ofelliptic PDEs, J. Comput. Phys. 332 (2017),257–273.

Brad J. C. Baxter and Simon Hubbert, Radial basisfunctions for the sphere, Recent progress inmultivariate approximation (Witten-Bommerholz,2000), Internat. Ser. Numer. Math., vol. 137,Birkhauser, Basel, 2001, pp. 33–47. MR 1877496

Roberto Cavoretto, Partition of unity algorithm fortwo-dimensional interpolation using compactlysupported radial basis functions, Commun. Appl. Ind.Math. 3 (2012), no. 2, e–431, 13. MR 3063253

E. Larsson, 2017-09-18 (67 : 77)

RBF–QROutline

Stable methods

Convergence theory

RBF-FD

References II

Roberto Cavoretto and Alessandra De Rossi, Ameshless interpolation algorithm using a cell-basedsearching procedure, Comput. Math. Appl. 67(2014), no. 5, 1024–1038. MR 3166529

, A trivariate interpolation algorithm using acube-partition searching procedure, SIAM J. Sci.Comput. 37 (2015), no. 4, A1891–A1908. MR3376134

R. Cavoretto, A. De Rossi, and E. Perracchione,Efficient computation of partition of unityinterpolants through a block-based searchingtechnique, Comput. Math. Appl. 71 (2016), no. 12,2568–2584. MR 3504167

E. Larsson, 2017-09-18 (68 : 77)

RBF–QROutline

Stable methods

Convergence theory

RBF-FD

References III

Roberto Cavoretto, Alessandra De Rossi, and EmmaPerracchione, RBF-PU interpolation with variablesubdomain sizes and shape parameters, AIPConference Proceedings 1776 (2016), no. 1, 070003.

T. A. Driscoll and B. Fornberg, Interpolation in thelimit of increasingly flat radial basis functions,Comput. Math. Appl. 43 (2002), no. 3-5, 413–422,Radial basis functions and partial differentialequations. MR 1883576

Stefano De Marchi and Gabriele Santin, A new stablebasis for radial basis function interpolation, J.Comput. Appl. Math. 253 (2013), 1–13. MR3056591

E. Larsson, 2017-09-18 (69 : 77)

RBF–QROutline

Stable methods

Convergence theory

RBF-FD

References IV

, Fast computation of orthonormal basis forRBF spaces through Krylov space methods, BIT 55(2015), no. 4, 949–966. MR 3434025

Gregory E. Fasshauer, Meshfree approximationmethods with MATLAB, InterdisciplinaryMathematical Sciences, vol. 6, World ScientificPublishing Co. Pte. Ltd., Hackensack, NJ, 2007. MR2357267

Natasha Flyer, Bengt Fornberg, Victor Bayona, andGregory A. Barnett, On the role of polynomials inRBF-FD approximations: I. Interpolation andaccuracy, J. Comput. Phys. 321 (2016), 21–38. MR3527556

E. Larsson, 2017-09-18 (70 : 77)

RBF–QROutline

Stable methods

Convergence theory

RBF-FD

References V

Natasha Flyer, Erik Lehto, Sebastien Blaise, Grady B.Wright, and Amik St-Cyr, A guide to RBF-generatedfinite differences for nonlinear transport: Shallowwater simulations on a sphere, J. Comput. Phys. 231(2012), no. 11, 4078–4095. MR 2911786

Bengt Fornberg, Elisabeth Larsson, and NatashaFlyer, Stable computations with Gaussian radial basisfunctions, SIAM J. Sci. Comput. 33 (2011), no. 2,869–892. MR 2801193

Gregory E. Fasshauer and Michael J. McCourt,Stable evaluation of Gaussian radial basis functioninterpolants, SIAM J. Sci. Comput. 34 (2012), no. 2,A737–A762. MR 2914302

E. Larsson, 2017-09-18 (71 : 77)

RBF–QROutline

Stable methods

Convergence theory

RBF-FD

References VI

Bengt Fornberg and Cecile Piret, A stable algorithmfor flat radial basis functions on a sphere, SIAM J.Sci. Comput. 30 (2007), no. 1, 60–80. MR 2377431

B. Fornberg and G. Wright, Stable computation ofmultiquadric interpolants for all values of the shapeparameter, Comput. Math. Appl. 48 (2004), no. 5-6,853–867. MR 2105258

Natasha Flyer and Grady B. Wright, A radial basisfunction method for the shallow water equations on asphere, Proc. R. Soc. Lond. Ser. A Math. Phys. Eng.Sci. 465 (2009), no. 2106, 1949–1976. MR 2500804

Alfa Heryudono and Elisabeth Larsson, FEM-RBF: Ageometrically flexible, efficient numerical solutiontechnique for partial differential equations with mixedregularity, Tech. report, Marie Curie FP7, 2012.

E. Larsson, 2017-09-18 (72 : 77)

RBF–QROutline

Stable methods

Convergence theory

RBF-FD

References VII

Alfa Heryudono, Elisabeth Larsson, Alison Ramage,and Lina von Sydow, Preconditioning for radial basisfunction partition of unity methods, J. Sci. Comput.67 (2016), no. 3, 1089–1109. MR 3493496

E. Larsson and B. Fornberg, Theoretical andcomputational aspects of multivariate interpolationwith increasingly flat radial basis functions, Comput.Math. Appl. 49 (2005), no. 1, 103–130. MR 2123189

Elisabeth Larsson, Erik Lehto, Alfa Heryudono, andBengt Fornberg, Stable computation ofdifferentiation matrices and scattered node stencilsbased on Gaussian radial basis functions, SIAM J.Sci. Comput. 35 (2013), no. 4, A2096–A2119. MR3092752

E. Larsson, 2017-09-18 (73 : 77)

RBF–QROutline

Stable methods

Convergence theory

RBF-FD

References VIII

Elisabeth Larsson, Victor Shcherbakov, and AlfaHeryudono, A least squares radial basis functionpartition of unity method for solving pdes, SIAM J.Sci. Comp. (2017).

Charles A. Micchelli, Interpolation of scattered data:distance matrices and conditionally positive definitefunctions, Constr. Approx. 2 (1986), no. 1, 11–22.

Francis J. Narcowich, Joseph D. Ward, and HolgerWendland, Sobolev bounds on functions withscattered zeros, with applications to radial basisfunction surface fitting, Math. Comp. 74 (2005),no. 250, 743–763. MR 2114646

E. Larsson, 2017-09-18 (74 : 77)

RBF–QROutline

Stable methods

Convergence theory

RBF-FD

References IX

Rodrigo B. Platte, Lloyd N. Trefethen, and Arno B. J.Kuijlaars, Impossibility of fast stable approximationof analytic functions from equispaced samples, SIAMRev. 53 (2011), no. 2, 308–318. MR 2806639

Christian Rieger and Barbara Zwicknagl, Samplinginequalities for infinitely smooth functions, withapplications to interpolation and machine learning,Adv. Comput. Math. 32 (2010), no. 1, 103–129. MR2574569

Robert Schaback, Multivariate interpolation bypolynomials and radial basis functions, Constr.Approx. 21 (2005), no. 3, 293–317. MR 2122312

, Convergence of unsymmetric kernel-basedmeshless collocation methods, SIAM J. Numer. Anal.45 (2007), no. 1, 333–351 (electronic). MR 2285858

E. Larsson, 2017-09-18 (75 : 77)

RBF–QROutline

Stable methods

Convergence theory

RBF-FD

References X

, All well-posed problems have uniformlystable and convergent discretizations, Numer. Math.132 (2016), no. 3, 597–630. MR 3457442

Victor Shcherbakov and Elisabeth Larsson, Radialbasis function partition of unity methods for pricingvanilla basket options, Comput. Math. Appl. 71(2016), no. 1, 185–200. MR 3441188

Ali Safdari-Vaighani, Alfa Heryudono, and ElisabethLarsson, A radial basis function partition of unitycollocation method for convection-diffusion equationsarising in financial applications, J. Sci. Comput. 64(2015), no. 2, 341–367. MR 3366079

E. Larsson, 2017-09-18 (76 : 77)

RBF–QROutline

Stable methods

Convergence theory

RBF-FD

References XI

Holger Wendland, Fast evaluation of radial basisfunctions: methods based on partition of unity,Approximation theory, X (St. Louis, MO, 2001),Innov. Appl. Math., Vanderbilt Univ. Press, Nashville,TN, 2002, pp. 473–483. MR 1924902

, Scattered data approximation, CambridgeMonographs on Applied and ComputationalMathematics, vol. 17, Cambridge University Press,Cambridge, 2005. MR 2131724

Grady B. Wright and Bengt Fornberg, Stablecomputations with flat radial basis functions usingvector-valued rational approximations, J. Comput.Phys. 331 (2017), no. C, 137–156.

E. Larsson, 2017-09-18 (77 : 77)

RBF{QR Radial basis function approximationRadial basis function approximation PhD student course in...

Documents