1
TRICAP 2009, Nuria, June 15, 2009
Everything should be made as simple
as possible, but not simpler.
A. Einstein (1879-1955)
Tensor-Structured Numerical Methods
for Solving Multidimensional Equations
Boris N. Khoromskijhttp://personal-homepages.mis.mpg.de/bokh
Max-Planck-Institute for
Mathematics in the Sciences
Leipzig
Challenging Computational Problems in High Dimensions B. Khoromskij, TRICAP 2009, 15.06.09 2
1. Motivating applications:
Modelling of large molecular systems such as proteins,
bio-molecules and nanostructures. Molecular dynamics.
Computational problems in financial math., machine learning,
stochastic PDEs, multiparametric optimization, etc.
2. Main computational difficulties:
Exponential scaling in dimension for traditional
O(nd)-methods of linear complexity, ”curse of dimensionality”.
3. Novel Approach:
Tensor-structured numerical methods for representation of
d-variate functions, operators and for solving physical
equations in Rd, all with linear O(dn)-scaling in dimension d,
Rnd
Rn ⊗ ... ⊗ R
n, Rmd×nd
Rmn ⊗ ... ⊗ R
mn.
Target mathematical equations B. Khoromskij, TRICAP 2009, 15.06.09 3
Linear/nonlinear spectral/b.v. problems in Rd:
Compute a pair (λ, u) ∈ R × H10 (Ω), s.t., 〈u, u〉 = 1,
Λu := − div (A grad u) + V u = λu in Ω,
u = 0 on ∂Ω.
BVPs: Find u ∈ H10 (Ω), s.t.,
Λu = F in Ω.
Specific features:
⊲ High spacial dimension: Ω = (−b, b)d ∈ Rd (d = 2, 3, ..., 100, ...).
⊲ Multiparametric equations: A(x, a), V (x, a), a ∈ RM
(M = 1, 2, ..., 100).
⊲ Nonlocal (integral) operator V , singular potentials.
⊲ Nonlinear operator V .
Toward multi-dimensional numerical modeling B. Khoromskij, TRICAP 2009, 15.06.09 4
1929, Dirac:
The fundamental laws necessary for the mathematical treatment of large
part of physics and the whole of chemistry are thus completely known,
and the difficulty lies only in the fact that application of these laws leads
to equations that are too complex to be solved.
1998, W. Kohn, A. Pople:
Nobel Prize in Chemistry for development of DFT, based on
computations via (separable) GTO basis sets.
Nowadays: Development of numerical tensor methods in
multi-dimensional modeling that includes:
– Effective nonlinear approximation of operators and functions in Rd,
– tensor-structured iterative methods,
– optimistic numerics in computational chemistry, sPDEs, NMLA,
– attempts to understanding the deep math. behind.
Ab initio models B. Khoromskij, TRICAP 2009, 15.06.09 5
The Hartree-Fock equation,
[−
1
2∆ − V (x) +
∫
R3
ρ(y)
‖x − y‖dy
]φi(x)−
1
2
∫
R3
τ(x, y)
‖x − y‖φi(y)dy = λφi(x)
with φi ∈ H1(R3),∫R3 φiφj = δij , 1 ≤ i, j ≤ Ne, where
τ(x, y) =Ne∑i=1
φi(x)φi(y) - electron density matrix,
ρ(x) = τ(x, x) - electron density,1
‖x‖ - Newton potential,
V (x) - Coulomb potential with singularities at the nuclei.
Canonical tensor format B. Khoromskij, TRICAP 2009, 15.06.09 6
Def. 1. (Canonical format) Class of tensors in
Vn := H1 ⊗ ... ⊗ Hd, Hℓ = Rn, that allow R-term representation
CR =
w ∈ H : w =
R∑
k=1
w(1)k ⊗ w
(2)k ⊗ . . . ⊗ w
(d)k , w
(ℓ)k ∈ Hℓ
.
w ∈ CR with w /∈ CR−1, has the tensor rank R.
Advantage:
Tremendous reduction of the computational cost,
nd → d R n (linear in d).
Limitations:
Nonstable computation of the nearly optimal rank-R
approximations, no directional adaptivity, lack of theory.
Orthogonal Tucker representation B. Khoromskij, TRICAP 2009, 15.06.09 7
Choose Vℓ ⊂ Hℓ, with an orthonormal basis
φ(ℓ)k : 1 ≤ k ≤ rℓ
,
rℓ := dimVℓ ≪ n (1 ≤ ℓ ≤ d), i.e., T (ℓ) = [φ(ℓ)1 ...φ
(ℓ)rℓ ] ∈ Srℓ
– the Stiefel manifold of the orthogonal n × rℓ matrices.
V(r) = V1 ⊗ V2 ⊗ . . . ⊗ Vd ⊂ H ≡ Vn.
v =r∑
k=1
bkφ(1)k1
⊗ φ(2)k2
⊗ . . . ⊗ φ(d)kd
∈ V(r), r = (r1, . . . , rd) ,
k = (k1, . . . , kd), 1 ≤ kℓ ≤ rℓ, 1 ≤ ℓ ≤ d, core tensor β = [bk].
Def 2. (Tucker format) Given r, define Tr,n ⊂⋃
Srℓ
V(r),
Tr,n =v ∈ Vn : v = β ×1 T (1) ×2 T (2)... ×d T (d), bk ∈ R
.
Storage: rd + rdn ≪ nd, r = maxℓ rℓ ≪ n, (r = O(log n)).
Two-level tensor format B. Khoromskij, TRICAP 2009, 15.06.09 8
Def. 3. (Two-level Tucker-canonical model), T CR,r.
Subclass T CR,r⊂ T r,n with β ∈ CR,r ⊂ Vr,
v =
(∑R
ν=1βνu(1)
ν ⊗ . . . ⊗ u(d)ν
)×1 T (1) ×2 T (2)... ×d T (d).
Storage: dRr + R + drn (linear scaling in d, n, R, r).
[BNK ’06]
AB
V
V
r3
I3
I2
r2
(3)
(2)
I1
I
I
I
2
3
1r2
r3
r1
r1
V (1)
I
I
I
2
3
1
b1
+ . . . + bR
U(3)1
U(2)2
U(1)1
U(3)R
U(2)R
U(1)R
B
.
.
Level I: Tucker decomposition (left). Level II: canonical decomposition of β (right).
Nonlinear approximation in tensor format B. Khoromskij, TRICAP 2009, 15.06.09 9
Problem 1. Best rank-R approximation of f ∈ Vn in CR.
Problem 2. Best rank-r orthogonal approx. of f ∈ Vn in Tr.
Problem 3. Best rank-(R, r) two-level orthogonal
approximation of a high-order tensor f ∈ Vn in the set T CR,r.
Tr and CR are not linear spaces ⇒ a nontrivial
nonlinear approximation problem.
Def. 4. (Tensor truncation TS : S1 → S).
Given A ∈ S1 ⊂ Vn, S1 ∈ Vn, CR0, find
TS(A) := argminT∈S
‖T − A‖, where S ∈ Tr, CR, T CR,r. (1)
Rem. d = 2: Celebrated theorem by E. Schmidt, 1907 on
best bilinear approximation of f(x, y) (mimics truncated SVD).
Evolution of Tensor Methods in Sci. Computing B. Khoromskij, TRICAP 2009, 15.06.09 10
⊲ HOSVD, ALS for best orthogonal Tucker approximation.
[De Lathauwer, De Moor, J. Vandewalle 2000], (Leuven).
Enhanced ALS for canonical approximation - Many contributions.
⊲ General concept of tensor approximation in mathematical physics.
[Beylkin, Mohlenkamp ’02-’05], (Colorado, Ohio).
[Hackbusch, BNK, Tyrtyshnikov ’03-’07], (Leipzig, Moscow).
⊲ Analytic methods of tensor approximation in higher dimensions.
[Braess, Gavrilyuk, Hackbusch, BNK ’05-’08], (Bohum, Eisenach, Leipzig).
⊲ Multigrid accelerated mixed canonical-to-Tucker-to-canonical tensor
approximation. ALS via reduced HOSVD. [BNK, Khoromskaia ’08], (Leipzig).
⊲ Cross approximation in R3. [Oseledets, Savostianov, Tyrtyshnikov ’08], (Moscow).
⊲ Tensor representation of functions/operators in the Hartree-Fock eq.
and its iterative solution in tensor format.
[BNK, Khoromskaia, Flad ’09], (Leipzig, Berlin).
⊲ Toward tensor methods for SPDEs [BNK, Schwab ’09] (Leipzig, Zurich).
Issues addressed in this talk B. Khoromskij, TRICAP 2009, 15.06.09 11
Recent theory and numerics.
1. Tensor representation of d-variate functions/operators:
⊲ Multi-dimensional integral operators in Rd,
⊲ Convolution, FFTd and Laplace transforms,
⊲ Elliptic Green’s functions, (∆ ± µ)−1.
2. Two-level Tucker-canonical multigrid accelerated tensor
approximation: theory and algorithms.
3. Tensor truncated solvers for spectral/b.v. probl. in Rd.
4. Electronic structure calculations: first results on solution
of the 3D nonlinear Hartree-Fock eq. in tensor format.
5. Solving stochactic PDEs in tensor format
(high-dimensional “stochastic” variable).
6. Preliminary numerical tests.
Preliminary numerics. Protein molecule. B. Khoromskij, TRICAP 2009, 15.06.09 12
The Poisson-Boltzmann equation for electrostatic potential of protein
∇ · [ε(x)∇ · φ(x)] − ε(x)h(x)2sinh[φ(x)] + 4πρ(x)/kT = 0, x ∈ R3.
If ε(x) = ε0, h(x) = h, ρ(x) = δ(x), then φ(x) = e−h‖x‖
‖x‖ .
Sh. Hayryan, Ch.-K. Hu, E.A. Hayryan, I. Pokorny, Parallel Solution of the Poisson-Boltzmann Equation
for Proteins, Lecture Notes in Computer Science, Vol. 2657 (2003), p. 54-62.
2040
6080
100120
2040
6080
1000
500
1000
2040
6080
100120
2040
6080
1000
500
1000
VILLIN (left); central slice and its rank-30 decomposition (middle-right).
Rank-30 Tucker approximation of FD solution on n× n× n grid (n ≈ 150),
provides the relative accuracy 10−1 ÷ 10−2 in the Euclidean norm
[E.A. Hayryan, V. Khoromskaia, 2009], in progress.
Aluminium cluster (14/172 atoms): 3D FEM∗ → Tucker (r = 20). B. Khoromskij, TRICAP 2009, 15.06.09
∗[V. Gavini, J. Knap, K. Bhattacharya, and M. Ortiz, Non-periodic finite-element formulation of
orbital-free density-functional theory, J. Mech. Phys. Solids 55(4) (2007), 669-696.]
−10−5
05
10
−10
−5
0
5
100
0.02
0.04
0.06
cluster1/sdv4 , initial ρ for Al, z=0
−10−5
05
10
−10
0
100
1
2
3
4
x 10−4
cluster1/subdiv4, abs. diff. for ρ, rT=20
−8 −6 −4 −2 0 2 4 6 8−0.4
−0.3
−0.2
−0.1
0
0.1
0.2
0.3
0.4
u1(1)
u2(1)
u3(1)
u4(1)
u5(1)
u6(1)
−10
0
10
−15
−10
−5
0
5
10
15
0
0.05
Al, ρ , cluster1/subdiv4, z=0
V. Khoromskaia, T. Blesgen, V. Gavini. Tensor Product Approximation of Aluminium Cluster in
Orbital-free DFT. MPI MIS, Leipzig, 2009, in preparation.
Tensor-structured linear operations B. Khoromskij, TRICAP 2009, 15.06.09 14
Linear transforms are reduced to 1D-operations + rank truncation.
Tensors A1, A2 in the canonical format
A1 =
R1X
k=1
cku(1)k ⊗ . . .⊗ u
(d)k , A2 =
R2X
m=1
bmv(1)m ⊗ . . .⊗ v
(d)m .
1. Euclidean inner product (complexity O(dR1R2n) ≪ nd),
〈A1, A2〉 :=
R1X
k=1
R2X
m=1
ckbm
dY
ℓ=1
Du(ℓ)k , v
(ℓ)m
E.
2. Hadamard product of A1, A2
A1 ⊙A2 :=
R1X
k=1
R2X
m=1
ckbm“u(1)k ⊙ v
(1)m
”⊗ . . .⊗
“u(d)k ⊙ v
(d)m
”TS7−→.
3. Convolution of two 3rd order tensors A1, A2,
A1 ∗A2 =
R1X
k=1
R2X
m=1
ckbm(u(1)m ∗ v
(1)k ) ⊗ (u
(2)m ∗ v
(2)k ) ⊗ (u
(3)m ∗ v
(3)k )
TS7−→
with linear scaling in n, O(R1R2n logn) ≪ n3 logn (3D FFT).
Tensor-structured numerical methods B. Khoromskij, TRICAP 2009, 15.06.09 15
Truncated preconditioned iteration via tensor structured
vectors and matrices in S ∈ Rnd
Rn ⊗ ... ⊗ Rn:
Solve BVPs in S: AU = F ,
Um+1 = Um − B−1(AUm − F ), Um+1 := TS(Um+1) ∈ S.
Solve EVPs in S: AU = λU ,
Um+1 = Um − B−1(AUm − λmUm),
Um+1 := TS(Um+1) ∈ S,
Um+1 := Um+1/‖Um+1‖, λm+1 = 〈AUm+1, Um+1〉S .
TS - nonlinear projection onto tensor structured manifold S.
B−1 - tensor approximation of (−∆ + z)−1
.
Simple instructive examples B. Khoromskij, TRICAP 2009, 15.06.09 16
Example 1. Negative Laplacian in Ω = (0, π)d.
(λ, u) ∈ R × H10 (Ω) \ 0 : −∆u = λu.
λi =d∑
ℓ=1
i2ℓ , ui =d∏
ℓ=1
sin(iℓxℓ), i = (i1, ..., id) ∈ Nd.
The eigenvectors are exactly in the rank-1 tensor format.
Example 2. The Schrodinger equation for hydrogen atom,
(−1
2∆ −
1
‖x‖)u = λu, x ∈ Ω := R
3.
The eigenpair with minimal eigenvalue is
u1(x) = e−‖x‖, λ1 = −0.5,
and e−‖x‖ can be proven to have the low canonical rank.
Two-level BTA for canonical input B. Khoromskij, TRICAP 2009, 15.06.09 17
Algorithm BTA (Vn → T r,n): Cost O(nd+1).
Algorithm C BTA (CR,n→T CR,r) has polynomial cost in R, n,
For n ≥ R, d = 3 : O(R2n) + O(nr4).
Use the two-level Tucker-canonical format
CR,n→T CR,r → T CR′,r⊂ CR′,n, R′ < R.
Level-I: compute the best rank-r orthogonal Tucker
approximation with CR,n-type input, so that the resultant core
is represented in the CR,r format.
Level-II: the small-size Tucker core in CR,r is approximated by
an element in CR′,r with R′ < R.
Two-Level Canonical-to-Tucker approx. via RHOSVD B. Khoromskij, TRICAP 2009, 15.06.09 18
Thm. (Solving Problem 3.) [BNK, Khoromskaia ’08]
(a) For A =RP
ν=1ξνu
(1)ν ⊗ ...⊗ u
(d)ν ∈ CR,n, the minimisation problem
A ∈ CR,n ⊂ Vn : A(r) = argminT∈Tr,n‖A− T‖Vn
,
is equivalent to the dual maximisation problem
[Z(1), ..., Z(d)] = argmaxV (ℓ)∈Mℓ
‚‚‚‚‚
RX
ν=1
ξν“V (1)T
u(1)ν
”⊗ ...⊗
“V (d)T
u(d)ν
”‚‚‚‚‚
2
Vr
.
(b) (reduced HOSVD of A). The maximizer is computed by the ALS
iteration, with the initial guess via rank-rℓ truncated SVD of the ℓ-mode
side-matrix U (ℓ) = [u(ℓ)1 ...u
(ℓ)R ] ∈ Rn×R (ℓ = 1, ..., d).
(c) (Error of RHOSVD). Let σℓ,1 ≥ σℓ,2... ≥ σℓ,min(n,R) be the singular
values of U (ℓ) ∈ Rn×R (ℓ = 1, ..., d). Then the RHOSVD approx. of A, A0(r)
,
exhibits the error bound (cf. the complete O(nd+1)-HOSVD, [De Lathauwer et al. 2000]),
‖A−A0(r)‖ ≤ ‖ξ‖
dX
ℓ=1
(
min(n,R)X
k=rℓ+1
σ2ℓ,k)1/2, ‖ξ‖ =
vuutRX
ν=1
ξ2ν .
Toward linear complexity B. Khoromskij, TRICAP 2009, 15.06.09 19
Multigrid accelerated method, [BNK, Khoromskaia ’08].
d = 3: Cost O(Rrn).
Main idea:
– Solve the sequence of approximation problems for An = Anm
with n = nm := n02m, m = 0, 1, ..., M . Unigrid method via
RHOSVD approximation of An0 only on the coarse grid.
– Use the coarse approximation of dominating subspaces.
– ALS iteration on the reduced data set over “most important
fibers” (MIFs) of projected unfolding matrices, computed via
maximal energy principle (on a coarse grid only).
– Multigrid → the localization of dominating subspaces via
incomplete data.
Example I: Newton/Helmholtz kernels in RdB. Khoromskij, TRICAP 2009, 15.06.09 20
[BNK ’08], fκ(x) = exp(iκ‖x‖)‖x‖ , x ∈ Rd, κ ∈ R.
(a) Separable approximation of the “regularised” Helmholtz-type kernels
f1,κ(‖x‖) :=sin(κ‖x‖)
‖x‖,
f2,κ(‖x‖) :=1
‖x‖−cos(κ‖x‖)
‖x‖=
2sin2( κ2‖x‖)
‖x‖.
Complexity of the rank-O(κ) canonical approximation, d = 3,
O(| log ε| (| log ε| + κ)n) ≪ O(κ3 log κ+ n3 logn) [cf. Beylkin et al. ’08]
High frequency regime:
κ = Cn ⇒ O(n2 log n) ≪ O(n3 log n).
(b) Separable approximation of the fundamental solution for ∆ in Rd,
f(x) := 1/‖x‖d−2, x ∈ Rd.
Example I: Newton/Helmholtz kernels in RdB. Khoromskij, TRICAP 2009, 15.06.09 21
Theorem. (Rank estimate for oscillating kernels.) [BNK ’08]
For given tolerance ε > 0, the function f1,κ : [0, 2π√d]d → R allows the
Tucker/canonical approximations, s.t.
σ(f1,κ,S) ≤ Cε with S = T r,CR,
r ≤ R ≤ Cd(| log ε| + κ), r = (r, ..., r).
Let f1(t) =sin2(κ/2
√t)
t, then the coefficients tensor for f2,κ : [0, 2π√
d]d → R,
w.r.t. the tensor product basis of p.w.c. basis-functions φi(x), and given
by
G = [Gi]i∈I with Gi = ‖xi‖f1(‖xi‖)
Z
Rd
1
‖x‖φi(x)dx,
allows the Tucker/canonical approximations, providing the rank estimate
r ≤ R ≤ Cd2| log ε| (| log ε| + κ).
Example I: Newton kernel in RdB. Khoromskij, TRICAP 2009, 15.06.09 22
Canonical approximation to 1/‖x‖d−2 for d = 5. Approx. error ≈ 10−6.
1 3 5 7 9 11 13 15 17 19 21 23 2510
−7
10−6
10−5
10−4
10−3
10−2
10−1
Newton, dim=5, 1/|x|3, MaxCrank=25, grid=128
Canonical rank
2−no
rm
Example I: Canonical approx. to projected 1/‖x‖ in R3B. Khoromskij, TRICAP 2009, 15.06.09 23
10 20 30 4010
−6
10−5
10−4
10−3
10−2
10−1
rank
n=64n=128n=256
Canonical approximation of the Coulomb potential via sinc-quadratures
(solid lines).
Algebraically recompressed approximations (marked solid lines).
L2-projection on p.w.c. basis functions over n× n× n grid.
[Bertoglio, Hackbusch, BNK ’08]
Example I: Helmholtz kernel in R3B. Khoromskij, TRICAP 2009, 15.06.09 24
2 4 6 8 10 12 142
4
6
8
10
12
14
16
18
20
22
κ
Tuc
ker
rank
f2(|x|) on [0,π]3
ε =10−3
ε =10−4
Convergence history for the Tucker model applied to f2,κ, κ ∈ [1, 15], with
fixed accuracy ε > 0.
Basic example II: Electronic structure calculations B. Khoromskij, TRICAP 2009, 15.06.09 25
The Hartree-Fock equation
»−
1
2∆ − V (x) +
Z
R3
ρ(y)
‖x− y‖dy
–φ(x) −
1
2
Z
R3
τ(x, y)
‖x− y‖φ(y)dy = λφ(y),
τ(x, y) =Ne/2Pi=1
φi(x)φi(y) - the electron density matrix,
ρ(y) = τ(y, y) - the electron density,
1‖x‖ - the Newton potential,
V (x) =PνZν/‖x− xν‖ - external nuclei potential.
Challenging features:
⊲ Nonlinear equations
⊲ Nonlocal (integral) operators.
⊲ Large spatial grids in 3D.
⊲ High accuracy.
Example II: Hartree potential on large grids B. Khoromskij, TRICAP 2009, 15.06.09 26
VH(x) :=
∫
R3
ρ(y)
|x − y|dy =
(ρ ∗
1
‖ · ‖
)(x).
Represent orbitals in “approximating basis” gk, e.g., the
GTO basis,
ρ(x) =
N/2∑
i=1
(ϕi)2, ϕi =
R0∑
k=1
ci,kgk(x), R0 ≈ 100,
gk = (x − Ak)βke−λk(x−Ak)2 , x ∈ R3.
O(n log n)-computation of VH and its Galerkin matrix in tensor
format, on large n × n × n grids, error O(h3), h = 1/n.
Use Canonical-to-Tucker-to-canonical transform on a
sequence of grids to reduces the initial rank, Rρ ≈ R20/2.
[V. Khoromskaia, BNK ’08].
Compared with MOLPRO analytic program.
Example II: Hartree potential on large grids B. Khoromskij, TRICAP 2009, 15.06.09 27
Fast tensor convolution in R3 vs. FFT,
(Matlab, time/sec, linear scaling in Rρ and n), Rρ = 861,r = 15.
n3 1283 2563 5123 10243 20483 40963 81923 163843
FFT3 4.3 55.4 582.8 ∼ 6000 – – – ∼ 2 years
C ∗ C 1.0 3.1 5.3 21.9 43.7 127.1 368.6 700.2
Example II: Hartree potential on large grids B. Khoromskij, TRICAP 2009, 15.06.09 28
−6 −4 −2 0 2 4 6
10−4
10−3
hart
ree
Abs. approx. error, VH
for H2O
a) atomic units
n=4096n=8192Ri−4096−8192
2000 4000 6000 8000 10000 12000 14000 160000
1
2
3
4
5
6
grid size
min
utes
H2O , r
T=20
C−2−T time3D conv. time
a) Absolute error of the tensor-product computation for the Hartree
potential of the HO2 molecule in the interval Ω = [−6, 6] × 0 × 0;
b) CPU times corresponding to n× n× n-grid, up to n = 16000.
Ex. II: Error in the Coulomb matrix J = Jkm B. Khoromskij, TRICAP 2009, 15.06.09 29
Coulomb (Galerkin) matrix is computed by tensor inner products in gk,
Jkm :=
Z
R3gk(x)VH(x)gm(x)dx, k,m = 1, . . . R0, x ∈ R
3.
a) Electron density of H2O in Ω = [−4, 4] × [−4, 4] × 0.
c) Absolute approx. error for the Coulomb matrix Jkm (≈ 10−6).
Ex. II: The Exchange Galerkin Matrix K = Kkm B. Khoromskij, TRICAP 2009, 15.06.09 30
[V. Khoromskaia ’09], in preparation. Linear scaling in n, cubic in R0.
Kkm := −1
2
Z
R3
Z
R3gk(x)
τ(x, y)
|x− y|gm(y)dxdy, k,m = 1, . . . R0.
Absolute L∞-error in the matrix elements of K for the density of CH4
and pseudodensity of CH3OH.
Univariate grid size n = 1024, 4096. Approximation error O(h3), h = 1/n.
Ex. III: The tensor-truncated iter. for H-F eq. B. Khoromskij, TRICAP 2009, 15.06.09 31
gµ ∈ H1(R3) : ψi =
NbX
µ=1
Cµigµ, i = 1, ...,N.
For C = Cµi ∈ RNb×N , and F (C) = H + J(C) −K(C), with Galerkin
matrices in the approximating basis gµ,
I → S, H = −1
2∆ + Vc → H, VH → J(C), K → K(C),
solve eigenvalue problem
F (C)C = SCΛ, Λ = diag(λ1, ..., λN )
C∗SC = IN .
Multilevel “fixed-point” tensor-truncated iteration:
initial guess C0, for J = K = 0, grid size n = n0, 2n0, ..., 2pn0,
eFkCk+1 = SCk+1Λk+1, Λk+1 = diag(λk+11 , ..., λk+1
N ) → Λ
C∗k+1SCk+1 = IN ,
where eFk, k = 0, 1, ..., is specified by extrapolation over F (Ck), F (Ck−1), ...
and J(C), K(C) are computed fast in tensor format.
[H.-J. Flad, V. Khoromskaia, BNK ’09], in preparation
Ex. III: The tensor-truncated iter. for H-F eq. B. Khoromskij, TRICAP 2009, 15.06.09 32
⊲ Multigrid convergence in eigenvalues (left).
⊲ Convergence in effective iterations scaled to finest grid (right).
2 4 6 8 10 1210
−4
10−3
10−2
10−1
100
iterations
P. CH4 abs. error , | λ − λ
n,it |
n=64n=128
n=256
n=512
n=1024
0 1 2 3 410
−4
10−3
10−2
10−1
100
conv.in eff.iterations, CH4, pseudo, n=512
Ex. III: Optimal scaling of tensor-truncated iteration B. Khoromskij, TRICAP 2009, 15.06.09 33
Linear scaling of the CPU time (per iteration) in the univariate grid size n.
200 400 600 800 10000
5
10
15
20
25
30
univariate grid size
min
ute
s
time per SCF iteration
Example IV. Spectral problems in high dimensions B. Khoromskij, TRICAP 2009, 15.06.09 34
−∆u = λu in [0, π]d.
n + 1 Time/it. δλ δu it.
64 0.03 2.0 · 10−4 1.5 · 10−3 4
256 0.05 1.6 · 10−5 9.4 · 10−4 4
1024 0.12 7.8 · 10−7 1.2 · 10−4 5
4096 0.51 4.9 · 10−8 3.4 · 10−5 5
16384 2.2 3.1 · 10−9 9.3 · 10−6 5
65536 10.6 1.9 · 10−10 2.8 · 10−6 5
131072 22.3 4.8 · 10−11 1.6 · 10−6 5
Minimal λ for 3D Laplacian on n× n× n grids, n = 2p − 1, p = 6, 8, ..., 17.
[Hackbusch, BNK, Sauter, Tyrtyshnikov ’08]
d Time/it δλ δu
3 0.9 3.1 · 10−6 4.5 · 10−4
10 2.9 3.1 · 10−6 3.8 · 10−4
50 14.7 3.1 · 10−6 3.1 · 10−4
Minimal λ for the d-Laplacian (d = 3, 10, 50), n = 512. [BNK ’08-’09]
Example V: Stochastic PDEs B. Khoromskij, TRICAP 2009, 15.06.09 35
[BNK, Ch. Schwab ’09]
Given an elliptic operator
A := −div (aM (y, x) grad) and f ∈ L2 (D) , D ∈ Rd, d = 1, 2, 3,
aM (y, x) is a smooth function of x ∈ D, y = (y1, ..., yM ) ∈ Γ := [−1, 1]M ,
Find uM ∈ L2(Γ) ×H10 (D), s.t.
AuM (y, x) = f(x) in D, ∀y ∈ Γ,
uM (y, x) = 0 on ∂D, ∀y ∈ Γ.
For random field that is linear in the stochastic variable:
aM (y, x) := a0(x) +MX
m=1
am(x)ym,
where am ∈ L∞(D), m = 0, ...,M , are defined by the truncated
Karhunen-Loeve expansion.
Example V: Stochastic PDEs in high dimensions B. Khoromskij, TRICAP 2009, 15.06.09 36
The tensor-truncated iterative method scales linearly in M ≤ 50.
Variable coefficients with exponential decay (α = 1, n = 63, R ≤ 5),
am(x) = 0.5 e−αmsin(mx), m = 1, 2, ....,M, x ∈ (0, π).
1 2 3 4 510
−4
10−3
10−2
10−1
Dim=10, alpha=1, rank=5, grid=63
rank
2−no
rm
1 2 3 4 510
−4
10−3
10−2
10−1
Dim=20, alpha=1, rank=5, grid=63
rank
2−no
rm
Rank approximation in tensor-truncated preconditioned iteration for
solving sPDE with d = 1, M = 10, 20.
Toward efficient and fast tensor methods in RdB. Khoromskij, TRICAP 2009, 15.06.09 37
Fundamental questions (no ultimate answers):
Is there a curse of dimensionality (amount of information)?
NO, for physically relevant data, cf. Kolmogorow’s paradigm.
Can we represent the basic operators in tensor format?
YES, for Green’s kernels, elliptic resolvent, convolution, f(A).
Can we solve the minimization problem (1) efficiently?
YES, the multigrid accelerated two-level approx., cross approx.
Can we expect the fast (exponential) convergence in the rank
parameters R, r = max rℓ?
YES, for physically relevant data (solutions of basic equations).
Sinc quadratures + algebraic “recompression“.
Can we solve the basic equations in Rd on nonlinear manifold S?
YES, by tensor-structured methods.
⊲ SCF tensor-truncated iteration for the Hartree-Fock equation.
⊲ Linear EVPs in Rd.
⊲ Stochastic PDEs.
