The Poisson matrix and KroneckerProducts
Tom Lyche
University of Oslo
Norway
The Poisson matrix and Kronecker Products – p. 1/27
Plan for the dayNumerical solution of the Poisson problem
The finite difference scheme
Formulation as a matrix equation
Formulation in standard form Ax = b.
The Poisson matrix A is block tridiagonal
LU factorization of a block tridiagonal matrix
Solving the Poisson problem, complexity
Kronecker product
Relation to Poisson problem
Basic properties
Eigenvalues and eigenvectors
Eigenpairs for the Poisson matrix
The Poisson matrix and Kronecker Products – p. 2/27
Numerical solution of the Poisson Problem
−(u +u ) =f u=0
u=0
u=0
u=0 yyxx
m ∈ N, h = 1/(m + 1), vj,k ≈ u(jh, kh)
jh
(jh,kh)kh j+1,k
j,k+1
j-1,k
j,k-1
0 (m+1)h
0
(m+1)h
j,k
The Poisson matrix and Kronecker Products – p. 3/27
Discrete Equation
g′′(t) ≈ g(t − h) − 2g(t) + g(t + h)
h2
j,k j+1,k
j,k+1
j-1,k
j,k-1v
v v v
v
From differential equation for j, k = 1, 2, . . . , m
(−vj−1,k + 2vj,k − vj+1,k) + (−vj,k−1 + 2vj,k − vj,k+1) = h2fj,k (1)
From boundary conditions
vj,k = 0 if j = 0, m + 1 or k = 0, m + 1 (2)
The Poisson matrix and Kronecker Products – p. 4/27
Matrix Equation TV + V T = h2F
2 −1
−1 2
v11 v12
v21 v22
+
v11 v12
v21 v22
2 −1
−1 2
=1
9
f11 f12
f21 f22
T := T m = tridiagm(−1, 2,−1) ∈ Rm,m
V :=
v11 · · · v1m
......
vm1 · · · vmm
∈ Rm,m F :=
f11 · · · f1m
......
fm1 · · · fmm
∈ Rm,m
(1), (2) ⇔ TV + V T = h2F
(
TV + V T)
jk=
m∑
i=1
T jivik +m
∑
i=1
vjiT ik = l.h.s in (1) = h2(F )jk
The Poisson matrix and Kronecker Products – p. 5/27
Convert TV + V T = h2F to Ax = b
>
v in gridj,k
1,1
1,2
1,3
2,1
2,2
2,3
3,1
3,2
3,3
1
4
7
2
5
8
3
6
9
x in gridi
j,k j+1,k
j,k+1
j-1,k
j,k-1
i i+1
i+m
i-1
i-m
4x-x
-x
-x
-x-v
v v -v
-v
- 4
-->
4xi − xi−1 − xi+1 − xi−m − xi+m = bi
The Poisson matrix and Kronecker Products – p. 6/27
Linear system Ax = b
For m = 3 or n = 9 we have the following linear system:
4x1 −x2 −x4 = h2f11
−x1 4x2 −x3 −x5 = h2f21
−x2 4x3 −x6 = h2f31
−x1 4x4 −x5 −x7 = h2f12
−x2 −x4 4x5 −x6 −x8 = h2f22
−x3 −x5 4x6 −x9 = h2f32
−x4 4x7 −x8 = h2f13
−x5 −x7 4x8 −x9 = h2f23
−x6 −x8 4x9 = h2f33
The Poisson matrix and Kronecker Products – p. 7/27
Blockstructure of A
A =
4 −1 0 −1 0 0 0 0 0
−1 4 −1 0 −1 0 0 0 0
0 −1 4 0 0 −1 0 0 0
−1 0 0 4 −1 0 −1 0 0
0 −1 0 −1 4 −1 0 −1 0
0 0 −1 0 −1 4 0 0 −1
0 0 0 −1 0 0 4 −1 0
0 0 0 0 −1 0 −1 4 −1
0 0 0 0 0 −1 0 −1 4
A =
T + 2I −I 0
−I T + 2I −I
0 −I T + 2I
Block tridiagonalThe Poisson matrix and Kronecker Products – p. 8/27
LU -factorization of a tridiagonal matrix
d1 c1
a2 d2 c2
. . .. . .
. . .
an−1 dn−1 cn−1
an dn
=
1
l2 1
. . .. . .
ln 1
u1 c1
. . .. . .
un−1 cn−1
un
.
(3)
Note that L has ones on the diagonal, and that A and U have the samesuper-diagonal.By equating entries in (3) we find
d1 = u1, ak = lkuk−1, dk = lkck−1 + uk, k = 2, 3, . . . , n.
The Poisson matrix and Kronecker Products – p. 9/27
The AlgorithmA = LU (LU -factorization)
Ly = b (forward substitution)
Ux = y (backward substituion)
u1 = d1, lk = ak
uk−1
, uk = dk − lkck−1, k = 2, 3, . . . , n.
y1 = b1, yk = bk − lkyk−1, k = 2, 3, . . . , n,
xn = yn/dn, xk = (yk − ckxk+1)/uk, k = n − 1, . . . , 2, 1.
This process is well defined if A is strictly diagonallydominant.
The number of arithmetic operations (flops) is O(n).
The Poisson matrix and Kronecker Products – p. 10/27
LU of Block Tridiagonal Matrix
D1 C1
A2 D2 C2
. . .. . .
. . .Am−1 Dm−1 Cm−1
Am Dm
=
1L2 1
. . .. . .Lm 1
U1 C1
. . .. . .
Um−1 Cm−1
Um
.
U1 = D1, Lk = AkU−1
k−1, Uk = Dk − LkCk−1, k = 2, 3, . . . , m. (4)
To solve the system Ax = b we partition b conformally with A in the formbT = [bT
1 , . . . , bTm]. The formulas for solving Ly = b and Ux = y are as
follows:
y1 = b1, yk = bk − Lkyk−1, k = 2, 3, . . . , m,
xm = U−1m ym, xk = U−1
k (yk − Ckxk+1), k = m − 1, . . . , 2, 1.(5)
The solution is then xT = [xT1 , . . . , xT
m].
The Poisson matrix and Kronecker Products – p. 11/27
Poisson ProblemThe discrete Poisson problem can be solved writing it as a blocktridiagonal matrix.
We can also use Cholesky factorization of a band matrix.
The matrix has bandwidth d =√
n.
Cholesky requires O(2nd2) = O(2n2) flops.
The work using block tridiagonal solver is also O(n2).
Need better methods (next time).
The Poisson matrix and Kronecker Products – p. 12/27
Kronecker ProductDefinition 1. For any positive integers p, q, r, s we define the Kronecker product of two
matrices A ∈ Rp,q and B ∈ R
r,s as a matrix C ∈ Rpr,qs given in block form as
C =
Ab1,1 Ab1,2 · · · Ab1,s
Ab2,1 Ab2,2 · · · Ab2,s
......
...
Abr,1 Abr,2 · · · Abr,s
.
We denote the Kronecker product of A and B by C = A ⊗ B.
Defined for rectangular matrices of any dimension
# of rows(columns) = product of # of rows(columns) in A and B
The Poisson matrix and Kronecker Products – p. 13/27
Kronecker Product Example
T =
2 −1
−1 2
, I =
1 0
0 1
T ⊗ I =
T 0
0 T
=
2 −1 0 0
−1 2 0 0
0 0 2 −1
0 0 −1 2
I ⊗ T =
2I −I
−I 2I
=
2 0 −1 0
0 2 0 −1
−1 0 2 0
0 −1 0 2
The Poisson matrix and Kronecker Products – p. 14/27
Poisson Matrix = T ⊗ I + I ⊗ T
A =
T
T
T
+
2I −I 0
−I 2I −I
0 −I 2I
= T ⊗ I + I ⊗ T
Definition 2. Let for positive integers r, s, k, A ∈ Rr,r , B ∈ R
s,s and Ik be the
identity matrix of order k. The sum A ⊗ Is + Ir ⊗ B is known as the Kronecker sum of
A and B.
The Poisson matrix is the Kronecker sum of T with itself.
The Poisson matrix and Kronecker Products – p. 15/27
Matrix equation ↔ KroneckerGiven A ∈ R
r,r, B ∈ Rs,s, F ∈ R
r,s. Find V ∈ Rr,s such that
AV BT = F
For B ∈ Rm,n define
vec(B) :=
b1
b2
...
bn
∈ Rmn, bj =
b1j
b2j
...
bmj
jth column
Lemma 1.
AV BT = F ⇔ (A ⊗ B) vec(V ) = vec(F ) (6)
AV + V BT = F ⇔ (A ⊗ Is + Ir ⊗ B) vec(V ) = vec(F ). (7)
The Poisson matrix and Kronecker Products – p. 16/27
ProofWe partition V , F , and BT by columns as V = (v1, . . . , vs),F = (f1, . . . , fs) and BT = (b1, . . . , bs). Then we have
(A ⊗ B) vec(V ) = vec(F )
⇔
Ab11 · · · Ab1s
......
Abs1 · · · Abss
v1
...
vs
=
f1
...
fs
⇔ A∑
j
bijvj = fi, i = 1, . . . , m
⇔ [AV b1, . . . , AV bs] = F ⇔ AV BT = F .
This proves (6)
The Poisson matrix and Kronecker Products – p. 17/27
Proof Continued
(A ⊗ Is + Ir ⊗ B) vec(V ) = vec(F )
⇔ (AV ITs + IrV BT ) = F ⇔ AV + V BT = F .
�
This gives a slick way to derive the Poisson matrix. Recall
AV + V BT = F ⇔ (A ⊗ Is + Ir ⊗ B) vec(V ) = vec(F )
Therefore,
TV + V T = h2F ⇔ (T ⊗ I + I ⊗ T ) vec(V ) = h2 vec(F )
The Poisson matrix and Kronecker Products – p. 18/27
Mixed Product Rule
(A ⊗ B)(C ⊗ D) = (AC) ⊗ (BD)
Proof If B ∈ Rr,t and D ∈ R
t,s for some integers r, t then
Ab1,1 · · · Ab1,t
......
Abr,1 · · · Abr,t
Cd1,1 · · · Cd1,s
......
Cdt,1 · · · Cdt,s
=
E1,1 · · · E1,s
......
Er,1 · · · Er,s
,
where for all i, j
Ei,j =
t∑
k=1
bi,kdk,jAC = (AC)(BD)i,j = ((AC) ⊗ (BD))i,j .
where in the last formula i, j refers to the ij-block in the Kroneckerproduct. �
The Poisson matrix and Kronecker Products – p. 19/27
Properties of Kronecker ProductsThe usual arithmetic rules hold. Note however that
(A ⊗ B)T = AT ⊗ BT
(A ⊗ B)−1 = A−1 ⊗ B−1 if A and B are nonsingular
A ⊗ B 6= B ⊗ A
(A ⊗ B)(C ⊗ D) = (AC) ⊗ (BD) (mixed product rule)
The mixed product rule is valid for any matrices as long as the productsAC and BD are defined.
The Poisson matrix and Kronecker Products – p. 20/27
Eigenvalues and EigenvectorsThe Kronecker product of two vectors u ∈ R
p and v ∈ Rr is a vector
u ⊗ v ∈ Rpr given by u ⊗ v =
[
uT v1, . . . , uT vr
]TSuppose now A ∈ R
r,r
and B ∈ Rs,s and
Aui = λiui, i = 1, . . . , r, Bvj = µjvj , j = 1, . . . , s,
then for i = 1, . . . , r, j = 1, . . . , s
(A ⊗ B)(ui ⊗ vj) = λiµj(ui ⊗ vj), (8)
(A ⊗ Is + Ir ⊗ B)(ui ⊗ vj) = (λi + µj)(ui ⊗ vj). (9)
Thus the eigenvalues of a Kronecker product(sum) are the products(sums) of the eigenvalues of the factors. The eigenvectors of a Kroneckerproduct(sum) are the products of the eigenvectors of the factors.
The Poisson matrix and Kronecker Products – p. 21/27
Proof of Eigen-formulaeThis follows directly from the mixed product rule. For (8)
(A ⊗ B)(ui ⊗ vj) = (Aui) ⊗ (Bvj) = (λiui) ⊗ (µjvj) = (λiµj)(ui ⊗ vj).
From (8)
(A ⊗ Is)(ui ⊗ vj) = λi(ui ⊗ vj), and (Ir ⊗ B)(ui ⊗ vj) = µj(ui ⊗ vj)
The result now follows by summing these relations. �
The Poisson matrix and Kronecker Products – p. 22/27
The Poisson Matrix A = T ⊗ I + I ⊗ T
Recall eigenvalues and eigenvectors of the T matrix. Let h = 1/(m + 1).
1. We know that Tsj = λjsj for j = 1, . . . , m, where
sj = (sin (jπh), sin (2jπh), . . . , sin (mjπh))T , (10)
λj = 2 − 2 cos (jπh) =4 sin2 (jπh
2). (11)
2. The eigenvalues are distinct and the eigenvectors are orthogonal
sTj sk =
1
2hδj,k, j, k = 1, . . . , m. (12)
The Poisson matrix and Kronecker Products – p. 23/27
Eigenvalues/-vectors A = T ⊗ I + I ⊗ T
1. We have Axj,k = λj,kxj,k for j, k = 1, . . . , m, where
xj,k = sj ⊗ sk, (13)
sj = (sin (jπh), sin (2jπh), . . . , sin (mjπh))T , (14)
λj,k = 4 sin2 (jπh
2) + 4 sin2 (
kπh
2). (15)
2. The eigenvectors are orthogonal
xTj,kxp,q =
1
4h2δj,pδk,q, j, k, p, q = 1, . . . , m. (16)
3. A is symmetric AT = (T ⊗ I + I ⊗ T )T = T T ⊗ IT + IT ⊗ T T = A
4. A is positive definite (positive eigenvalues)
The Poisson matrix and Kronecker Products – p. 24/27
A = T ⊗ I + I ⊗ T for m = 2
s1 =
sin(π3)
sin( 2π3
)
=
√3
2
1
1
, s2 =
sin( 2π3
)
sin( 4π3
)
=
√3
2
1
−1
λ1 = 4 sin2 (π
6) = 1, λ2 = 4 sin2 (
2π
6) = 3
Axij = µijxij , µij = λi + λj , xij = si ⊗ sj i, j = 1, 2
µ11 = 2, µ12 = µ21 = 4, µ22 = 6.
s1 ⊗ s1 =3
4[1, 1, 1, 1]T , s1 ⊗ s2 =
3
4[1, 1,−1,−1]T ,
s2 ⊗ s1 =3
4[1,−1, 1,−1]T , s2 ⊗ s2 =
3
4[1,−1,−1, 1]T
xTj,kxp,q =
9
4δj,pδk,q, j, k, p, q = 1, 2.
The Poisson matrix and Kronecker Products – p. 25/27
Check
A =
2 −1
−1 2
⊗
1 0
0 1
+
1 0
0 1
⊗
2 −1
−1 2
4 −1 −1 0
−1 4 0 −1
−1 0 4 −1
0 −1 −1 4
1
1
1
1
= 2
1
1
1
1
,
4 −1 −1 0
−1 4 0 −1
−1 0 4 −1
0 −1 −1 4
1
1
−1
−1
= 4
1
1
−1
−1
etc.
The Poisson matrix and Kronecker Products – p. 26/27
SummaryStudied the Poisson matrix A
Shown that A can be written as a Kronecker sum
studied properties of general Kronecker products and Kroneckersums
used this to derive properties of A
The Poisson matrix and Kronecker Products – p. 27/27