Lecture 10: Miscellaneous
Applications
1
1. Linear integral operators
Let L2 = L2[a, b], the Lebesgue square integrable functions on
the finite interval [a, b]. Let K(s, t) be an L2–kernel on
a ≤ s, t,≤ b,
i.e.,
∫ b
a
∫ b
a
|K(s, t)|2ds dt exists and is finite.
Consider the two operators T1, T2 ∈ B(L2, L2) defined by
(T1x)(s) =
∫ b
a
K(s, t)x(t)dt , a ≤ s ≤ b ,
(T2x)(s) = x(s) −∫ b
a
K(s, t)x(t)dt , a ≤ s ≤ b ,
called Fredholm integral operators of the first kind and the
second kind, respectively. Then
(a) R(T2) is closed,
(b) R(T1) is nonclosed unless it is finite dimensional.
2
The Fredholm integral equation of the 2nd kind
x(s) − λ
∫ b
a
K(s, t) x(t) dt = y(s) , a ≤ s ≤ b , (1)
is also written as(I − λK)x = y ,
where λ and all functions are complex, and [a, b] is a bounded
interval.
We need the following facts from the Fredholm theory of integral
equations. For any λ, K as above
(a) (I − λK) ∈ B(L2, L2) ,
(b) (I − λK)∗ = I − λK∗ , where K∗(s, t) = K(t, s) .
(c) The null spaces N(I − λK) and N(I − λK∗) have equal finite
dimensions,
dim N(I − λK) = dim N(I − λK∗) = n(λ) , say . (2)
3
Fredholm (cont’d)
(d) A scalar λ is called a regular value of K if n(λ) = 0, in
which case the operator I − λK has an inverse
(I − λK)−1 ∈ B(L2, L2) written as
(I − λK)−1 = I + λR , (3)
where R = R(s, t; λ) is an L2–kernel called the resolvent of K.
(e) A scalar λ is called an eigenvalue of K if n(λ) > 0, in which
case any nonzero x ∈ N(I − λK) is called an eigenfunction of K
corresponding to λ.
For any λ and, in particular, for any eigenvalue λ, both range
spaces R(I − λK) and R(I − λK∗) are closed and,
R(I − λK) = N(I − λK∗)⊥ , R(I − λK∗) = N(I − λK)⊥ . (4)
4
Fredholm (cont’d)
(f) If λ is a regular value of K then (1) has, for any y ∈ L2, a
unique solution given by
x = (I + λR)y ,
or,
x(s) = y(s) + λ
∫ b
a
R(s, t, λ) y(t) dt , a ≤ s ≤ b . (5)
(g) If λ is an eigenvalue of K then (1) is consistent if and only if
y is orthogonal to every u ∈ N(I − λK∗), in which case the
general solution of (1) is
x = x0 +
n(λ)∑
i=1
cixi , ci arbitrary scalars ,
x0 a particular solution, {x1, . . . ,xn(λ)} a basis of N(I − λK).
5
Pseudo resolvents
Let λ be an eigenvalue of K. Following Hurwitz, an L2–kernel
R = R(s, t, λ) is called a pseudo resolvent of K if for any
y ∈ R(I − λK), the function
x(s) = y(s) + λ
∫ b
a
R(s, t, λ) y(t) dt (5)
is a solution of (1).
Hurwitz constructed a pseudo resolvent as follows.
Let λ0 be an eigenvalue of K, and let {x1, . . . ,xn} and
{u1, . . . ,un} be o.n. bases of N(I − λ0K) and N(I − λ0K∗)
respectively. Then λ0 is a regular value of the kernel
K0(s, t) = K(s, t) − 1
λ0
n∑
i=1
ui(s) xi(t) , (6)
6
Pseudo resolvents (cont’d)
The eigenvalue λ0 is a regular value of
K0(s, t) = K(s, t) − 1
λ0
n∑
i=1
ui(s) xi(t) , (6)
written for short as K0 = K − 1
λ0
n∑
i=1
uix∗i
and the resolvent R0 of K0 is a pseudo resolvent of K, satisfying
(I + λ0R0)(I − λ0K)x = x , for all x ∈ R(I − λ0K∗)
(I − λ0K)(I + λ0R0)y = y , for all y ∈ R(I − λ0K) (7)
(I + λ0R0)ui = xi , i = 1, . . . , n .
If R is a pseudo resolvent of K, then I + λR is a {1}–inverse of
I − λK. As with {1}–inverses, the pseudo resolvent is not unique.
7
Characterization of pseudo resolvents
The pseudo resolvent is not unique: For R0,ui,xi as above, and
any scalars cij , the kernel R0 +∑n
i,j,=1 cijxiu∗j is a pseudo
resolvent of K.
Theorem (Hurwitz). Let K be an L2–kernel, λ0 be an eigenvalue
of K and {x1, . . . ,xn} and {u1, . . . ,un} be orthonormal bases of
N(I − λ0K) and N(I − λ0K∗) respectively. An L2–kernel R is a
pseudo resolvent of K if and only if
R = K + λ0KR − 1
λ0
n∑
i=1
βiu∗i , (8a)
R = K + λ0RK − 1
λ0
n∑
i=1
xiα∗i , (8b)
where αi, βi ∈ L2 satisfy
〈αi,xj〉 = δij , 〈βi,uj〉 = δij , i, j = 1, . . . , n . (9)
8
Characterization (cont’d)
Here KR stands for the kernel
KR(s, t) =
∫ b
a
K(s, u)R(u, t) du
If λ is a regular value of K then (8a)–(8b) reduce to
R = K + λKR , R = K + λRK ,
which uniquely determines the resolvent R(s, t, λ).
9
Degenerate kernels
A kernel K(s, t) is called degenerate if it is a finite sum of
products of L2 functions, as follows:
K(s, t) =m∑
i=1
fi(s) gi(t) . (10)
Degenerate kernels are convenient because they reduce the integral
equation (1) to a finite system of linear equations. Also, any
L2–kernel can be approximated, arbitrarily close, by a degenerate
kernel.
Let K(s, t) be given by (10). Then
(a) The scalar λ is an eigenvalue of (10) if and only if 1/λ is an
eigenvalue of the m × m matrix
B = [bij ] , where bij =
∫ b
a
fj(s) gi(s) ds .
10
Degenerate kernels (cont’d)
(b) Any eigenfunction of K [K∗] corresponding to an eigenvalue λ
[λ] is a linear combination of the m functions f1, . . . , fm
[g1, . . . , gm].
(c) If λ is a regular value of (10), then the resolvent at λ is
R(s, t, ; λ) =
det
0... f1(s) · · · fm(s)
· · · · · · · · · · · · · · ·
−g1(t)...
...... I − λB
−gm(t)...
det(I − λB).
11
Example
Consider the equation
x(s) − λ
∫ 1
−1
(1 + 3st) x(t) dt = y(s) (11)
with K(s, t) = 1 + 3st. The resolvent is
R(s, t; λ) =1 + 3st
1 − 2λ.
K has a single eigenvalue λ = 12 and an o.n. basis of N(I − 1
2K) is
{x1(s) =
1√2, x2(s) =
√3√2
s
}
which, by symmetry, is also an orthonormal basis of N(I − 12K∗).
12
Example (cont’d)
From (6) we get
K0(s, t) = K(s, t) − 1
λ0
∑ui(s) xi(t)
= (1 + 3st) − 2
(1√2
1√2
+
√3√2s
√3√2t
)
= 0 ,
and the resolvent of K0(s, t) is therefore
R0(s, t; λ) = 0 .
(a) If λ 6= 12 , then for each y ∈ L2[−1, 1] equation (11) has a unique
solution,
x(s) = y(s) + λ
∫ 1
−1
1 + 3st
1 − 2λy(t) dt .
13
Example (cont’d)
(b) If λ = 12 , then (11) is consistent if and only if
∫ 1
−1
y(t) dt = 0 ,
∫ 1
−1
t y(t) dt = 0 ,
in which case the general solution is
x(s) = y(s) + c1 + c2s , c1, c2 arbitrary .
14
2. Linear systems theory
Systems modeled by linear differential equations call for symbolic
computation of generalized inverses for matrices whose elements are
rational functions.
As example, consider the homogeneous system
A(D)x(t) = 0 (1)
where x(t) : [0−,∞) → Rn, D :=d
dt,
A(D) = AqDq + · · · + A1D + A0 , (2)
and Ai ∈ Rm×n , i = 0, 1, . . . , q. Let L denote the Laplace
transform, and let x̂(s) = L(x(t)). The system (1) transforms to
A(s)x̂(s) = b̂(s) ,
allowing algebraic solution.
15
Linear systems theory (cont’d)
Theorem (Jones, Karampetakis and Pugh) The system (1)
has a solution if and only if
A(s)A(s)†b̂(s) = b̂(s) (3)
in which case the general solution is
x(t) = L−1(x̂(s)) = L−1{
A(s)†b̂(s) + (In − A(s)†A(s))y(s)}
(4)
where y(s) ∈ Rn(s) is arbitrary. �
16
3. Tchebycheff approximation
A Tchebycheff approximate solution of the system
Ax = b (1)
is a vector x minimizing the Tchebycheff norm
‖r‖∞ = maxi=1,...,m
{|ri|}
of the residual vector
r = b− Ax . (2)
Let A ∈ C(n+1)×nn and b ∈ Cn+1 be such that (1) is inconsistent.
Then (1) has a unique Tchebycheff approximate solution given by
x = A†(b + r) , (3)
17
Tchebycheff approximation (cont’d)
where the residual r = [ri] is
ri =
n+1∑j=1
|(PN(A∗)b)j |2
n+1∑j=1
|(PN(A∗)b)j |
(PN(A∗)b)i
|(PN(A∗)b)i|, i ∈ 1, n + 1 . (4)
Proof. From
r(x) − b = −Ax ∈ R(A)
it follows that any residual r satisfies
PN(A∗)r = PN(A∗)b
or equivalently
〈PN(A∗)b, r〉 = 〈b, PN(A∗)b〉 , (5)
since dimN(A∗) = 1 and b 6∈ R(A).
18
Tchebycheff approximation (cont’d)
Equation (5) represents the hyperplane of residuals. A routine
computation now shows, that among all residuals r satisfying (5)
there is a unique residual of minimum Tchebycheff norm
given by (4), from which (3) follows since N(A) = {0}.
19
4. Interval linear programming
For two vectors u = (ui),v = (vi) ∈ Rm let
u ≤ v
denote the fact that ui ≤ vi for i ∈ 1, m. A linear programming
problem of the form
maximize {cT x : a ≤ Ax ≤ b} , (1)
with given a,b ∈ Rm; c ∈ Rn; A ∈ Rm×n, is called an interval
linear program and denoted by IP (a,b, c, A) or simply by IP .
The IP (1) is consistent (also feasible) if the set
F = {x ∈ Rn : a ≤ Ax ≤ b} 6= ∅ (2)
in which case the elements of F are the feasible solutions of
IP (a,b, c, A).
20
Interval linear programming (cont’d)
A consistent IP (a,b, c, A) is bounded if
max {cT x : x ∈ F}
is finite, in which case the optimal solutions of IP (a,b, c, A) are
its feasible solutions x0 which satisfy
cT x0 = max{cT x : x ∈ F} .
Lemma. Let a,b ∈ Rm; c ∈ Rn; A ∈ Rm×n be such that
IP (a,b, c, A) is consistent. Then IP (a,b, c, A) is bounded if and
only if
c ∈ N(A)⊥ . (3)
Proof. F = F + N(A), etc. �
21
Interval linear programming (cont’d)
Let η : Rm × R
m × Rm → R
m be defined for u,v,w ∈ Rm by
ηi =
ui if wi < 0,
vi if wi > 0,
λiui + (1 − λi)vi where 0 ≤ λi ≤ 1 , if wi = 0
(4)
Theorem. Let a,b ∈ Rm; c ∈ R
n; A ∈ Rm×nm (full row-rank) be
such that IP (a,b, c, A) is consistent and bounded, and let A(1) be
any {1}–inverse of A. Then the general optimal solution of
IP (a,b, c, A) is
x = A(1)η(a,b, A(1)Tc) + y , y ∈ N(A) . (5)
Proof. For u = Ax, the problem (1) is
max {cT A(1)u : a ≤ u ≤ b} , etc.
Note: The rank assumption is a severe restriction of usefulness.
22
5. Nonlinear least squares solutions
Let f : Rn → R
m, and let
Jf (x) =
(∂fi(x)
∂xj
).
If the Newton method
x+ := x − Jf (x)† f(x)
converges to x∞, plus 2 more if ’s, then
Jf (x∞)† f(x∞) = 0
and x∞ is a stationary point of ‖f(x)‖2.
A Maple code for a Newton method using the Moore–Penrose
inverse of the Jacobi matrix is available, contact the instructor or
see http://benisrael.net/Newton-MP.pdf
23