Abstract The effect of wavefront distortion due to turbulence in the Earth’s atmo- sphere has made it necessary to develop tools for ground-based telescopes that compensate for this aberration. Therefore, the technology of Adaptive Optics (AO) has been investigated and used in astronomy. However, AO sys- tems require to solve large linear ill-conditioned systems. This thesis deals with possible approaches to find regularized solutions of such systems. We start with an introduction to Adaptive Optics and the underlying mathematical modeling. The second chapter deals with regularization of ill- posed problems and introduces some regularization methods. In Chapter 3, we discuss whether some bases are likely to yield sparse representations of the desired solution and present the algorithm of iterative soft-thresholding that promotes sparsity. Finally, we introduce an accelerated version of this method and conclude by giving some numerical results. 1
Transcript
Abstract
The effect of wavefront distortion due to turbulence in the Earth’s atmo-sphere has made it necessary to develop tools for ground-based telescopesthat compensate for this aberration. Therefore, the technology of AdaptiveOptics (AO) has been investigated and used in astronomy. However, AO sys-tems require to solve large linear ill-conditioned systems. This thesis dealswith possible approaches to find regularized solutions of such systems.
We start with an introduction to Adaptive Optics and the underlyingmathematical modeling. The second chapter deals with regularization of ill-posed problems and introduces some regularization methods. In Chapter 3,we discuss whether some bases are likely to yield sparse representations ofthe desired solution and present the algorithm of iterative soft-thresholdingthat promotes sparsity. Finally, we introduce an accelerated version of thismethod and conclude by giving some numerical results.
1
Zusammenfassung
Turbulenzen in der erdnahen Atmosphare, auch Luftunruhe genannt, verur-sachen Fehler in Bildern von Himmelskorpern, die von erdstationiertenTeleskopen erzeugt werden. Daher ist es notwendig, diese Teleskope mit derTechnik der Adaptiven Optik (AO) auszustatten, die die verzerrten Wellen-fronten, die das Teleskop erreichen, korrigiert. Ein Teil dieser Korrekturdurch AO Systeme verlangt das Losen großdimensionierter linearer schlechtkonditionierter Probleme. Die vorliegende Arbeit befasst sich mit moglichenZugangen eine regularisierte Losung solcher Probleme zu geben.
Zu Beginn geben wir eine kurze Einfuhrung in Adaptive Optik und demzugrundeliegenden mathematischen Modell. Das zweite Kapitel beinhalteteinen Uberblick uber die Theorie der Regularisierung schlecht gestellter Prob-leme und fuhrt bekannte Regularisierungsmethoden ein. In Kapitel 3 behan-deln wir die Frage, ob es zur gesuchten Losung eine Basis gibt, in der sie sichgut, also mit wenig Koeffizienten ungleich 0, darstellen lasst. Weiters stellenwir einen iterativen Algorithmus vor, der auf Soft-Thresholding basiert unddunnbesetzte Losungen bevorzugt. Abschließend behandeln wir eine beschle-unigte Version dieser Methode und prasentieren numerische Ergebnisse.
2
Acknowledgement
I owe my deepest gratitude to my supervisor Prof. Ronny Ramlau whosupported me throughout my work on this thesis and always took the timeto answer my questions and make suggestions. I would also like to thank mycolleagues Dr. Mariya Zhariy and Dr. Tapio Helin who encouraged me andhelped me to develop an understanding of the subject. Last but not least Iwould like to show my gratitude to my family and friends.
3
List of Figures
1.1 Correction of the incoming wavefront. . . . . . . . . . . . . . . 71.2 Achievable improvement with adaptive optics. On the left:
Image dominated by atmospheric turbulence. On the right:Corrected image clearly shows the binary star. . . . . . . . . . 8
1.3 Configuration of an AO system. . . . . . . . . . . . . . . . . . 91.4 A deformable mirror. . . . . . . . . . . . . . . . . . . . . . . . 91.5 Schematic representation of the Shack-Hartmann WFS. . . . . 101.6 Discretization of the aperture for n = 10. The circle de-
scribes the pupil of the telescope. Shaded squares are thepupil-masked subapertures and dots represent DM actuators. . 13
1.7 Condition number w.r.t. matrix size. . . . . . . . . . . . . . . 142.1 Original phase screen. . . . . . . . . . . . . . . . . . . . . . . 302.2 Reconstructed phase screens for different noise levels, τ = 1.2. 312.3 Relative error in φ w.r.t. noise level δ, τ = 1.2. . . . . . . . . . 323.1 φ and ψ for Daubechies wavelets. . . . . . . . . . . . . . . . . 443.2 1D slice of original phase screen. . . . . . . . . . . . . . . . . . 483.3 Sparsity pattern of P for the Haar basis. . . . . . . . . . . . . 503.4 Sparsity pattern of P for the Daubechies wavelets. . . . . . . . 503.5 Reconstructed 1D phase screens for Haar basis, p = 1 and
different noise levels. . . . . . . . . . . . . . . . . . . . . . . . 513.6 Error in φ w.r.t. noise level δ. Haar basis, p = 1. . . . . . . . . 523.7 Error in φ w.r.t. to noise level δ for FISTA with p = 1. . . . . 554.1 Performance of ISTA and FISTA for p = 1. . . . . . . . . . . . 564.2 Performance of ISTA and FISTA for p = 1.5 and p = 1.9. . . . 584.3 Distribution of absolute values of wavelet coefficients around
zero for different values of p. x-axis: absolute values of coeffi-cients, y-axis: number of coefficients. . . . . . . . . . . . . . . 59
3.4 A Fast Iterative Soft-Thresholding Algorithm . . . . . . . . . 52
4 Numerical Results and Conclusion 56
References 60
5
1 Introduction
We start with a (very) brief introduction to Adaptive Optics. For a detaileddiscussion we refer to [8].
1.1 Imaging Through the Atmosphere
When light from an astronomical object, e.g. a star, propagates throughthe atmosphere, it undergoes some turbulence when entering the Earth’satmosphere. This is due to the interaction between different temperaturelayers and wind speeds. An indicator for turbulence in fluids is the Reynoldsnumber. If the Reynolds number is below a critical value, the flow willbe laminar, otherwise turbulence will occur. The Reynolds number is adimensionless parameter given as
Re =inertial forces
viscous forces=V l
kν,
where V is a characteristic velocity, l is a characteristic size and kν is thekinematic viscosity of the fluid. Close to Earth, due to the solar heatingof its surface, convection currents are caused and the kinematic viscosityis kν = 15 · 10−6m2s−1. For typical characteristic velocities (V > 1m/s)and characteristic lengths l of several meters to kilometers, this results inReynolds numbers Re > 106, which are sufficiently large for turbulence tooccur, [12].
Since a beam of light that propagates through the atmosphere suffersfrom this turbulence, images of astronomical objects taken from Earth areblurred and distorted. This is the biggest challenge for Earth-based as-tronomy. Adaptive Optics deals with developing devices for ground-basedtelescopes that can compensate this distortion. The main idea of AdaptiveOptics is the following:Since the object of interest is assumed to be far away from Earth, the prop-agating wavefronts are almost plane. Due to atmospheric turbulence theybecome perturbed and are no longer planar. A beam of light can be de-scribed by an electric field of the form
Ae+iφatm,
where A is the amplitude and φatm is the phase of the beam. If a mirror canbe shaped according to the conjugated phase, i.e.
Ae−iφatm,
6
Figure 1.1: Correction of the incoming wavefront.
and the light emitted by the astronomical object is reflected at this mirrorone obtains plane wavefronts and has thus corrected the distortion (see Fig.1.11). Of course, this is only a very basic and schematic description of anAdaptive Optics (AO) system. The main challenge is to achieve a high-speed,real-time correction for the turbulence. Figure 1.2 2 shows an example of howmuch Adaptive Optics can improve the quality of an image.
1.2 Adaptive Optics Components
We now want to go into more detail about how an AO system looks like, i.e.what its main components are. Typically, an AO system consists of
1. a deformable mirror (DM),
2. a wavefront sensor (WFS) and
3. a control computer
(see Fig. 1.3 3). In order to compensate for the atmospheric turbulence, the
1http://en.wikipedia.org/wiki/Adaptive optics2[16], p. 43[16], p. 2
7
Figure 1.2: Achievable improvement with adaptive optics. On the left: Imagedominated by atmospheric turbulence. On the right: Corrected image clearlyshows the binary star.
shape of the DM is adapted in real-time to follow the wavefront aberrations.The control computer receives measurement signals from the WFS and gen-erates control signals to drive the DM. The WFS is located after the DM inthe optical path. Thus, it measures the residual in the wavefront perturba-tion after the DM correction has been applied. The aim is to minimize thisresidual by the AO control loop. Eventually, the corrected wavefront is sentto an astronomical instrument, such as an imaging camera, which is locatedin the focal plane of the telescope.
Deformable Mirrors
A deformable mirror consists of a continuous reflective facesheet that is de-formed by a set of actuators placed at the back of it (see Fig. 1.4 4). Thereare several designs for the DM and especially for the actuators. The lattercan be electromechanical, electromagnetic, piezoelectric or magnetostrictiveunits. Most commonly, the actuators are piezoelectric elements. The mostimportant parameters in the design of DMs are
- the shape of the influence functions (see Chapter 1.3).
Figure 1.4: A deformable mirror.
Wavefront Sensors
The WFS measures the wavefront distortions caused by atmospheric tur-bulence. However, most wavefront sensors do not measure the wavefrontdirectly but rather its first derivative (wavefront slopes) or second derivative(curvature). The most popular WFS is the Shack-Hartmann type.
For the Shack-Hartmann WFS, the lenslet array is optically conjugatedto the pupil plane of the telescope. This n × n grid of tiny lenses spatiallysamples the distorted wavefront. Each lens forms a small part of the image,corresponding to a part of the aperture (subaperture), onto a detector locatedin the focal plane of the lenslet array. If the wavefront in the pupil is plane,the subaperture images, called spots, form a regular grid. A distorted wave-
9
front results in a displacement of the spots (see Fig. 1.5 5). It is intuitivelyunderstood that the displacements of the spots in two orthogonal directionsx and y are proportional to the average wavefront slopes sx, sy in x and yover the corresponding subapertures. Thus, a Shack-Hartmann WFS mea-sures the local gradient error. The wavefront is then reconstructed from the
array s =
[
sxsy
]
∈ R2n2
of the measured slopes.
Figure 1.5: Schematic representation of the Shack-Hartmann WFS.
1.3 Mathematical Modeling
Our aim is to reconstruct the incoming wavefront φatm = φatm(x) ∈ H1(R2)from given WFS measurements s ∈ R
2n2
in order to compute the DM com-mands that are needed to shape the DM such that it is optically conjugatedto φatm.There are different setups of how the DM actuators are arranged. The mostcommonly used one for Shack-Hartmann WFSs is Fried geometry. In thisgeometry, the actuators are located at the corners of the subapertures.
By introducing a phase-to-WFS interaction operator G : H1(R2) −→R
For a Shack-Hartmann WFS, Gφatm is the gradient of the phase averagedover those subapertures that are located entirely inside the pupil, i.e. G hasthe following structure:
G = [Gx, Gy] = [MΓx,MΓy],
where for the i-th subaperture Ωi
[Γxφatm]i =
∫
Ωi
∂φatm∂x
dx
and similarly for Γy. Furthermore, the mask M is defined as the diagonalmatrix whose i-th diagonal entry is 1 if Ωi is completely contained in thepupil and 0 otherwise.Since H1(R2) ⊂ L2(R2) and L2(R2) = L2(R) ⊗ L2(R) we can use a basisbi∞i=1 of L2(R) to represent the wavefront φatm ∈ H1(R2):
φatm(x, y) =
∞∑
i,j=1
ϕi,jbi(x)bj(y) = Bϕ, (1.2)
where ϕi,j is the (i,j)-th coefficient of φatm in this basis representation and Bdenotes the operator that maps the coefficients to φatm ∈ H1(R2).By inserting (1.2) in (1.1) we get
s = GBϕ = Pϕ, (1.3)
where P = GB denotes the DM-to-WFS (or Poke) operator and ϕ = (ϕi,j)i,j∈N.For the actual computation of the corrected wavefront φrec which is generatedby the deformable mirror we use a discrete approach. Therefore we computea solution in finite dimensional spaces Xk ⊂ L2(R2) for which
X1 ⊂ X2 ⊂ X3 ⊂ . . . ,⋃
k∈N
Xk = L2(R2),
where k is the dimension of Xk. If the telescope pupil consists of n × nsubapertures and if we neglect the mask M , we end up with N := (n + 1)2
DM commands. Hence, we consider the space XN and represent φrec via thebasis bin+1
i=1 of the corresponding subspace of L2(R) and coefficients ai,j:
φrec(x, y) =
n+1∑
i,j=1
ai,jbi(x)bj(y). (1.4)
11
We assume that bin+1i=1 is a basis that is shift invariant and locally supported,
and we can thus define hj for j = (j2 − 1) · (n + 1) + j1 by
hj(x, y) := b(x− xj1
∆
)
b(y − yj2
∆
)
.
Here, ∆ := 1n
and b is defined such that
b(x− xj1
∆
)
= bj1(x),
b(y − yj2
∆
)
= bj2(y).
With this new representation we get
φrec = Ha =
N∑
j=1
ajhj , (1.5)
where a ∈ RN is the vector of DM commands, hj’s are the influence functions
and H : RN −→ H1(R2) is the DM-to-phase operator. Note that we needsufficiently smooth influence functions in order to guarantee that H maps toH1(R2). The influence functions are called bilinear, if
b(z) =
− |z| + 1 |z| ≤ 1,
0 otherwise,
and they are called bicubic, if b is a cubic b-spline supported on the interval[−2, 2].By inserting (1.5) in (1.1) we can introduce the DM-to-WFS matrix (Pokematrix) P as the product GH , i.e.
s = GHa = Pa. (1.6)
The entries of the Poke matrix P = [Px,Py] are then determined by
Pxi,j =
∫
Ωi
∂hj(x, y)
∂xd(x, y), (1.7)
=
∫ yi2+1
yi2
(
hj(xi1+1, y) − hj(xi1 , y))
dy,
Pyi,j =
∫
Ωi
∂hj(x, y)
∂yd(x, y), (1.8)
=
∫ xi1+1
xi1
(
hj(x, yi2+1) − hj(x, yi2))
dx,
12
where Ωi = [xi1 , xi1+1]× [yi2, yi2+1]. In practice, the slopes s will not be givenexactly but perturbed data sδ will be available, i.e. sδ = s + ηδ for somerandom noise vector ηδ. Then, (1.6) reads
sδ = Pa. (1.9)
Note that the total number of slope measurements is approximately twicethe number of DM commands. This redundancy in the measurements hasthe beneficial effect of smoothing random errors, which do not accumulatein contrast to the case where there is no redundancy.
1.4 Bilinear Influence Functions
In this section, we assume to have a discretization with n-by-n subapertures(see Fig. 1.4). For bilinear influence functions (1.7) and (1.8) reduce to
The operator PTP, where P is the Poke matrix, has zero values in its spec-
Figure 1.6: Discretization of the aperture for n = 10. The circle describesthe pupil of the telescope. Shaded squares are the pupil-masked subaperturesand dots represent DM actuators.
trum which means that there is no unique solution of the problem. For illus-
tration purposes, we determine κ(PTP) = |λmax(PT P)||λmin(PT P)| by taking λmin(P
TP)
as the smallest non-zero eigenvalue in norm. Otherwise, κ(PTP) = ∞ forany n and no comparison could be possible. Figure 1.7 shows how κ(PTP)grows as n gets larger.
13
10 20 30 40 50 60 70 80 90 1000
500
1000
1500
2000
2500
3000
3500
4000
4500
n, where pupil has n x n subapertures
κ(P
T P
)
Figure 1.7: Condition number w.r.t. matrix size.
Large condition numbers can cause a problem for noisy data. Consider alinear equation system
Ax = b. (1.10)
If the data is noisy we need to change the equation to
A(x+ ∆x) = b+ ∆b
and the change in x is∆x = A−1∆b. (1.11)
The corresponding norm estimates to (1.10) and (1.11) are
‖∆x‖2 ≤∥
∥A−1∥
∥
2‖∆b‖2 , (1.12)
‖b‖2 ≤ ‖A‖2 ‖x‖2 . (1.13)
These two inequations yield an estimate for the relative error in x:
‖∆x‖‖x‖ ≤ κ(A)
‖∆b‖‖b‖ .
Therefore, if the condition number κ(A) is large, a small perturbation in bcan cause a large error in x.
14
But even if κ(A) is not large, small eigenvalues of A can still cause problems.If all the eigenvalues of a symmetric matrix A are small, the condition number
κ(A) =|λmax(A)||λmin(A)|
will be moderate. Nevertheless, the eigenvalues of A−1 will be large, sincethey are reciprocal to the eigenvalues of A. Since we get ∆x by multiplyingwith A−1 (see (1.11)), the data error can still be amplified.Systems with large condition numbers are called ill-conditioned. Growingcondition numbers are often due to the fact that the underlying continuousproblem is ill-posed, a property that is defined in the next chapter. There,we also introduce methods for solving ill-posed problems which we can usefor tackling the ill-conditioned system (1.9).
15
2 Linear Ill-Posed Problems
The following introduction to ill-posed problems is mainly based on [7].Let T : X → Y be a bounded linear operator and consider X and Y to
be Hilbert spaces. We are interested in solving
Tx = y (2.1)
for given y ∈ Y .The problem is called ill-posed if it is not well-posed. The following definitionof well-posedness goes back to J. Hadamard:
Definition 2.1. A problem is well-posed if and only if the following propertieshold:
(i) For all admissible data, a solution exists: R(T ) = Y.
(ii) For all admissible data, the solution is unique: N (T ) = 0.
(iii) The solution depends continuously on the data: T−1 ∈ L(Y ,X ).
Therefore, a problem is ill-posed if one of these properties is violated.If the last criterion is not fulfilled, serious problems can occur when ap-
plying usual numerical methods to the problem, since they become unsta-ble. So-called regularization methods make it possible to recover informationabout the solution as stably as possible.
Uniqueness can be ensured by reformulating the notion of the solution.
Definition 2.2. Let T : X → Y be a bounded linear operator.An element x ∈ X is a least-squares solution of Tx = y if
‖Tx− y‖ = inf‖Tz − y‖ |z ∈ X.
If in addition
‖x‖ = inf‖z‖ |z is least-squares solution of Tx = y
then x is called best-approximate or generalized solution.
Thus, the best-approximate solution is defined as the least-squares solu-tion of minimal norm.
We can now define the Moore-Penrose inverse which is, roughly speaking,the operator that maps y onto the best-approximate solution of (2.1).
16
Definition 2.3. Let
T := T |N (T )⊥ : N (T )⊥ → R(T ).
The Moore-Penrose generalized inverse T † of T is defined as the unique linearextension of T−1 s.t.
D(T †) := R(T ) ⊕R(T )⊥ (2.2)
andN (T †) = R(T )⊥. (2.3)
Moreover, for y ∈ D(T †), we define x† := T †y.
Proposition 2.1. Let P and Q be the orthogonal projectors onto N (T ) andR(T ) respectively. Then, the following Moore-Penrose equations hold:
TT †T = T,
T †TT † = T †,
T †T = I − P,
Q|D(T †) = TT †.
Proof. See [7], Prop. 2.3.
Proposition 2.2. The generalized inverse T † is continuous if and only ifR(T ) is closed.
Proof. See [7], Prop. 2.4.
The next proposition gives the connection between the generalized inverseand least-squares solutions:
Proposition 2.3. Let y ∈ D(T †). Then, x† is the unique best-approximatesolution of Tx = y. Furthermore, x† + N (T ) is the set of all least-squaressolutions.
Proof. See [7], Thm. 2.5.
Let T ∗ denote the adjoint of T which is defined as the operator that mapsfrom Y to X such that for all x ∈ X and y ∈ Y
〈Tx, y〉 = 〈x, T ∗y〉.
Then, the least-squares solutions can be characterized by the normal equa-tion:
17
Proposition 2.4. Let y ∈ D(T †). Then x ∈ X is a least-squares solution of(2.1) if and only if it is a solution of the normal equation
T ∗Tx = T ∗y, (2.4)
where T ∗ is the adjoint of T .
Proof. See [7], Thm. 2.6.
2.1 Compact Operators
Let us now consider compact operators which are an important class of op-erators that lead to ill-posed problems. They are of interest because undersuitable assumptions integral operators are compact and many problems canbe formulated as integral equations. In the following, K always denotes acompact operator.
Definition 2.4. An operator K ∈ L(X ,Y) is compact if for all boundedsubsets B of X it holds that R(B) is a compact set.
A self-adjoint compact linear operator can be represented by its eigensys-tem which will help introducing regularization methods.
In the following, 〈·, ·〉 should denote the inner products of the respectiveHilbert spaces X and Y . By taking all non-zero eigenvalues λn of K anda corresponding complete system of eigenvectors vn we obtain the followingrepresentation for all x ∈ X :
Kx =∞∑
n=1
λn〈x, vn〉vn. (2.5)
If K is not self-adjoint, we can find a decomposition by its singular system(σn; vn, un) which is defined as follows:
The operator K∗K is self-adjoint since
〈K∗Kx, y〉 = 〈Kx,Ky〉 = 〈x,K∗Ky〉.
Therefore, we can do a decomposition w.r.t. the complete eigensystem(λn, vn) of K∗K:
K∗Kx =∞∑
n=1
λn〈x, vn〉vn. (2.6)
Due to〈K∗Kx, x〉 = 〈Kx,Kx〉 = ‖Kx‖2 ≥ 0
18
we have that K∗K is positive semi-definite. Thus, all the eigenvalues λn arenon-negative and we can define σn and un such that
σn := +√
λn, (2.7)
Kvn = σnun. (2.8)
Similar to (2.8) we get
σ2nvn = K∗σnun, i.e.
σnvn = K∗un,
since a possible zero eigenvalue λn is not used for the decomposition.By applying the operator K to the last equation we get
Kσnvn = KK∗un
or equivalentlyσ2nun = KK∗un
which means that un is an eigenvector of KK∗ to the eigenvalue σ2n.
Moreover, it is easy to show that (un)n∈N is an orthonormal system:
〈un, um〉 =1
σnσm〈Kvn, Kvm〉
=1
σnσm〈K∗Kvn, vm〉
=σnσm
〈vn, vm〉 = δnm.
For the proof of the next proposition, we need two fundamental theorems offunctional calculus, which can be found e.g. in [9]:
Theorem 2.1. Let K : X → Y be a compact operator and K∗ its adjoint.Then
N (K) = R(K∗)⊥,
N (K∗) = R(K)⊥.
Theorem 2.2. Let H be a Hilbert space and let S be a complete subspace ofH. Then, H is given as the direct sum
H = S ⊕ S⊥.
We can now state the following
19
Proposition 2.5. Let K : X → Y be a compact operator. The followingproperties hold:
R(K∗K) = R(K∗), (2.9)
R(KK∗) = R(K). (2.10)
Proof. It is sufficient to show the first equality, since the second one is itsimmediate consequence. Due to Theorems 2.1 and 2.2 we only need to show
N (K) = N (K∗K). (2.11)
For x ∈ X it is obvious that Kx = 0 implicates K∗Kx = 0, i.e.
N (K) ⊆ N (K∗K).
Now let x ∈ N (K∗K). Because of K∗Kx = 0, we get that Kx ∈ N (K∗) =R(K)⊥. But since Kx ∈ R(K), it follows that
Kx = 0, i.e. x ∈ N (K).
Hence,N (K∗K) ⊆ N (K)
and therefore, (2.11) holds.
From this proposition we conclude that (un)n∈N and (vn)n∈N span R(K)and R(K∗) respectively. Therefore, for x ∈ X and y ∈ Y we get the followingsingular value decomposition:
Kx =
∞∑
n=1
σn〈x, vn〉un, (2.12)
K∗y =
∞∑
n=1
σn〈y, un〉vn. (2.13)
If R(K) is finite-dimensional, then K has only finitely many singular values.Otherwise there is exactly one accumulation point for the singular valueswhich is 0:
limn→∞
σn = 0.
The range R(K) is closed if and only if it is finite-dimensional. This yields,together with Proposition 2.2,
Proposition 2.6. Let K : X → Y be a compact operator. Then, the gener-alized inverse K† is continuous if and only if dim R(K) <∞.
20
Therefore, the generalized inverse of a compact operator with infinite di-mensional range cannot be continuous. This means that the best-approximatesolution does not depend continuously on the right-hand side which makesthe equation ill-posed.The next statement is of main importance:
Proposition 2.7. Let (σn; vn, un) be a singular system for the compact linearoperator K and y ∈ Y. Then
y ∈ D(K†) ⇐⇒∞∑
n=1
|〈y, un〉|2σ2n
<∞ (2.14)
and for y ∈ D(K†)
K†y =
∞∑
n=1
〈y, un〉σn
vn. (2.15)
Proof. See [7], Thm. 2.8.
Equivalence (2.14) is called Picard criterion and gives a necessary andsufficient condition for the existence of a best-approximate solution. It statesthat the coefficients (〈y, un〉)∞n=1 have to decay fast enough w.r.t. the singularvalues σn.If such a solution exists, then equation (2.15) yields a formula for computingit. Note that error components corresponding to small singular values canbe drastically amplified. In case that dim R(K) <∞, there are only finitelymany singular values and therefore the amplification is bounded. But still,it can be unacceptably large.
In order to introduce regularization operators we need the notion of afunction of a self-adjoint operator. Recall that if (σn; vn, un) is a singularsystem for the compact operator K : X → Y we get for all x ∈ X
K∗Kx =
∞∑
n=1
σ2n〈x, vn〉vn, (2.16)
since (σ2n; vn) is an eigensystem of K∗K.
For λ ∈ R, x ∈ X and P as the orthogonal projector onto N (K∗K) we define
Eλx :=∞∑
n=1σ2
n<λ
〈x, vn〉vn (+Px). (2.17)
The component Px is meant to appear only for λ > 0.The operator Eλ is an orthogonal projector onto
Xλ := span vn|n ∈ N, σ2n < λ (+N (K∗K) , if λ > 0).
21
Obviously, Eλ = 0 for λ ≤ 0. For the case that λ > σ21 , we have Eλ = I since
Xλ = R(K∗K) + N (K∗K) = X .We can show a monotonicity property of the spectral family Eλ. For allλ ≤ µ the following holds:
〈Eλx, x〉 =∞∑
n=1σ2
n<λ
|〈x, vn〉|2 (+ ‖Px‖2)
≤∞∑
n=1σ2
n<µ
|〈x, vn〉|2 (+ ‖Px‖2) = 〈Eµx, x〉.
Additionally, Eλ is piecewise constant with jumps at λ = σ2n (and at λ = 0
if and only if N (K∗K) 6= 0) of magnitude∞∑
n=1σ2
n=λ
〈·, vn〉vn.
Recall that the integral w.r.t. a piecewise constant weight function is definedas the sum over all function values at the jumps of the integrand multiplied bythe heights of these jumps. Without going into details of measure theory, westate that the following representation is justified (note that the monotonicityproperty shown above is crucial):
K∗Kx =
∞∑
n=1
σ2n〈x, vn〉vn =
∫
λ∈R+
λdEλx. (2.18)
Moreover, one can define a piecewise continuous function f of a self-adjoint(not necessarily compact) operator as
f(T ∗T )x :=
∫ ∞
0
f(λ)dEλx
and the norm of its evaluation by
‖f(T ∗T )x‖2 :=
∫ ∞
0
f(λ) d ‖Eλx‖2 .
For compact operators this reduces to
f(K∗K)x :=
∞∑
n=1
f(σ2n)〈x, vn〉vn
and
‖f(K∗K)x‖2 :=
∞∑
n=1
f 2(σ2n)|〈x, vn〉|2.
22
2.2 Regularization Operators
Let us go back to our original problem. For a linear bounded operator T :X → Y and a given right-hand side y ∈ Y , we want to find a solution x ∈ Xsuch that
Tx = y. (2.19)
We have introduced the notion of a generalized inverse T †. If this operatorexists, the best-approximate solution is given as
x† = T †y. (2.20)
In most applications, the right-hand side y is not given exactly. Then, weare only given an approximation yδ and a bound δ on the noise level suchthat
∥
∥y − yδ∥
∥ ≤ δ. (2.21)
is guaranteed.In the ill-posed case T † is not continuous and thus, T †yδ is not necessarily
a good approximation of T †y even if it exists. Therefore, we introduce thenotion of regularization which roughly speaking means to approximate anill-posed problem by a family of well-posed problems. The aim is to find anapproximation of x† that on one hand depends continuously on yδ in orderto compute it in a stable way. On the other hand, it should tend to x† if thenoise level δ approaches zero.
We do not want to determine this approximation only for a specific right-hand side, but rather approximate the unbounded operator T † by a familyof continuous parameter-dependent operators Rα in the way that for anappropriate choice of α = α(δ, yδ)
xδα := Rαyδ
tends to x† for δ → 0.
Definition 2.5. Let T : X → Y be a bounded linear operator between Hilbertspaces and α0 ∈ (0,∞]. For every α ∈ (0, α0), let
Rα : Y → X
be a continuous operator. The family Rα is called a regularization for T †
if for all y ∈ D(T †) there exists a parameter choice rule α = α(δ, yδ) suchthat the following holds:
limδ→0
sup∥
∥Rα(δ,yδ)yδ − T †y
∥
∥ |yδ ∈ Y ,∥
∥y − yδ∥
∥ ≤ δ = 0. (2.22)
23
The parameter choice rule
α : R+ × Y → (0, α0)
has to fulfill
limδ→0
supα(δ, yδ)|yδ ∈ Y ,∥
∥y − yδ∥
∥ ≤ δ = 0. (2.23)
Therefore, a regularization method has two components: a regularizationoperator and a parameter choice rule. If α depends only on δ, we call itan a-priori, otherwise an a-posteriori parameter choice rule. However, dueto a theorem by Bakushinskii, α cannot depend on yδ only. The theoremstates that in this case, convergence of the regularization method impliesthe boundedness of T †, i.e. the well-posedness of the problem. Thus, forregularization of an ill-posed problem α cannot be chosen independently ofδ.
The questions that arise are how to construct a family of regularizationoperators and how to choose parameter choice rules that yield convergence.The following proposition gives an answer to the first question for the caseof linear operator equations.
Proposition 2.8. Let Rα be a continuous operator for all α > 0. Then, thefamily Rα is a regularization of T † if
Rαα→0−−→ T † pointwise on D(T †).
In this case, for all y ∈ D(T †) there exists an a-priori parameter choice ruleα(δ) such that (Rα, α) is a convergent regularization method for Tx = y.
Proof. See [7], Prop. 3.4.
Thus, we need to construct the regularization operators Rα such thatthey converge pointwise towards T †. We will now discuss possible ways ofconstructing the regularization operators in case T is linear.
One can extend the notion of the spectral family to self-adjoint boundedbut not necessarily compact operators. Now, let Eλ be the spectral familyof T ∗T . If T ∗T is continuously invertible, then (T ∗T )−1 =
∫
1λdEλ. By
Proposition 2.4 we get for the best-approximate solution
x† =
∫
λ∈R+
1
λdEλT
∗y. (2.24)
If R(T ) is non-closed, i.e. in the case of ill-posedness, the eigenvalues λaccumulate in 0, which means that the above integral has a pole in 0. The
24
crucial idea of regularization is to replace 1/λ by a family of functions gα(λ)that have to fulfill some continuity conditions. We can now replace (2.24) by
xα :=
∫
λ∈R+
gα(λ)dEλT∗y
and define the family of regularization operators according to
Rα :=
∫
λ∈R+
gα(λ)dEλT∗. (2.25)
The following proposition states under which assumptions on gα the con-vergence can be guaranteed:
Proposition 2.9. Let for all α > 0, gα : [0, ‖T‖2] → R be piecewise contin-uous and constructed in the way that for all λ ∈ (0, ‖T‖2]
λgα(λ) ≤ C
and
limα→0
gα(λ) =1
λ.
Then, we have that for all y ∈ D(T †)
limα→0
xα = x†
holds.
Proof. See [7], Thm. 4.1.
Since the operator Rα is continuous,∥
∥y − yδ∥
∥ ≤ δ implies the bounded-ness of the error between xα and
xδα :=
∫
λ∈R+
gα(λ)dEλT∗yδ.
We definerα(λ) := 1 − λgα(λ)
and state the following convergence result for a-priori parameter choice rules:
Proposition 2.10. Let gα fulfill the assumptions of Proposition 2.9 andassume that µ > 0. Furthermore, let for all α ∈ (0, α0) and λ ∈ [0, ‖T‖2]and some cµ > 0
λµ|rα(λ)| ≤ cµαµ (2.26)
25
hold. If
Gα := sup|gα(λ)||λ ∈ [0, ‖T‖2] = O(α−1) as α→ 0
and the so-called source condition
x† ∈ R((T ∗T )µ)
is satisfied, then the parameter choice rule according to
α ∼ δ2
2µ+1
yields∥
∥xδα − x†∥
∥ = O(δ2µ
2µ+1 ).
Proof. See [7], Cor. 4.4.
Source conditions usually imply smoothness and boundary conditions onthe exact solution. An example for how to choose gα is
gα(λ) :=
1/λ λ ≥ α,
0 λ < α.
This method is called truncated singular value expansion. The assumptionsof the previous propositions hold with C = 1, cµ = 1, arbitrary µ > 0 andGα = 1/α. In general, determining µ > 0 such that the source condition(2.10) holds, is not possible. Therefore, we briefly introduce a-posteriori pa-rameter choice rules. The most common a-posteriori choice rule is Morozov’sdiscrepancy principle which can be formulated as follows:For gα fulfilling the same assumptions as in Proposition 2.9 and a constantτ chosen according to
τ > sup|rα(λ)||α > 0, λ ∈ [0, ‖T‖2],
the regularization parameter defined by the discrepancy principle is
α(δ, yδ) := supα > 0|∥
∥Txδα − yδ∥
∥ ≤ τδ. (2.27)
Remark 2.1. The underlying idea of the discrepancy principle is the factthat since the right-hand side of (2.19) is only known up to a noise level δ, itdoes not make sense to search for an approximate solution x with a residual∥
∥T x− yδ∥
∥ < δ. We should only ask for a solution such that the residual isof order δ. In addition, a smaller regularization parameter implicates lessstability, which is why we take the largest possible value for α. This is whatis done by using the discrepancy principle.
26
Proposition 2.11. The regularization method (Rα, α) where α is defined by(2.27) is convergent for all y ∈ R(T ). Moreover, we have
∥
∥
∥xδα(δ,yδ) − x†
∥
∥
∥= O(δ
2µ2µ+1 ) (2.28)
for all µ ∈ (0, µ0 − 1/2]. Here, µ0 denotes the largest number µ for which(2.26) holds.
Proof. See [7], Thm. 4.17.
Remark 2.2. It is sufficient, that the parameter choice rule α(δ, yδ) satisfies∥
∥Txδα − yδ∥
∥ ≤ τδ ≤∥
∥Txδβ − yδ∥
∥
for some β with α ≤ β ≤ 2β. This is crucial for the numerical realization ofthe discrepancy principle.
Finally, we introduce three regularization methods. The most commonlyused is Tikhonov regularization for which
gα(λ) :=1
λ+ α.
Due to the definition of xδα and since λ+α are the eigenvalues of T ∗T +αI,we have
xδα =
∫
λ∈R+
gα(λ)dEλT∗yδ = (T ∗T + αI)−1T ∗yδ, (2.29)
i.e.(T ∗T + αI)xδα = T ∗yδ,
which can be regarded as a regularized form of the normal equation.By applying Tikhonov regularization to a compact operator K with singularsystem (σn; vn, un), we get
xδα =∞∑
n=1
σnσ2n + α
〈yδ, un〉vn.
Compared to the original singular value expansion, the factor 1σn
is now re-placed by σn
σ2n+α
which is bounded for n→ ∞.
Proposition 2.12. Let xδα be defined as in (2.29). Then it is the uniqueminimizer of the Tikhonov functional
x 7→∥
∥Tx− yδ∥
∥
2+ α ‖x‖2 .
27
Proof. See [7], Thm. 5.1.
This proposition clearly shows what regularization does. One tries to finda solution that on the one hand minimizes the residual as far as possible andon the other hand enforces stability by introducing the penalty term ‖x‖.The factor α ensures that the second term tends to 0 as the noise vanishes.
Corollary 1. In case that α(δ, yδ) is chosen according to the discrepancyprinciple Tikhonov regularization converges and yields (2.28).
Proof. This is an immediate consequence of Proposition 2.11.
Another wide-spread regularization method is the so-called Landwebermethod. It uses only discrete values for α and is therefore an iterativemethod. Here, the family of functions approximating 1/λ is defined by
gk(λ) :=1 − (1 − λ)k
λ, k ∈ N.
Finally, we mention a version of the conjugate gradient method. The CGalgorithm is a very efficient solver for self-adjoint positive (semi)-definitewell-posed linear equations.
In the case of an ill-posed equation Tx = yδ, we apply the CG methodto the corresponding normal equation
T ∗Tx = T ∗yδ
and call this the CGNE method (conjugate gradients for the normal equa-tion). Unlike Landweber iteration, the CGNE method is not based on a fixedsequence of polynomials gk and rk since these polynomials now dependon the given right-hand side. This ensures higher flexibility, however, thedrawback is that xδk depends non-linearly on the data yδ.
Proposition 2.13. In case that k(δ, yδ) is chosen according to the discrep-ancy principle both CGNE and Landweber iteration converge and yield (2.28).
Proof. See [7], Thm. 6.5. and Thm. 7.12.
Remark 2.3. As for the CGNE method, in the non-attainable casey ∈ D(T †) \ R(T ), yδ needs to be replaced by Qyδ in (2.27).Since
After this brief introduction to ill-posed problems we go back to our originalproblem (1.9), which is ill-conditioned as discussed in Section 1.4. Thereforeit is necessary to apply regularization methods in order to get better results.
We try to reconstruct the phase screen φatm shown in Figure 2.1 that hasa resolution of 28 × 28 by applying the CGNE method. For the time being,we are only interested in reconstructing φatm from given slope measurementss, i.e. we do not consider a telescope with a given number of subapertures.
We compute the corresponding Poke matrix P as if the number of sub-apertures was equal to the resolution of φatm and the pupil mask was ne-glected. What needs to be done is to evaluate the operator that maps φatmto the slope measurements s = [sx, sy]. Since we assume Fried geometry, thecomponents of s are given by
(sx)i =
∫
Ωi
∂φatm∂x
d(x, y), (2.33)
(sy)i =
∫
Ωi
∂φatm∂y
d(x, y). (2.34)
By applying the theorem of Fubini and the midpoint rule for approximating
29
50 100 150 200 250
50
100
150
200
250
Figure 2.1: Original phase screen.
the integrals, we get
(sx)i =
∫ yi2+1
yi2
[φatm(xi1+1, y)− φatm(xi1 , y)]dy
≈ ∆yi22
· [φatm(xi1+1, yi2+1) − φatm(xi1 , yi2+1)
+ φatm(xi1+1, yi2) − φatm(xi1 , yi2)],
(sy)i =
∫ xi1+1
xi1
[φatm(x, yi2+1) − φatm(x, yi2)]dx
≈ ∆xi12
· [φatm(xi1+1, yi2+1) − φatm(xi1+1, yi2)
+ φatm(xi1 , yi2+1) − φatm(xi1 , yi2)],
with ∆xi1 := xi1+1 − xi1 and ∆yi2 := yi2+1 − yi2. We then generate sδ byadding a random noise vector to s with normally distributed components.The noisy data should satisfy
∥
∥s− sδ∥
∥
rms≤ δ
for a given noise level δ. Here, the norm of a vector s of length m is definedas
‖s‖rms :=(
∑
s2i
m− 1
)1/2
.
30
noise level δ = 0%
50 100 150 200 250
50
100
150
200
250
noise level δ = 2.5%
50 100 150 200 250
50
100
150
200
250
noise level δ = 5%
50 100 150 200 250
50
100
150
200
250
noise level δ = 10%
50 100 150 200 250
50
100
150
200
250
Figure 2.2: Reconstructed phase screens for different noise levels, τ = 1.2.
Then, we apply the CGNE algorithm with the discrepancy principle to
Pa = sδ
in order to determine the point-wise values aj . The reconstructed wavefrontis then given by
φδrec =∑
j
ajhj.
Figure 2.2 shows the reconstructed phase screens for different choices of δ.The crucial point is that the error
If it is known that the desired solution of Tx = y is likely to be sparse insome basis ϕγ : γ ∈ Γ of X , one could consider regularization methods thatyield solutions that are sparse w.r.t. ϕγ : γ ∈ Γ. Therefore, we discuss anadditional regularization method for linear inverse problems that promotessparsity. The underlying idea is to replace the quadratic penalty term in(2.12) by a weighted ℓp-norm of the coefficients of x w.r.t. an orthonormalbasis ϕγ : γ ∈ Γ of X . Thus, we aim at minimizing
Φw,p(x) = ‖Tx− y‖2 +∑
γ∈Γ
wγ|〈x, ϕγ〉|p, (3.1)
where p ∈ [1, 2] and w = (wγ)γ∈Γ is a sequence of strictly positive weights.For the choice w ≡ 1, the penalty term is the ordinary ℓp-norm of the coef-ficients. Another possible choice leads to a penalty term that is equivalentto the Besov norm (see Section 3.3.1). Keeping the weights fixed and de-creasing p from 2 to 1, yields an increase in the penalization of coefficientsthat are smaller than 1 and a decrease in the penalization of those that arelarger than 1. Thus, the more we decrease p, the more we are likely to get ageneralized solution that has a sparse expansion w.r.t. ϕγ : γ ∈ Γ.
3.1 An Iterative Soft-Thresholding Algorithm
The following algorithm was introduced in [6]. Minimizing the functional in(3.1) can be rewritten in a variational formulation. Since ϕγ : γ ∈ Γ is anONB, we get
Φw,p(x) = 〈x, T ∗Tx〉 − 2〈x, T ∗y〉 + 〈y, y〉+∑
γ∈Γ
wγ|xγ|p
=∑
γ∈Γ
xγ〈T ∗Tx, ϕγ〉 − 2∑
γ∈Γ
xγ〈T ∗y, ϕγ〉 + 〈y, y〉+∑
γ∈Γ
wγ|xγ|p,
where xγ is a shortcut for 〈x, ϕγ〉. Differentiating w.r.t. to x, yields
2·∑
γ∈Γ
〈T ∗Tx, ϕγ〉ϕγ−2·∑
γ∈Γ
〈T ∗y, ϕγ〉ϕγ+∑
γ∈Γ
wγp|〈x, ϕγ〉|p−1sign(〈x, ϕγ〉)ϕγ = 0,
which implies that each coefficient is zero, i.e. for all γ ∈ Γ
〈T ∗Tx, ϕγ〉 − 〈T ∗y, ϕγ〉 +wγp
2|〈x, ϕγ〉|p−1sign(〈x, ϕγ〉) = 0. (3.2)
33
Here, we implicitly set the derivative of the absolute value to be zero inthe origin. Both the coupling of the equations due to T ∗Tx and the non-linearity of the equations make the above system hard to solve. Therefore,one introduces a surrogate functional that has nicer properties and mini-mizes it instead of Φw,p(x). As we will state later, the surrogate functionalintroduced below approximates Φw,p.
The new surrogate functional is constructed by adding an additional func-tional to Φw,p(x):
ΦSURw,p (x; a) :=Φw,p(x) − ‖Tx− Ta‖2 + C ‖x− a‖2 ,
= ‖Tx− y‖2 +∑
γ∈Γ
wγ |xγ|p − ‖Tx− Ta‖2 + C ‖x− a‖2 ,
=C ‖x‖2 − 2〈x, T ∗y − T ∗Ta+ Ca〉 +∑
γ∈Γ
wγ |xγ|p+
+ ‖y‖2 − ‖Ta‖2 + C ‖a‖2 ,
=∑
γ∈Γ
(
Cx2γ − 2xγ(Ca+ T ∗y − T ∗Ta)γ + wγ|xγ|p
)
+
+ ‖y‖2 − ‖Ta‖2 + C ‖a‖2 .
(3.3)
where C is a constant fulfilling ‖T ∗T‖ < C. Note that instead of introducingC we could also rescale the equation such that ‖T‖ < 1.Since Φw,p(x) and Ψ(x; a) := C ‖x− a‖2−‖Tx− Ta‖2 are both strictly con-vex in x (for any 1 ≤ p ≤ 2 and any a), ΦSUR
w,p (x; a) is also strictly convex inx and therefore has a unique minimizer for any a.The main advantage of this surrogate functional is that the variational equa-tions of xγ are no longer coupled. We can now define an iterative algorithmthat will lead us to a minimizer of the original functional Φw,p(x):
x0arbitrary; ∀n ∈ N xn = arg minx∈X
(ΦSURw,p (x; xn−1)).
Thus, we first determine the minimizer x1 of the surrogate functional witha = x0 and then, in each iteration minimize the functional for a = xn−1.
Let us now consider the case p = 1, that is most likely to yield sparsesolutions.Then, the summand in (3.3) is differentiable in xγ only for xγ 6= 0. In thiscase, we end up with the following variational equation
2Cxγ − 2(
Ca+ T ∗(y − Ta))
γ+ wγsign(xγ) = 0.
34
Thus, for xγ > 0 we get
xγ = aγ +1
C
(
T ∗(y − Ta))
γ− wγ
2C,
which is only valid for
aγ +1
C
(
T ∗(y − Ta))
γ>wγ2C
. (3.4)
For xγ < 0, we similarly get
xγ = aγ +1
C
(
T ∗(y − Ta))
γ+wγ2C
,
which is true only if
aγ +1
C
(
T ∗(y − Ta))
γ< −wγ
2C. (3.5)
If neither (3.4) nor (3.5) holds, we set xγ = 0.Let Sw,1 : R −→ R be a function defined as
Sw,1(t) :=
t− w/2 if t ≥ w/2,
0 if |t| < w/2,
t+ w/2 if t ≤ −w/2.
Then,
xγ = Swγ/C,1
(
aγ +1
C(T ∗(y − Ta))γ
)
.
Therefore for p = 1, the iterative method reads as follows:
∀γ ∈ Γ xnγ = Swγ/C,1
(
xn−1γ +
1
C(T ∗(y − Txn−1))γ
)
.
If p > 1, the summand in (3.3) is differentiable in xγ and minimizationreduces to solving the variational equation
2Cxγ − 2(
Ca+ T ∗(y − Ta))
γ+ pwγsign(xγ)|xγ |p−1 = 0. (3.6)
Since for any w ≥ 0 and p > 1, the real function Fw,p(x) = x+wp2
sign(x)|x|p−1
is bijective on R, we can define
Sw,p := (Fw,p)−1
35
and can again find the minimizer of (3.3) via
xγ = Swγ ,p
(
aγ +1
C(T ∗(y − Ta))γ
)
.
Unlike for p = 1, there is no explicit formula for Sw,p. Thus, when imple-menting this method for p > 1, we need an algorithm to solve the non-linearequation (3.6), which is discussed in Section 3.3.2.
We now want to turn to convergence results of this method. The follow-ing proposition states the existence of a unique minimizer of the surrogatefunctional:
Proposition 3.1. Let T be an operator mapping from a Hilbert space Xto another Hilbert space Y, assume ‖T ∗T‖ < 1 and y ∈ Y. Additionally,suppose that ϕγγ∈Γ is an orthonormal basis of X and that w = wγγ∈Γ isa sequence of strictly positive elements. Then, for arbitrarily chosen a ∈ X ,
ΦSUR
w,p (x; a) = ‖Tx− y‖2 +∑
γ∈Γ
wγ|xγ|p − ‖Tx− Ta‖2 + C ‖x− a‖2
has a unique minimizer in X that is given by
x =∑
γ∈Γ
Swγ ,p
(
aγ + (T ∗(y − Ta))γ
)
ϕγ.
Proof. See [6], Prop. 2.1.
The next proposition guarantees that the iterates of the algorithm usingsurrogate functionals converge to a minimizer of the original functional Φw,p:
Proposition 3.2. Assume that the conditions of the previous statement hold.If in addition, the sequence w = wγ is uniformly bounded from below bya constant c > 0, i.e. if for all γ ∈ Γ wγ ≥ c, then, for any x0 ∈ X theiterates
xn =∑
γ∈Γ
Swγ ,p
(
xn−1γ + (T ∗(y − Txn−1))γ
)
ϕγ
strongly converge to a minimizer of
Φw,p(x) = ‖Tx− y‖2 +∑
γ∈Γ
wγ|〈x, ϕγ〉|p. (3.7)
Proof. See [6], Thm. 3.1.
36
So far, it is not clear, whether minimizing (3.7) is related to solvingTx = y. The question is, how the penalized functional (3.7) can lead to aregularization method for the original problem.
First of all, we need to introduce an additional parameter α in order to beable to vary the weight of the penalty term. Thus, we consider the functional
Φα,w,p(x) = ‖Tx− y‖2 + α∑
γ∈Γ
wγ|〈x, ϕγ〉|p.
Due to Definition 2.5 the family Φα,w,p is a regularization for T † if for ally ∈ D(T †) there exists a parameter choice rule α = α(δ, yδ) such that
limδ→0
sup∥
∥Φα(δ,yδ),w,pyδ − T †y
∥
∥ |yδ ∈ Y ,∥
∥y − yδ∥
∥ ≤ δ = 0.
The parameter choice rule has to fulfill
limδ→0
supα(δ, yδ)|yδ ∈ Y ,∥
∥y − yδ∥
∥ ≤ δ = 0.
The following proposition states, that with an additional requirement forα(δ, yδ) the functionals Φα,w,p yield a regularization method for our linearsystem.
Proposition 3.3. Let T : X −→ Y be a bounded operator with ‖T‖ < 1and assume that p ∈ [1, 2] and that w = wγγ∈Γ is uniformly bounded frombelow by c > 0. For any y ∈ D(T †) and any α > 0, define x∗α,w,p;y to be theminimizer of Φα,w,p;y(x). If α = α(δ) satisfies
limδ→0
α(δ) = 0 (3.8)
and
limδ→0
δ2
α(δ)= 0 (3.9)
then we havelimδ→0
sup∥
∥x∗α(δ),w,p;y − T †y∥
∥ = 0.
Proof. See [6], Thm. 4.1.
Remark 3.1. As proven in [1], if α is chosen according to the discrepancyprinciple, it satisfies (3.8) and (3.9) and can be thus used here instead of ana-priori choice rule.
37
3.2 Wavelets
To apply the algorithm discussed in this chapter, the first issue that has to beclarified is the choice of the orthonormal basis ϕγ of X . In Section 1.4 weused the basis of bilinear influence functions. This basis is not orthogonal andthus cannot be considered for the algorithm based on surrogate functionals.
As mentioned in [10] the distorted incoming wavefronts tend to have afractal structure. Hence, wavelets might be a good choice for a basis in whichto expand the phase screens. Before proceeding any further, we first want togive a brief introduction to wavelets.
For a detailed discussion on wavelets we refer to [5].
Definition 3.1. Let ψ ∈ L2 be a function mapping from R to R. If ψ fulfillsthe admissibilty condition
Cψ := 2π
∫
R
|ξ|−1|ψ(ξ)|2dξ <∞, (3.10)
where ψ denotes the Fourier transform of ψ, it is called a wavelet.
From a wavelet ψ we can generate a family of wavelets according to
ψa,b(x) := |a|− 1
2ψ(x− b
a
)
, a, b ∈ R, a 6= 0, (3.11)
and refer to the function ψ as the mother wavelet.The factor |a|− 1
2 ensures that the L2-norm is independent of a and b, i.e.∥
∥ψa,b∥
∥ = ‖ψ‖ . (3.12)
The continuous wavelet transform of a function f ∈ L2(R) is defined via
(Twavf)(a, b) := |a|− 1
2
∫
R
f(x)ψ(x− b
a
)
dx = 〈f, ψa,b〉.
Note that due to (3.12) and if ψ is compactly supported, functions ψa,b witha high frequency a show a small support, whereas ψa,b with a low frequencyhave a large support. This basic property of wavelets shows its major advan-tage in signal processing, compared to the Fourier transform, since it allowsa good localization in both space and time.
We now turn to the question of reconstructing a function from its wavelettransform. We assume that ‖ψ‖ = 1. Since ψ fulfills (3.10), we can recoverany f ∈ L2(R) from its wavelet transform according to
f = C−1ψ
∫
R
∫
R
a−2(Twavf)(a, b)ψa,bdadb.
38
If ψ ∈ L1(R), then condition (3.10) can only hold if∫
R
ψ(x)dx = 0.
For implementation issues, we need to discuss how to discretize wavelet trans-forms. For convenience, we only consider the case in which f can be recon-structed by using only positive values of a. W.l.o.g, we fix a0 > 1 and b0 > 0and restrict a and b to the discrete values
a = a−j0 and b = kb0a−j0 , j, k ∈ Z,
which yields
ψj,k(x) = ψa−j0,kb0a
−j0 (x) = a
j/20 ψ
(x− kb0a−j0
a−j0
)
= aj/20 ψ(aj0x− kb0).
Here, b0 is chosen such that the functions ψ(x − kb0) cover the entire realaxis. Then, for any fixed j, this property will also hold for the functions ψj,k.
As in the continuous case, the question arises whether or not we canreconstruct f out of 〈f, ψj,k〉 in a stable way. Another question, dual to thefirst one, is whether any function f can be written as a superposition ofelementary building blocks ψj,k.
If the ψj,k constitute an orthonormal basis of L2(R), we can ensure thatany f ∈ L2(R) is characterized by the coefficients 〈f, ψj,k〉. In addition, wecan represent the L2-norm of any f ∈ L2(R) according to
‖f‖2 =∑
j,k∈Z
|〈f, ψj,k〉|2.
Thus, if f ∈ L2(R), then 〈f, ψj,k〉j,k∈Z ∈ ℓ2(Z), which guarantees a stablereconstruction. Since ψj,k is a basis, the reconstruction is simply given by
f =∑
j,k∈Z
〈f, ψj,k〉ψj,k.
The question that arises is, how to construct orthonormal wavelet bases. Weintroduce multiresolution analysis in order to tackle this issue.
3.2.1 Multiresolution Analysis
A sequence of successive approximation spaces Vj ⊂ L2(R) is called multires-olution analysis if it satisfies
· · · ⊂ V−2 ⊂ V−1 ⊂ V0 ⊂ V1 ⊂ V2 ⊂ · · · , (3.13)
39
⋃
j∈Z
Vj = L2(R), (3.14)
⋂
j∈Z
Vj = 0. (3.15)
In addition we require that all the spaces are scaled versions of the space V0,i.e.
∀j ∈ Z f ∈ Vj ⇐⇒ f(2−j·) ∈ V0 (3.16)
and that V0 is translation invariant, i.e.
∀k ∈ Z f ∈ V0 =⇒ f(· − k) ∈ V0. (3.17)
An immediate consequence of (3.16) and (3.17) is that
∀k, j ∈ Z f ∈ Vj =⇒ f(· − 2−jk) ∈ Vj.
The last requirement that has to be met is the existence of φ ∈ V0 s.t.
φ0,k : k ∈ Z is an orthonormal basis of V0, (3.18)
where φj,k(x) := 2j/2φ(2jx − k) for k, j ∈ Z. Due to (3.16), it follows thatφj,k : k ∈ Z is an orthonormal basis of Vj. We call φ the scaling functionof the multiresolution analysis.
If we define Pj to be the orthogonal projector onto Vj , then (3.14) ensuresthat limj→−∞ Pjf = f for all f ∈ L2(R). As stated below, Conditions (3.13)- (3.18) guarantee the existence of an orthonormal wavelet basis ψj,k : j, k ∈Z of L2(R), where ψj,k(x) := 2j/2ψ(2jx− k), s.t.
Pj+1f = Pjf +∑
k∈Z
〈f, ψj,k〉ψj,k. (3.19)
The mother wavelet ψ can be constructed in the following way:For all j ∈ Z we define Wj to be the orthogonal complement of Vj in
Vj+1, i.e.Vj+1 = Vj ⊕Wj .
Obviously, we have thatWj ⊥Wl for j 6= l.
Thus, for l < n
Vn = Vl
n−1⊕
j=l
Wj
40
and together with (3.14) and (3.15), this yields
L2(R) =⊕
j∈Z
Wj . (3.20)
For the spaces Wj we again have the scaling property
f ∈Wj ⇐⇒ f(2−j·) ∈W0, (3.21)
which is due to (3.16). Since (3.19) is equivalent to ψj,k : k ∈ Z beingan orthonormal basis of Wj, we get with (3.20) that ψj,k : j, k ∈ Z isan orthonormal basis of L2(R). Moreover we can conclude from (3.21) thatif ψ0,k : k ∈ Z is an orthonormal basis of W0, then ψj,k : k ∈ Z isan orthonormal basis of Wj. Thus, we need to find ψ ∈ W0 such thatψ(· − k) : k ∈ Z is an orthonormal basis of W0. In order to introduce howthis can be done, we need some definitions.
First of all, since φ ∈ V0 ⊂ V1 and φ1,k : k ∈ Z is an orthonormal basisof V1, we can define
hk = 〈φ, φ1,k〉and represent φ according to
φ =∑
k∈Z
hkφ1,k. (3.22)
We know thatφ1,k(x) =
√2φ(2x− k)
and thus, can rewrite (3.22) as
φ(x) =√
2∑
k∈Z
hkφ(2x− k)
or equivalently, as
φ(ξ) =1√2
∑
k∈Z
hke−ik ξ
2 φ(ξ
2
)
with convergence of the sums in the L2 sense. Therefore, by defining
m0(ξ) :=1√2
∑
k∈Z
hke−ikξ, (3.23)
we get
φ(ξ) = m0
(ξ
2
)
φ(ξ
2
)
.
One possible way to construct ψ, is given by the following
41
Proposition 3.4. Let (Vj)j∈Z be a sequence of closed subspaces of L2(R)which satisfies (3.13) - (3.18). Then, there exists an orthonormal basis ofwavelets ψj,k : j, k ∈ Z of L2(R), s.t.
Pj+1 = Pj +∑
k∈Z
〈·, ψj,k〉ψj,k.
One possibility for constructing the mother wavelet ψ is
ψ(ξ) = eiξ2m0
(ξ
2+ π)
φ(ξ
2
)
or equivalently,
ψ =∑
k∈Z
(−1)k−1h−k−1φ1,k =√
2∑
k∈Z
(−1)k−1h−k−1φ(2 · −k),
where m0 is defined via (3.23) and φ is chosen s.t. (3.18) is satisfied.
Proof. See [5], Thm. 5.1.1.
Remark 3.2. The orthonormality of the φ(·−k) leads to the following prop-erty of m0:
|m0(ζ)|2 + |m0(ζ + π)|2 = 1 a.e. (3.24)
3.2.2 Orthonormal Bases of Compactly Supported Wavelets
If we go back to the problem of phase reconstruction, we see that if the basisfunctions we choose have compact support, we get the nice property thatthe Poke matrix will be sparse. Thus, we are interested in wavelets that arecompactly supported. In order to ensure that all ψj,k, j, k ∈ Z have compactsupport, we only need that the mother wavelet ψ is compactly supported.Furthermore, this is ensured if the scaling function φ has compact support,since then, only finitely many hk are non-zero and thus, ψ reduces to a finitelinear combination of compactly supported functions.
For compactly supported φ the 2π-periodic functionm0 becomes a trigono-metric polynomial. One can show that if m0 is a trigonometric polynomialwith m0(0) = 1 and fulfilling (3.24), then under some further assumptionswe get the following result: If we define φ and ψ according to
φ(ξ) :=1√2π
∞∏
j=1
m0(2−jξ),
ψ(ξ) := −e−iξ/2m0
(ξ
2+ π)
φ(
ξ/2)
,
42
then, ψj,k : j, k ∈ Z is an orthonormal basis of L2(R)([5], Thm. 6.3.6).Thus, we need to find m0 that satisfies (3.24). In addition, we are inter-
ested in making φ and ψ reasonably regular. One can show that imposingsome regularity constraints implies that m0 should be of the form
m0(ξ) =(1 + eiξ
2
)N
L(ξ),
where N ≥ 1 and L is a trigonometric polynomial.
Proposition 3.5. A trigonometric polynomial m0 of the form
m0(ξ) =(1 + eiξ
2
)N
L(ξ),
fulfills (3.24) if and only if L(ξ) = |L(ξ)|2 can be written as
L(ξ) = P (sin2 ξ/2),
with
P (y) = PN (y) + yNR(1
2− y)
,
where
PN(y) =
N−1∑
k=0
(
N − 1 + kk
)
yk
and R is an odd polynomial, chosen s.t. P (y) ≥ 0 for y ∈ [0, 1].
Proof. See [5], Prop. 6.1.2.
This proposition completely characterizes |m0|2. With spectral factoriza-tion one can extract the ”square root” (for further details we refer to Chapter6 of [5]). One important class of compactly supported orthonormal waveletbases is the family of Daubechies wavelets, first introduced in [4], which cor-responds to R ≡ 0. By varying N we get the different Daubechies wavelets(which we abbreviate with dbN) and the regularity increases with N . Exceptfor N = 1 which corresponds to the Haar basis (introduced in 3.3.3), there isno closed representation for φ and ψ. Nonetheless, their graph can be com-puted up to arbitrarily high precision by applying the Cascade Algorithm(see [5]). Figure 3.1 shows the graphs of φ and ψ for N = 2 and N = 4.
Remark 3.3. The phase screen shown in Figure 2.1 can be very well com-pressed with wavelets. If we decompose the image w.r.t. the Haar waveletsand set all wavelet coefficients that have an absolute value smaller than 10 tozero, we get 91.81% zero coefficients and 99.95% retained energy. Thus, the
43
0 1 2 3−0.5
0
0.5
1
1.5N=2
0 1 2 3−2
−1
0
1
2N=2
0 2 4 6 8−0.5
0
0.5
1
1.5N=4
0 2 4 6 8−1
−0.5
0
0.5
1
1.5N=4
φ
ψφ
ψ
Figure 3.1: φ and ψ for Daubechies wavelets.
image can be very well approximated with images that are sparse in the Haarwavelet. For the 1 D slice of the phase screen in Figure 3.2 we get 80.15%zero coefficients for a retained energy of 99.49% by setting the threshold to 9.We get similar results for other wavelet bases. The retained energy is definedas
‖acomp‖22
‖φ‖22
where acomp is the coefficient vector of the compressed signal and φ is theoriginal signal.
3.3 Implementation
For simplicity (see Section 3.3.3 for more details), we consider the issue ofphase reconstruction in 1 D, i.e. we aim at reconstructing a ”slice” of the 2D phase screen.
44
3.3.1 Choosing the Weights wγ
The first question we have to answer is how to choose the weights wγ. InProposition 3.3, one of the necessary conditions is that the sequence w =(wγ)γ∈Γ has to be uniformly bounded below away from zero. This is the onlycondition that has to be guaranteed for the weights. For the implementationwe take two interesting choices of the wγ.
The first one is wγ = 1 for all γ ∈ Γ. In this case we get
Φw,p(x) = ‖Tx− y‖2 + ‖x‖pp ,where ‖·‖p denotes the ℓp-norm and x = (〈x, ϕγ〉)γ∈Γ.
The second choice is based on the fact that wavelets do not only consti-tute orthonormal bases of L2(R) but also bases for a variety of other Banachspaces of functions, such as Holder spaces, Sobolev spaces and, more gener-ally, Besov spaces. Roughly speaking, the Besov spaces Bs
p,q(Rd) consist of
functions that ”have s derivatives in Lp”. The parameter q provides someadditional fine-tuning to the definition of these spaces. The norm ‖x‖Bs
p,qis
related to the modulus of continuity ω of x which is defined as a functionω : [0,∞] −→ [0,∞] such that
|x(s) − x(t)| ≤ ω(|s− t|)for all s and t in the domain of x. We refer to [11] for further details and onlywant to point out that the Besov norm is equivalent to a norm that can becomputed from the wavelet coefficients. More precisely, we assume that thescaling function φ and the mother wavelet ψ fulfill the smoothness property
of being in CL(R), with L > s and that σ := s + d(
12− 1
p
)
≥ 0. Since we
only consider d = 1, we get σ = s +(
12− 1
p
)
. We define the norm 9 · 9s;p,q
according to
9x9s;p,q =
(
∞∑
j=0
(
2jσp∑
γ∈Γ,|γ|=j
|〈x, ψγ〉p|)q/p
)1/q
, (3.25)
where |γ| denotes the scale of the wavelet ψγ .This norm is then equivalent to the Besov norm, i.e. there exist A > 0
and B > 0 s.t.A 9 x9s;p,q ≤ ‖x‖Bs
p,q≤ B 9 x 9s;p,q .
The condition σ ≥ 0 ensures that Bsp,q(R) is a subspace of L2(R). We will
restrict ourselves to choosing q to be equal to p, since then (3.25) reduces to
9x9s;p,q =
(
∞∑
j=0
(
2jσp∑
γ∈Γ,|γ|=j
|〈x, ψγ〉p|)
)1/p
.
45
Both choices of weights should yield sparse solutions for p → 1 and will becompared in Chapter 4.
3.3.2 The Shrinkage Function Swγ ,p
In order to speed up the computation, we implement a vector-valued versionSw,p = (Swγ ,p)γ∈Γ of the shrinkage function. For p = 1 we do this by usingthe MATLAB built-in function wthresh.
As mentioned in Section 3.1, for p > 1 we cannot find Swγ ,p explicitly butmoreover have to find the solution xγ of the non-linear equation
xγ + ksign(xγ)|xγ|p−1 = c, (3.26)
where k = pαwγ
2Cand c = aγ + 1/C
(
T ∗(y − Ta))
γ. At a first glance, using
Newton’s method for solving (3.26) seems to be a good idea. Unfortunately,this method fails in some cases. For instance, if p = 1.2, C = 4.0008,wγ = 23(3/2−1/p)p and c = 0.2, the iterates of the Newton method oscillatebetween x2k
γ = 0.4720 and x2k+1γ = −0.2677. However, if the factor k is small
enough, the method works. We can achieve this by either starting with asmall regularization parameter α or with a large value for C. The latter hasthe drawback of slower convergence and should thus be avoided if possible.On the other hand, in the case of data noise, α cannot be chosen arbitrarilysmall. To ensure that the implemented algorithm always works, we add themethod of bisection which will be used in case the Newton method fails. Thisalgorithm is much slower than the Newton method, but it converges for anycontinuous strictly monotonic function f for which limx→−∞ f(x) = −∞ andlimx→∞ f(x) = ∞, [14].
Algorithm 2 Bisection for solving f(x) = x+ ksign(x)|x|p−1 = c
a := min(c, 0), b := max(c, 0), m := a;while |a− b| > ǫ ∨ |f(m)| > ǫ do
m := a+ b−a2
,if f(m) > 0 then
b = m,else
a = m,end if
end while
x = m;
46
Proposition 3.6. Let a < b ∈ R be chosen such that the unique solution x∗
of f(x) = c fulfills a ≤ x∗ ≤ b and let the iterates of the method of bisectionbe denoted by xk. Then, the method converges with the rate
|xk − x∗| ≤ 2−k|b− a|.
Proof. By induction. For k = 0 we know that
|x0 − x∗| ≤ |b− a|.
Now suppose that|xk − x∗| ≤ 2−k|b− a|
which means that in the k-th step the interval we are restricting our solutionto is of length 2−k|b − a|. In the k + 1-th step we reduce this interval bytaking only half of it, i.e.
|xk+1 − x∗| ≤ 1
22−k|b− a| = 2−(k+1)|b− a|.
Again, we implement both the Newton and bisection algorithm such thatthe computation is done for vectors.
3.3.3 Building the Poke Matrix
In order to test the algorithm, we need to determine the Poke matrix accord-ing to the chosen wavelet basis. In addition, we have to compute the slopemeasurements for the given wavefront. In 2 D, the entries of the Poke matrixP = [Px,Py] are given by
Pxi,ℓ =
∫
Ωi
∂ψj,k(x, y)
∂xd(x, y),
=
∫ yi2+1
yi2
(
ψj,k(xi1+1, y) − ψj,k(xi1 , y))
dy,
Pyi,ℓ =
∫
Ωi
∂ψj,k(x, y)
∂yd(x, y),
=
∫ xi1+1
xi1
(
ψj,k(x, yi2+1) − ψj,k(x, yi2))
dx,
where ℓ corresponds to the linear indexing of (j, k). In order to computethis integral we can apply a quadrature rule. However, we will need the
47
evaluation of the wavelet functions ψj,k at some given points. This is not aneasy task, since we cannot use the MATLAB wavelet toolbox to evaluate thewavelets and would thus need to build the wavelet family manually.
Hence, for simplicity, we only want to reconstruct a 1D ”slice” φatm :[0, 1] −→ R of the phase screen (shown in Fig. 3.2). We assume thatφatm ∈ L2 and that it has zero mean. Furthermore, the resolution in whichφatm is actually given is 28. We choose the scale j to range from 0 to 7.In 1D, computing the slopes reduces to
0 50 100 150 200 250 300−60
−40
−20
0
20
40
60
80
100
120
Figure 3.2: 1D slice of original phase screen.
si =
∫
Ωi
φ′atm(x)dx = φatm(xi+1) − φatm(xi),
and the Poke matrix P is determined by
Pi,ℓ = ψj,k(xi+1) − ψj,k(xi),
where ℓ corresponds to the linear indexing of (j, k). Here, we still need theevaluation of the wavelet functions which can be implemented in MATLABby decomposing signals (i.e. computing their coefficients) that have only onenon-zero value which is equal to 1. In order to test the introduced algorithm,we use it to solve
Pa = s,
48
for the wavelet coefficient vector a of the reconstructed phase screen φrec,which we finally compute according to
φrec =∑
k
a0,kφ0,k +
7∑
j=1k
aj,kψj,k.
We start with the simplest possible wavelets: Haar wavelets. In this case,a0 = 2, b0 = 1 and the mother wavelet is defined as
ψ(x) =
1 if 0 ≤ x <1
2,
−1 if1
2≤ x < 1,
0 otherwise.
In order to justify using these wavelets, we need to ensure that they constitutean orthonormal basis of L2(R), i.e. that
1. the ψj,k are orthonormal and
2. any function in L2(R) can be approximated, up to any desired precision,by a finite linear combination of the ψj,k.
Orthonormality is easy to show. Since two Haar wavelets of the same scale jdo not overlap, it holds that 〈ψj,k, ψj,k′〉 = δk,k′. If they are of different scalesj < j′, then the support of ψj,k lies in a region, where ψj′,k′ is constant.Therefore, the scalar product 〈ψj,k, ψj′,k′〉 is proportional to the integral ofψj,k itself and thus, is zero. We skip the proof of the second statement, whichcan be found in [5].
Remark 3.4. The Haar wavelet basis can also be constructed with the mul-tiresolution analysis for
φ(x) =
− 1 0 ≤ x < 1,
0 otherwise.
Since the basis functions have compact support, the Poke matrix is sparse.Figure 3.3 shows the sparsity pattern of P. Figures 3.5 and 3.6 illustrate thebehaviour of the algorithm for p = 1 and Haar wavelets for different noiselevels. The reconstruction error approaches zero as the noise level tends tozero. Here, the implementation of the Haar basis is done independently,
49
50 100 150 200 250
50
100
150
200
250 −1.4
−1.2
−1
−0.8
−0.6
−0.4
−0.2
0
0.2
0.4
0.6
Figure 3.3: Sparsity pattern of P for the Haar basis.
db2
50 100 150 200 250
50
100
150
200
250
db4
50 100 150 200 250
50
100
150
200
250
−0.5
0
0.5
1
−1
−0.5
0
0.5
1
Figure 3.4: Sparsity pattern of P for the Daubechies wavelets.
which is very complex for other Daubechies wavelets. For the latter we use theMATLAB wavelet toolbox to evaluate the basis functions at different valuesin order to build the Poke matrix and to compute φrec from its wavelet
50
0 50 100 150 200 250 30050
100
150
200
250||s−sδ|| = 0.005
0 50 100 150 200 250 30050
100
150
200
250||s−sδ|| = 0.05
0 50 100 150 200 250 30050
100
150
200
250||s−sδ|| = 0.1
0 50 100 150 200 250 3000
50
100
150
200
250||s−sδ|| = 0.25
φrecδ
φatm
Figure 3.5: Reconstructed 1D phase screens for Haar basis, p = 1 and differ-ent noise levels.
coefficients. Figure 3.4 shows the sparsity pattern of P for db2 and db4.We compare the performance of the algorithm for different values of p anddifferent wavelet bases in Chapter 4.
3.3.4 The Regularization Parameter α
From Proposition 3.3 we know the necessary requirements on the regular-ization parameter α to ensure convergence of the regularized solution to thegeneralized solution. However, we need to specify how to determine α inthe implementation. Instead of an a-priori parameter choice rule we use thediscrepancy principle. The question is, how to compute
α(δ, yδ) = supα > 0|∥
∥Txδα − yδ∥
∥ ≤ τδ
51
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10
10
20
30
40
50
60
70
80
90
||s−sδ||
||φat
m −
φre
cδ
||
Figure 3.6: Error in φ w.r.t. noise level δ. Haar basis, p = 1.
for a given τ ∈ (1, 2). We do this by starting with α1 = q < 1 for which wedetermine xδα1
. If for the residual we get∥
∥Txδα1− yδ
∥
∥ > τδ, we decrease theregularization parameter by taking α2 = q2 < α1, compute the residual andcheck again if the chosen parameter is an element of the set
α > 0|∥
∥Txδα − yδ∥
∥ ≤ τδ. (3.27)
Thus, we introduce an outer loop to the minimization step in which we setαk = qk and determine xδαk
. If n is such that for the first time αn is an elementof (3.27), we stop the iteration. The procedure of iterative soft-thresholdingis described in Algorithm 3.
3.4 A Fast Iterative Soft-Thresholding Algorithm
The advantage of the iterative soft-thresholding algorithm introduced in Sec-tion 3.1 is its simplicity. However, its main drawback is that it convergesslowly. This is verified by the numerical results in Chapter 4. In fact, it isproven in [3] that under some assumptions on the operator T the methodconverges linearly, with a constant of the linear rate that can be arbitrarilyclose to 1. Therefore, accelerated versions of ISTA have been investigated.This is very crucial, since the performance of a single iteration step dependson the size of the system. In the case of phase reconstruction the systemgrows for higher resolutions and especially when changing from 1 D to 2 D.Thus, we want a method that, for any given accuracy, needs fewer iterationsthan ISTA.
52
Algorithm 3 ISTA
xδα0= x∗, r0 =
∥
∥Txδα0− yδ
∥
∥ , C = ‖T ∗T‖ + ǫ, k = 0;while rk > τδ do
k = k + 1,
αk = αk−1 · q,res = 1/C · (T ∗yδ − T ∗Txδαk ,0
),
xδαk ,1= Sαkw/C,p(x
δαk ,0
+ res),
i = 1,
while
‚
‚
‚xδ
αk,i−xδαk,i−1
‚
‚
‚
rms‚
‚
‚xδ
αk,i−1
‚
‚
‚
rms
> ǫ do
res = 1/C · (T ∗yδ − T ∗Txδαk ,i),
xδαk ,i+1 = Sαkw/C,p(xδαk,i
+ res),
i = i+ 1,
end while
rk =∥
∥Txδαk− yδ
∥
∥ .
end while
53
We consider a fast iterative soft-thresholding algorithm (FISTA) thatwas introduced in [2]. It preserves the computational simplicity of ISTA butimproves the convergence rate significantly. As proven in [2], by choosingC > 0 as before, i.e. fulfilling ‖T ∗T‖ < C and by defining x∗ to be theminimizer of Φw,p, we get the following linear convergence rate for the iteratesxn of ISTA:
∀n ≥ 1 Φw,p(xn) − Φw,p(x∗) ≤ C ‖x0 − x∗‖2
n.
In contrast, FISTA has the improved property of quadratic convergence:
∀n ≥ 1 Φw,p(xn) − Φw,p(x∗) ≤ 4C ‖x0 − x∗‖2
(n+ 1)2.
The main idea of FISTA is that in each step, instead of minimizing accordingto
x0 arbitrary; ∀n ∈ N xn = arg-min(ΦSURw,p (x; xn−1)),
we take not only the last iterate xn−1 for the update but rather introduce yn
that depends on both xn−1 and xn−2 (see Algorithm 4). Figure 3.7 shows
Algorithm 4 FISTA
Initial guess x0, y1 = x0, t1 = 1.For n ≥ 1
xn = arg-min(ΦSURw,p (x; yn)),
tn+1 =1 +
√
1 + 4t2n2
,
yn+1 = xn +tn − 1
tn+1(xn − xn−1).
how the reconstruction error tends to zero for δ → 0.
54
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10
10
20
30
40
50
60
70
80
90
100
||s − sδ||
||φat
m −
φre
cδ
||
Figure 3.7: Error in φ w.r.t. to noise level δ for FISTA with p = 1.
55
4 Numerical Results and Conclusion
We tested ISTA and FISTA on the slice of a phase screen from Figure 3.2for different values of p and found that FISTA clearly outperforms ISTA. Inorder to achieve
‖Pa− s‖rms < 10−4,
we needed 6 to 15 times more iterations with ISTA compared to FISTA (seeTable 1). Figures 4.1 and 4.2 show the performance of ISTA and FISTA forp = 1, 1.5, 1.9, the db2 wavelets and the weights according to the ℓp-norms.The value ǫ for the stopping criterion of the inner while-loop was set to 10−10
and C to∥
∥PTP∥
∥+ 10−3.
0 1 2 3 4 5 6 7 8 9
x 106
0
0.2
0.4
0.6
0.8
1
1.2
1.4
1.6
1.8
2
number of iterations
||P a
− s
||
ISTAFISTA
Figure 4.1: Performance of ISTA and FISTA for p = 1.
In addition, we compared FISTA for p = 2 with the CGNE method.The difference cannot be measured in number of iterations since the twoalgorithms do not have the same structure. However, for the same exampleas before (but with ǫ = 10−5 and the residual measured in terms of thenormal equation) the CPU time needed for FISTA was 67.65 seconds whilethe CGNE method took only 0.0998 seconds.
We also considered the issue of sparsity. We used MATLAB to compressthe original signal such that we got 74.07% zero coefficients in the db2 basis(which yields a retained energy of 99.98%) and reconstructed the slice of thephase screen with FISTA (p = 1). The reconstructed signal had 48.15% zerocoefficients. Using other values of p yielded in getting no zero coefficientsbut a large number of small coefficients. Figure 4.3 shows the distribution of
56
Table 1: Number of iterations for ISTA and FISTA, ǫ = 10−10, δ = 10−4.
Table 2: Zero coefficients in percent after compression in MATLAB. Retainedenergy: 99.99%.
Haar db2 db3 db4 db553.96% 65.19% 65.59% 66.09% 68.15%
the wavelet coefficients of the solutions obtained by taking different valuesof p. One can clearly see how sparsity is promoted by decreasing p from2 to 1. Furthermore, we were interested in whether a certain Daubechieswavelet basis would perform better in compression. We found that for afixed percentage of retained energy, there is only slight improvement withincreasing the order N of the wavelet. However, the Haar basis performssignificantly worse than higher order Daubechies wavelets (see Table 2). Forp = 1 we also tested the difference between choosing ℓ1 and Besov weights.Since with the latter we put a higher penalty on coefficients corresponding tofiner scales, we promote sparsity even more than with ℓ1 weights. A numericalexample showed that instead of 9 · 105 iterations which FISTA takes for ℓ1
weights, the number of iterations could be reduced by choosing Besov weightsto 8.55 · 105 (s = 1) and 6.98 · 105 (s = 2) iterations.
To conclude, we think that the wavelet based approach for the problemof phase reconstruction might be very promising, due to the fractal structureof phase screens. It is this structure that makes compression with waveletbases possible. Thus, considering algorithms that promote sparsity in someorthonormal wavelet basis is a good approach. However, performance is onemain drawback of these algorithms since even FISTA is very slow compared tofast methods like CGNE. In addition, this approach for phase reconstructionwas only tested on 1 D signals throughout this work. The next importantstep would be to extend it to 2 D.
57
0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2
x 107
0
0.5
1
1.5
2
2.5
3
3.5
number of iterations
||Pa
− s
||
(a) p = 1.5
0 1 2 3 4 5 6 7 8 9 10
x 106
0
0.5
1
1.5
2
2.5
3
number of iterations
||Pa
− s
||
ISTAFISTA
(b) p = 1.9
Figure 4.2: Performance of ISTA and FISTA for p = 1.5 and p = 1.9.
58
0 2 4 6 8
x 10−3
0
1
2
3
4
5
6p=1.8
0 2 4 6 8
x 10−3
0
10
20
30
40
50
60p=1.4
0 1 2 3 4
x 10−3
0
20
40
60
80
100p=1.2
0 1 2 3 4 5
x 10−3
0
50
100
150
200p=1
Figure 4.3: Distribution of absolute values of wavelet coefficients aroundzero for different values of p. x-axis: absolute values of coefficients, y-axis:number of coefficients.
59
References
[1] Anzengruber S. W., Ramlau R., 2010, Morozov’s discrepancy principlefor Tikhonov-type functionals with nonlinear operators. Inverse Prob-lems Vol. 26 No. 2.
[2] Beck A., Teboulle M., 2009, A Fast Iterative Shrinkage-ThresholdingAlgorithm for Linear Inverse Problems. SIAM J. Imaging Sciences Vol.2 No. 1, pp. 183-202.
[3] Bredies K., Lorenz D. A., 2008, Linear convergence of iterative soft-thresholding. Journal of Fourier Analysis and Applications, 2008.
[4] Daubechies I., 1988, Orthonormal bases of compactly supportedwavelets. Comm. Pure Appl. Math. Vol. 41, pp. 909-996.
[5] Daubechies I., 1992, Ten lectures on wavelets. SIAM.
[6] Daubechies I., Defrise M., De Mol C., 2004, An iterative thresholdingalgorithm for linear inverse problems with a sparsity constraint. Comm.Pure Appl. Math. 51 1413-1541.
[7] Engl H. W., Hanke M., Neubauer A., 1996, Regularization of inverseproblems. Kluwer Academic Publishers.
[8] Hardy J. W., 1998, Adaptive Optics for Astronomical Telescopes. Ox-ford University Press.
[9] Heuser H., 1975, Funktionalanalysis. Teubner.
[10] Lane R. G., Glindemann A., Dainty J. C., 1992, Simulation of a Kol-mogorov phase screen. Wave in random media 2, 209-224.
[11] Meyer Y., 1992, Wavelets and Operators. Cambridge University Press.
[12] Paterson C., 2005, Constraints of ground-based observations: The at-mosphere. Optics in Astrophysics, NATO Science Series, pp. 1-10.,Springer.
[13] Quiros-Pacheco F., 2007, Reconstruction and Control Laws for Multi-conjugate Adaptive Optics in Astronomy. Ph.D. thesis, Imperial Col-lege.
[14] Ramlau R., 2008, Regularization properties of Tikhonov regularizationwith sparsity constraints. Electronic Transactions on Numerical Anal-ysis Vol. 20, pp. 54-74.
[16] Tyson R. K., 2000, Introduction to Adaptive Optics. SPIE Press.
[17] Vogel C.R., Yang Q., 2006, Multigrid algorithm for least-squares wave-front reconstruction. Applied Optics Vol. 45 No. 4.
61
Eidesstattliche Erklarung
Ich erklare an Eides statt, dass ich die vorliegende Masterarbeit selbststandigund ohne fremde Hilfe verfasst, andere als die angegebenen Quellen und Hilf-smittel nicht benutzt bzw. die wortlich oder sinngemaß entnommenen Stellenals solche kenntlich gemacht habe.
