Advanced Transform Methods

transcript

Advanced Transform MethodsProfessor Sir Michael Brady FRS FREng

Department of Engineering ScienceOxford University

Hilary Term 2006

Outline

• Recap on Fourier domain• Limitations of the Fourier transform• Gabor’s great idea• Analytic signal, local phase & features• Bases for spaces of functions• Wavelets• The Riesz transform, monogenic signal &

features again

Recap on Fourier domain

• Frequency for signal/image processing– “Frequency freaks” vs “feature creatures”

• Fourier’s idea– Was using sine/cosine lucky or insightful?

• Efficiency: the FFT• The importance of phase• Homomorphic filtering• Image enhancement & restoration

Frequency for signal/image analysis

+ “noise” is often high frequency– Unfortunately, so are transients such as edges, corners, …– The human visual system ignores constant signals but detects

signal changes– Filtering out high frequencies can damage edges

+ Textures often have repeated patterns that correspond to peaks in Fourier spectrum– Unfortunately, this is too simple for texture analysis in practice

+ Frequency codes for position in MRI+ Sadly, Fourier theory is often ignored in image analysis –

at cost of poorer results than necessary– Example of rotational invariance of feature detection

Frequency freaks

Sinusoidal grating

Campbell and Robson’s contrast sensitivity function CSF

Example of a CSF: threshold contrast vs spatial frequency (cycles/degree or lp/mm)

Output from the monogenic signal

a. Corners detected by a conventional algorithm as a shape rotates;

c. Edges detected by the Canny edge detector as the shape rotates

b. Corners detected after Fourier analysis of the filter response is analysed as a function of orientation, leading to a filter whose response is anisotropic

d. Edges after similar analysis

Source: DPhil thesis by Jason Merron, 1997

Fourier’s idea (1807)Fourier presented a memoir to the Institut de France in which he claimed that all continuous functions of a real variable f(t) can be represented as series of harmonically related sines and cosines. The complete mathematical story was not worked out until 1960!

Definition: Let f(t) be a continuous function of a real variable t, then the Fourier Transform of f is defined to be:

∫∞

∞−

−→ℑ dtetfuf jut)( :)(

ℑNote that is an operator that transforms a function f of one variable into another:

)()(: 11 ℜ→ℜℑ LL∫ +∞<ℜ dtL f(t) :is that functions, integrable of space (Hilbert) theis )( where 1

Inverse FT usually exists

An important result is that for most functions f and for most t the inverse transform exists too:

ff ˆby )( denote :otation ℑN

∫∞

∞−

ℜ∈ℜ∈

dueuftf

jut)(ˆ21)(

, allalmost for then ,)( If 1

Power and phase

)(ˆ)(ˆ

)(ˆ)(ˆ)(ˆ

)(ˆ)(ˆ :form lexponentiain Express

)(ˆ)(ˆ)(ˆfunction luedcomplex va a is )(ˆ that Note

ufufuf

ufjufufuf

The first of these is the square root of the power spectrum of f, the second is the phase angle of f.

Most of signal processing is concerned with the power spectrum; but (as we shall see) phase dominates in image analysis

Decomposing an image into its spatial frequency components

Rectangle function and sinc

)(sinc)(

)sin()(

21||,021||,1

πππ

⎪⎩

⎪⎨

Rectangle function plays a key role in image/signal processing. In the frequency domain it defines a perfect bandpass filter, in the space domain a perfect windowing operator.

Sinc interpolation plays an important role in MRI image analysis, and geometric alignment of data sets.

Fourier transform properties

)(ˆ)(derivativeFrequency )(ˆ)()(derivative Time

)(ˆ)( Scaling

)(ˆModulation)(ˆ)(nTranslatio

)(ˆ)(ˆ21))((nConvolutio

)(2)(ˆInverse)(ˆ)(property

ufjtufjutf

sufsstf

uufeufettf

ufuhtfh

uftfuftf

−−

Examples, and a question

sinc)(

)()(21cos

λωδλωδλ

A reminder from the FT course:

OK, so “all continuous functions of a real variable f(t) can be represented as series of harmonically related sines and cosines”

But was Fourier's observation just lucky? Or, is there a deeperreason why cosines and sines - complex exponentials - should play a key role?

Linear, time invariant filtering

βαβαβα

, constantsany for )()()( iflinear isoperator An gfgf LLL L +=+

Linearity is a simplifying assumption that is made throughout much of engineering science, even if it rarely holds exactly.

))(( ))(( then),((t) if is,That .by delayed also isoutput then the

, timeaby delayed is input to theifinvariant timecalled is operator An

tftftff

τττ

=−−=

linear, time-invariant operators: impulses and convolution

)()( denote and1011

function Dirac the)(by Denote

u −=⎩⎨⎧

Linear, time-invariant systems are characterised by their response toDirac functions. Assume for the moment that the followingintegrations are defined: ∫

∞−

= dttuftf u )()()( δ

dttuftf u )()()( impliesLinearity δLL ∫∞

∞−

Impulse response

The impulse response h of the linear, time-invariant operator L is

)()( tth δL=

)()()(

thatso ),()( that proves invariance time

duutfuh

duuthuftf

∫∞

∞−

The impulse response function of a linear time-invariant operator is a convolution

complex exponentials are eigenfunctions of convolution operators

ωfrequency at the of FT theis )()(ˆ eigenvalue The

hdueuhwh

dueuhe

jwujwt

utjwjwt

∞−

… so linearity impulse response convolution sines & cosines

So Fourier's insight was deep, though it may also have been lucky!

Convolution theorem and FFTWe have seen that linear, time-invariant operators correspond precisely with those defined by convolution. That is, L is a linear, time-invariant operator if, and only if, there is a function h(t), called the impulse-response function of L so that Lf = h*f.

The convolution theorem is the core of the application of Fourier theory to signal and image processing, since the naïve shift-and-multiply implementation of convolution is intrinsically expensive, havingcomplexity O(N2) where N is the desired number of values of u.

( ) fhfh

LfhLhLf

and )( then ,)( and )( If :Theoremn Convolutio^

ℜ∈∗ℜ∈ℜ∈

Convolution in the time/space domain = multiplication in Fourier domain

Practical importance of the convolution theorem

),(),(),(),( vuGvuFyxgyxf ⇔∗

Smoothing an image with a Gaussian kernel. Could do it in the space domain, using separability of the Gaussian, OR1. Compute the FT of the image and of the

Gaussian kernel

2. Multiply their FTs

3. Compute inverse FT of productThis only makes sense because there is a FAST FT algorithm

The FFT

∑∑−

0)(1)(1)(ˆ

Assume N = 2n for some n, so that N = 2M for some M. Substituting:

⎥⎦

⎤⎢⎣

⎡++= ∑∑

utM WWtf

0)12(1)2(1

21)(ˆ

So that:

[ ]uMWufufuf 2oddeven )(ˆ)(ˆ

21)(ˆ +=

FFT algorithmGiven a signal {f(0), f(1), …., f(63)}, it is broken down in to:

• {f(0), f(2), … , f(62)} and

• {f(1), f(3), …, f(63)

That is, the FT of a signal of length 64 can be computed from two FTs of length 32. These in turn are broken down in to their odd and even bits

{f(0), f(4), …}; {f(2), f(6), …}; {f(1), f(5), …}; {f(3), f(7),..}

So it can be computed from 4 of length 16. In general, For a signal of length N, the algorithm takes time N log N instead of N2 by the naïve method.

There is nothing magical about this process, it follows from simple properties of sines and cosines. Exactly, the same idea works for wavelets

∫ ∫

∫ ∫∞

∞−

dudvevufyxf

dxdyeyxfvuf

vufyxf

),(ˆ),(

),(),(ˆ

definecan then we ,integrable is ),(ˆ if and ,integrable and continuous is ),( If

We can define the power and phase as for 1D FT

FT pair in 2Dv

)()(),( YvsncXuXYsncvuF ππ=

FT of a circular disk

ρρπρ /)()(),(||,0||,1

1 aaJFvuFarar

==⎩⎨⎧

2D version of a sinc function

Important in ultrasound image analysis

2D FT of an X shape

Discrete FT

−∞=

enTfuf

nTtnTftf

nTfnTtftf

)2(ˆ1)(ˆ

:is intervalsat uniformly samplingby obtained signal discrete theof FT The Theorem

)()(ˆ

:find we1, is delta a of FT thefact that theand ,on theorem translati theUsing

)()()(

:sum Dirac weighteda toscorrespond )( sample discreteA ).( into sampled is )(function The 0

This shows that sampling f at intervals T is equivalent to making its Fourier transform periodic by summing all its translations.

Discrete image filteringThe properties of 2D space-invariant operators are essentially the same as in 1D. A linear operator L is space-invariant if, for any

),(),(

then),,(),(

qmpnfmnf

−−=

An image can be decomposed as a sum of discrete Diracs and then linearity and space-invariance proves that any L has an associated impulse response function h(n,m) so that application of L is equivalent to a 2D convolution with h

SeparabilityThe impulse response function h(n,m) is separable if it is a product of two one-dimensional functions:

)()(),( 21 mhnhmnh =In the case of a separable impulse response function, the two-dimensional convolution can be rewritten as a 1D convolution along the columns followed by a 1D convolution along the rows (or viceversa).

( ) ∑ ∑∞

−∞=

−∞=⎟⎟⎠

⎞⎜⎜⎝

⎛−−=∗

p qqpfqmhpnhmnhf ),()()(),( 21

Reduces compute complexity to O(2N); the discrete FT can be computed by column then by row.Example: Gaussian convolution and pipeline architectures

FT of a Gaussian

2222/2

,)(),(

1)(222

vueFvuF

yxrerf r

ρρπσ

σρπ

• FT of a Gaussian is a Gaussian

• Note the inverse scale relation

Power spectrum of FT of an image

f(x,y)

|F(u,v)|

original Low pass High pass

Image with periodic structure

f(x,y) |F(u,v)|

FT has peaks at spatial frequencies of repeated textures

Forensic application

|F(u,v)|

Remove peaks, equals periodic background

Energy and Phase

Seems like phase carries less information about the image …

The importance of phaseOppenheim’s demonstration

FFT FFT

Do you see: I1, I2, neither, or both?

1),(),(ˆ vujevuPvuI Φ= ),(22

2),(),(ˆ vujevuPvuI Φ=

Keble Another dinosaur

Keble phases the dinosaur

Keble Another dinosaur

the dinosaur phases Keble

Step edge detection by differentiation

Signal position, x

Profile of a step intensity change superimposed on a ramp

Basic feature detection -conventional approach

Signal position, x

ity/ d

Constant gradient + max at the step. Note amplified noise

Differentiation by the central difference is equivalent to convolution with the odd symmetric kernel

Differentiation as convolution

2)1()1()( −−+

ififdx

Approximate differentiation as central difference:

⎥⎦⎤

⎢⎣⎡−

Problem is that differentiation amplifies noise, suggesting pre-filtering the image with a noise reducing, smoothing filter

Convolution with the derivative of a Gaussian

)()( xfdx

xdG∗σ

Step detection by finding “significant” maxima of the convolution of the signal with a derivative of a Gaussian

Step superimposed on a ramp signal… take second derivative

Gaussian derivatives in 2D

),(),(

yxfeex

yxfexyxfyxGx

∗⎟⎠

⎞⎜⎝

⎛∗⎟⎠

⎞⎜⎝

⎛ −=

∗⎟⎠⎞

⎜⎝⎛ −

=∗∂∂

−−

σπσπ

πσGaussian is separable: equivalent to convolution in the x-direction and then in the y-direction

Directional derivative of Gaussian shown as a surface

Example

X-derivative Y-derivative

Gradient magnitude

Basic feature detection –zero crossingsin

Signal position, x

ity/ d

Zero crossing at step. But doubly amplified noise

Gaussian scale space

This shows a Gaussian blurred signal Gσ*S(t) as σ increases (from bottom to top)

The positions of the zero-crossings of the second derivative of the blurred signal are marked. This produces a “fingerprint”

Homomorphic filteringIllumination model: i(x,y)= illum(x,y)reflec(x,y) where i(x,y)is the image, illum(x,y) is the spatially slowly-varying illumination, and reflec(x,y) is the reflectance of visible surfaces, assumed piecewise constant.

Taking logs to convert to a sum (indicated by Capitalised names),then apply the FT:

)),(),(()),(),((),(

:FT inverse then takeand ),(filter suitable aApply ),(),(),(ˆ

vuReflecvuHvuIllumvuHyxEnh

vuHvuReflecvuIllumvuI

−− ℑ+ℑ=

Then the enhanced image is obtained by taking exponents of this equation.

Homomorphic filtering: example

Left: Original image. Right: processed using homomorphic filteringto achieve contrast enhancement and dynamic range compression.

Homomorphic filtering in practice

• The choice of the filter H(u,v) is critical– it must affect low and high frequencies in different

ways – ``illumination varies spatially slowly'' suggests low-

pass– H(u,v) is typically ``1-Butterworth''

• Note that noise was ignored …• Worse, reflectance also has a low frequency

component, so separation is often hard– ``bias field'' in MRI image enhancement

Signal/image restoration using the Fourier domain

The objective is to restore a degraded image to its original form. An observed image can often be modelled as:

),(),(),(),( yxnyxfyxhyxg +∗=

where f is the signal/image to be restored, h is the point spread function of the imaging process, n is additive noise, and g is the signal/image with which we are confronted. The aim is to restore f given g and some knowledge of n and h. Evidently: ),(ˆ),(ˆ),(ˆ),(ˆ vunvufvuhvug +=

-ˆˆ~ :noise gintroducin-re or,ˆ

:bygiven is ˆ of ~ estimatean moment, for the ˆ Ignoring

Evidently, problems if or is small, so noise dominates0ˆ ≈h g

Wiener filter

signal theofdensity spectralpower theis ),(~),(

noise theofdensity spectralpower theis),(ˆ ),(

),(/),(),(

where),(),(ˆ

),(ˆ),(ˆ

where),,(ˆ),(ˆ),(~

vufvuS

vunvuS

vuSvuSvuk

vukvuhvuhvuw

vuwvugvuf

constant suitable a toequalset is or ely,approximatknown are ,Often

attenuated are sfrequenciehigh then , largefor ),(ˆ),( if

filter) inverse (ie ),(ˆ/1),(~ then ,0),( If

vuvuhvuk

vuhvufvuk

Wiener filter example

Here, K is set to the constant value indicated

Inadequacies of the FT & attempts to overcome them

• The discrete world, signal changes, localisation

• Localisation vs SNR• Gaussian “scale space”• Isotropic and anisotropic diffusion for

image analysis• Estimating orientation in images :

steerable filters

The transient world

A sine function with a small discontinuity.The Fourier Transform consists of two delta functions, indicating the single dominant signal. Note the infinite tiny wiggles –across the entire spectrum.It detects that there is a discontinuity; but cannot say where

The basic problem with the FT for signal/image processing

Using the FT of the Rect function, we see that the FT of any function that is bounded in time, has frequencies over an infinite range. Conversely, the inverse FT of a perfect band-pass filter has temporal components over an infinite range.

We may detect that a transient has occurred; but we cannot detect when! If we are to treat transients in signal/image processing, we must find a way to overcome this problem. (The great idea in MRI is to add a gradient field to provide localisation; but that is infeasible for most signal and image processing.)

upperlower tttttf ≥≤= or for ,0)(

Space/time & frequency

• Shannon (1943) • Fourier (1807)

– the FFT– Human visual system “Frequency freaks” vs “feature

creatures”• Gabor (1950)

– The Heisenberg map• Wavelets (1978)

– the transient world

Heisenberg plane

This is a visual representation which aims to convey the way in which sampling is done between time (space) and frequency/scale

Shannon or time-sampled representation

The signal is sampled in the time domain, and the FT of each time sample covers all frequencies

The diagram attempts to capture the idea that Shannon/Nyquist theory samples signals in time; but that a rectangle function in time has infinite extent in frequency

Representation of the Fourier Transform

ω The signal is sampled in the frequency domain, and the inverse FT of each frequency sample is spread over all times

The practical import of the Fourier Transform* is that global structure of a signal (eg a sinusoid) is converted to a single localised point.

*This is the key property of ALL transforms: global is mapped to local, so is easier to process.

Time/frequency trade-offCan we construct a function f whose energy is both well localised in time and whose FT has energy concentrated in a small frequency neighbourhood?Note that, in general, to try to reduce the time spread of f, we might scale time by some s<1 while keeping its energy constant:

)(ˆ)(ˆ that Note

that so ,)(1)( 22

sufsuf

Since the FT of the scaled function is inevitably dilated by the scale parameter, we lose in frequency what we gain in time. In short, scaling doesn’t address the problem.

Is there a way to beat this problem? If not, what is the best trade-off we can manage?

Gabor’s great idea

• The windowed FT• Gabor “atoms”

– sinusoids multiplied by a Gaussian “window”• Uncertainty principle & Heisenberg map• Owens-Morrone-Kovesi theory of feature

detection– Analytic signal, Hilbert transform and

quadrature filters

The windowed FT

The windowed FT:

The function g is called a “window” function – it localises the Fourier integral in the neighbourhood of t = t0.

Various windowing functions have been proposed; but the most common is the Gaussian, as used by Denis Gabor(1950)

∫∞

∞−

−= dtettgtfutf tju0)()(),( 000S

Examples of the Windowed FT

[ intervalfrequency over the spread isenergy Its

)(ˆ),(

:(WFT) FT windowedhasIt

).(2)( FT has )( wavesinusoidalA

uugeutf

uuufetf

Sinusoidal wave

Dirac function

[ intervalfrequency over the spread isenergy Its

:(WFT) FT windowedhas )()(function DiracA

ttgeutf

Gabor functions

( )Gaussian. a toaccording off dies that sinusoid a is This

:frequency at the modulated is )( window(Gaussian) symmetric and real a

00ttGetg

tjuut −=

tCetg t 5cos)( 2/2−=

Gabor functions are sometimes called Morlet wavelets, after the French mathematician who independently discovered them

Gabor functions have uniform resolution in the

Heisenberg plane

dttgtt

∫∫

−=22

, oft independen is of spread timeThe

uniformly.it they tile:planeg Heisenber theacross resolutionconstant a have functionsGabor

).,( oft independen is that ),(sizea has functionGabor a ofbox g Heisenber that thefollowsIt

.ˆ FTfor the holdsproperty similar A

Gabor Transform in Heisenberg space

σtThe time and frequency samples are windowed by a Gaussian whose standard deviations are

The Heisenberg box of a Gaborfunction has a size (σt,,σξ) that is independent of the space time point (t0,ξ0)

They have constant resolution across scale space, so limited usefulness

Uncertainty principle

duufuuf

dttfttf

duufuf

dttftf

)(ˆ)(2

1ˆ of Variance

)()(1 of Variance

1 over ˆ of Average

)(1over of Average

Heisenberg uncertainty:4122 ≥ut σσ

The inequality is an equality if and only if

complex are , where,)(2)( baeaetf utbjut −−=

These modulated Gaussians are called Gabor chirps

Linear chirp

phase. theof derivative theas defined isfrequency ousinstantane The

.2)(frequency at maximum its reacheshat Gaussian t a is ),(, timefixed aFor 41

),( ),(

is chirplinear theof spectogram the

1)( indowGaussian w aFor

ith time.linearly w increases that frequency" ousinstantane"an has )( chirplinear The

41)2(2

attuutPt

utfutP

⎟⎟⎠

⎞⎜⎜⎝

σπσ

Instantaneous frequency

The thickness of the line is intended to indicate Gaussian window

Spectogram of a more complex signal

Top: the signal

Spectogram of the signal, which can be seen to contain a linear chirp, a quadratic chirp whose frequency decreases, and two modulated Gaussian functions (the black spots)

The lower figure shows the complex phase of the WFT of the signal in regions where the modulus of the spectogram is non-zero.

Earlier, we looked at step edge detection

However, even a moderately complex real world image has a wide variety of intensity change types

Feature detection

• There’s more to images than steps and corners– ramps, thin lines, and complex composite changes, ...

• The performance of feature detectors based on idealised models of a step/ridge degrade poorly on other kinds of features to which the Human Visual System is evidently sensitive

• Feature detectors based on amplitude of the gradient (local energy) are not invariant to contrast or brightness

Robyn Owens’ analysis of the Fourier components of “features”

Step up: all Fourier components have phase

Ridge up and down: all Fourier components have phase π/2 or 3π/2

a feature is defined as a point where there is local phase congruency

Phase congruencyLocally, a one-dimensional signal can be expressed as:

PC is amplitude-weighted, mean local

phase. eous)(instantan local theis )(and signal, theof (energy) amplitude eous)(instantan local theis )( where

)(cos)()(

φ∑=

Note that, as we saw earlier, the concept of local phase depends on “windowing” the signal. It also requires that we be able to estimate the local phase at each time point t. We discuss that on the next slide

∑∑ −

ttf))(cos(

max)(by defined is

point timeat the )( signal theof congruency phase The

Phase “congruency”

11,φA

22 ,φA

nnA φ,∑

iiiA φcos

The ratio of the amplitude-weighted sum of phasors (the red arrow) to the sum of amplitudes (the green arrow) is a measure of “congruency”

(a) original image

(b) PC feature map

(c) Canny edge strength image

(d) raw PC image

Boat image. Note the faithful reconstruction of the rigging

(a) original image

(b) PC feature map

(d) raw PC image

Estimating local phase• Suppose that we have “windowed” the signal at

time point t for example by localling blurring with a Gaussian

• To estimate phase, we need to calculate something like sin, cos locally – this suggests using even and odd Gabor functions

• However, Gabor filtered signals have negative frequencies and non-zero DC – these cause problems in practice

• Generally, we seek a quadrature pair of filters –pairs of filters whose Fourier transforms are rotated by π/2

Estimating phase

• Define the Hilbert Transform• Hilbert transform corresponds to rotation

by π/2 in the Fourier domain – Just like sine, cos – which is why the Hilbert

Transform is key to estimating local phase• Define the Analytic Signal• Finally, show how to estimate (local)

Hilbert transform)( ansformFourier tr uf

)(ˆ transform

Hilbert of ansformFourier tr

ufH)sgn()()(ˆ :have We

)sgn( function step

Heaviside theis 1 of ransform Fourier t theSince

:by defined is)( functiona of )( * The

tftfTransform Hilbert

⎟⎠

⎞⎜⎝

⎛⎟⎠⎞

⎜⎝⎛∗=

−⋅=

∫∞

∞−

τττ

Hilbert transform rotates by π/2 in the Fourier domain

( )( ) irH

ufjufuf

then),(ˆ)(ˆ)(ˆLet

⎥⎦

⎤⎢⎣

⎡⎥⎦

⎤⎢⎣

⎡−

=⎥⎦

⎤⎢⎣

θθθθ

cossinsincos

The case of interest is when θ=-π/2

Recall rotation in the plane:

Analytical Signal & Hilbert Transform

)()()(tan phase local

)()()(energy local

define

tous enables This .2

by rotated isbut )( as magnitude same thehas

)(ˆ ransform Hilbert t thedomain, Fourier thein that recall We

)( of Transform Hilbert theis )( where)()()(

:)( theusing defined are)( signala of phase) ,(amplitude properties local theally,Mathematic

tftftA

tftftjftftf

SignalAnalytic

Analytic signal

)()(ˆ)(ˆ Evidently, usignufufH ⋅=

Original signal FT of f

Hilbert transform of f

Analytic signal for f

Analytic signal and sine/cosineThe Fourier representation of a signal f(x) can be defined as :

∫∞

)(cos()()( ωωφωω dxAxf

Now include the imaginary part:

∫∫

∫∞∞

))(sin()())(cos()(

)()(ˆ

ωωφωωωωφωω

ωω ωφω

dxAjdxA

deAxf xj

Evidently, the imaginary part is a phase shifted version of the real part. Again, the imaginary part is the Hilbert Transform of f(x)

Implementation of Phase Congruency (Kovesi)

Need to estimate/correct for noise and estimate frequency spread

)01.0 (typically ,)(

)()( Calculate 3.

)()()(

)()( :Calculate 2.

)()()(

)()()(),()()(

,...,1for 1.function.Gabor (log) odd andeven the)(),(by Denote

∗=∗=

tAtWtPC

tHtFtW

tOtEtA

tMtftOtMtftE

nnntMtM

Calculation of phase congruency via the FFT

(a) original image

(b) PC feature map

(d) raw PC image

(a) original image

(b) PC feature map

(d) raw PC image

(a) original image

(b) PC feature map

(d) raw PC image

Demonstrating invariance to contrast. Canny disappears in shadow regions

(a) original image

(b) PC feature map

(d) raw PC image

PC: from signals to images

So far, we have described how to compute the phase congruency PC(x) at each point x of a signal. Now we ask how to extend the idea to images.

The immediate problem is that the Hilbert transform is only defined for signals, not for images.

Phase congruency of imagesAt each image location (x,y), we can choose a direction, θ, and interpolate the signal S(θ;x,y) in direction θpassing through x,y. We can then compute the PC(θ;x,y)

Finally, we can define the phase congruency for the image point (x,y) from some quantity computed from

{PC(θi;x,y):i=1..n}

Kovesi suggested using the averageof the directional phase congruences. That is what is shown in the results presented thus far.

Aerials are not steps, and are hard to detect

Output from a PC implementation developed by Brady and Liang in which directional phase congruency responses are combined in a rule-based manner

Phase congruency• Good news:

– responds to a set of diverse features– geometrically and photometrically invariant

• Bad news: – Being a normalised quantity, PC responds strongly to noise– PC is only meaningful if there is a spread of frequencies in a

signal; but it is not obvious how to define this spread– PC seems to require a different interpretation of scale,

suggesting that high-pass filtering rather than low-pass filtering should be used.

– PC can be defined in 1D, but, until recently it was not obvious how to calculate it in 2D, 3D, ...

Noise sensitivity of phase congruency

The Windowed FT seems to solve all our problems of time/space trade-off

…. or does it?

Sine & discontinuity

The Fourier Transform OR THE WINDOWED FOURIER TRANSFORM consists of two delta functions, indicating the single dominant signal. The wiggles show that it detects thatthere is a discontinuity; but cannot say where

Wavelets find the Sine & discontinuity

timeThe wavelet transform indicates thatthere is a discontinuity and shows where

Wavelets enable the discontinuity to be localised accurately

Note that the discontinuity appears at all scales, with higher SNR at coarse scales; better localisation at finer scales. Evidently, the feature exhibits congruency

This is a Scalogram. The vertical axis shows the scale nof analysis. The horizontal axis is time t. Dark means low values, light means high.

Example wavelets

A wavelet is a function of effectively limited duration whose average is zero

Wavelet analysis is breaking up a given signal into shifted and scaled versions of a basic “mother” wavelet

db2 Biorthogonal2.6

Wavelet definition

A wavelet is a function with zero average

which is dilated with a “scale parameter” s, and translated by some t0.

The function shape ω(t) is called the mother wavelet

∫∞

∞−

= 0)( dttψ

)(1)( 0,0 s

tst−

= ψψ

Wavelet transformThe wavelet transform of some signal f(t) at scale s, and at position t0 is defined from the correlation:

dtstttf

sstf ∫

∞−

⎟⎠⎞

⎜⎝⎛ −

= 0*0 )(1),( ψW

filters. bandpass dilated with ransform wavelet t thecomputesn convolutio Thefilter. bandpass a offunction transfer theis ˆ that appearsit

,0)(ˆ Since ).(ˆ)(ˆ is of FT The

where,),( that Notice

ψψψψψ

∫∞

∞−

⎟⎠⎞

⎜⎝⎛ −=

dttsusu

Continuous Wavelet Transform

∫∞

∞−

= dttpositionscaletfpositionscaleCWT ),,()(),( ψ

Wavelet scaling

attf ψ

ttf ψ

Five easy steps to a CWT1. Choose a wavelet and compare it to a section at the start of a signal

2. Calculate a number C that represents how closely correlated the wavelet is with this section of the signal. More precisely, if the signal energy and wavelet energy are both equal to 1, C may be interpreted as a correlation coefficient. Note that this depends on the wavelet.

Five easy steps to a CWT3. Shift the wavelet to the right and repeat steps 1, 2 until you have covered the whole signal.

4. Scale (stretch) the wavelet and repeat 1-3.

5. Repeat steps 1-4 for all desired scales

CWT scalogram

CWT scalogram as a surface

CWT of the noisy signal shown at the top, scalogram in the second row, final approximation signal in third row, and loci of maxima of scalogram over scale at bottom.

This uses the db4 wavelet

CWT of the noisy signal shown at the top, scalogramin the second row, final approximation signal in third row, and loci of maxima of scalogram over scale at bottom.This uses the Haar wavelet

Scale and frequency

• Low scale a compressed wavelet rapidly changing details high frequency ω

• high scale a stretched wavelet slowly changing, coarse features low frequency ω

Discrete wavelet transform

• Problem is that the CWT is a fair amount of work – it is impractical in most circumstances

• It turns out – remarkably – that if we limit ourselves to positions & scales based on powers of two – so called dyadic scales – we obtain the discrete wavelet transform (DWT)

• There is a fast DWT equivalent to the FFT• The first Fast DWT was developed by Mallat

(1987). Several have been developed since.

Approximation & detail

• approximations are high scale (low frequency) components of a signal

• details are the low scale (high frequency) components

• do this successively at several scales, as defined by DWT

Downsampling

Using complementary filters doubles the amount of data, so over a range of scales you drown in data.

Downsampling by 2 maintains the same amount of data at each scale. With a proper choice of filter pair, this yields the DWT.

We remember that this is the key to the FFT!

Downsampling II

Discrete multiresolution approximations aj[n] at scales 2j computed with cubic splines

Approximation Components

Detail Components

Original Signal

Discrete Wavelet Transform

We’ll see later that this can be efficiently extended to 2, 3, …dimensions

DWT of the signal shown in first row. The approximation at level 5 is shown in the second row, and the five detail signals are shown in rows 3 to 7.

Here we use the Haar wavelet.

DWT of the signal shown in first row. The approximation at level 5 is shown in the second row, and the five detail signals are shown in rows 3 to 7.

Here we use the db4 wavelet.

Extension of the FWT to 2 dimensions

The extension to 3D, 4D is straightforward.

Mallat’s depiction of the 2D FWT

Decomposition & reconstruction

This is the basis for efficient wavelet coding and decoding of signals, used increasingly in JPEG2000 and MPEG2000

Coding fidelity

Flushing the first several D’s can reduce the coding length, without sacrificing significant information

Discrete Wavelet Transform

We’ll see later that this can be efficiently extended to 2, 3, …dimensions

Wavelet bases & the fast WT1. Function spaces, orthonormal bases of functions, linear

approximation & projection 2. An approach is developed to multiresolution

approximations of a signal. This uses a sequence of subspaces of .

3. We show how approximation can be summarised by a single function θ.

4. This leads to the development of a scaling function ø(t) from which everything else is derived!

5. ø(t) yields a low-pass filter h(t), from which a high-pass filter g(t) is developed. Mallat calls the pair g,h a conjugate mirror pair.

6. finally the wavelet ψ(t) is derived from ø,g,h and the scheme leads directly to the Fast Wavelet Transform.

{ }Zjj ∈

V )(2 ℜL

Example of g, h

Solid line is the FT of h on [-π, π] and the dotted line is the FT of g. Note how h is approximately low pass and g is high pass. These are the filters from a cubic spline multiresolution wavelet.

Function spacesThe familiar idea of a vector space can be generalised to functions.For present purposes it is possible to skip most of the mathematics(of Banach, Hilbert, and Riesz spaces).

We need to define an inner product of two functions f(t), g(t). This is usually taken to be: ∫

∞−

>=< dttgtfgf )()(, *

with the resulting norm: 21

, >=< fff

00, all ,0,

>≥<><+><>=+<

ffgfgf

gfgfgfffggf

µλµλ

Orthonormal function bases{ }

if lorthonorma is )( functions ofset A

≠>=<

φφφφB

αφα s"coordinate"for :as expressed becan class in the function every if functions of class afor basis a is

∑ ><=n

nnff φφ, Evidently,

{ } Znjnue ∈ is basis lorthonormaFourier The

Linear approximation { }

∑−

:bygiven is basis theof functions first over the function a ofion approximatlinear A

basis.function lorthonormaan is Suppose

Since the basis is orthonormal, the approximation error is given by the sum of the remaining squared inner products:

∑∞

><=−=Mn

nM fffM 22 ,)( φε

the Fourier basis yields efficient linear approximations of uniformly smooth signals, with a projection over the M lower frequency sinusoids.in a wavelet basis, the signal is projected over the M larger scale wavelets, equivalent to approximating the signal at a fixed resolution.

Multiresolution approximationSuppose that the wavelet basis that we seek is:

{ } 22

21)( thatrecall weand )( ,),(, 2 ⎟⎟

⎞⎜⎜⎝

jnjZnjnjnt-tt ψψψ

Then, for any signal f(t) that we wish to analyse using the waveletbasis, we are interested in the partial sums of wavelet coefficients:

∑∞

−∞=

njnjf ,,, ψψ

The jth term of this sum can be interpreted as the difference between the approximations of f(t) at the resolutions 2-(j+1) and 2-j.

Multiresolution approximations compute the approximation of thesignal f(t) as orthogonal projections on different spaces Vj. The multiresolution approximation is composed of embedded grids ofapproximation.

Approximation spaces

jV1−jV

1 scalecoarser aat ionapproximat thecompute n toinformationecessary

theall contains scale aat tion (approxima

)2()(2

, ).( of

subspaces of sequence a develop We

∈−⇔∈

∈⎟⎠⎞

⎜⎝⎛⇔∈

⊆∀ℜ

{ }0 subspace resolutionfinest for the basis a are

n)-(t ons translatiits that so )(function a is ThereV

Znt ∈θθ

Examples of approximation spaces

• Piecewise constant approximations: the space Vj is the set of functions f(t) that are constant in the interval [n2j,(n+1)2j]. The approximation at resolution 2j is the closest piecewise constant function on intervals of size 2j . In this case θ(t) is the constant box function.

• Shannon approximations: the space Vj is the set of frequency band limited functions whose FT has a support included in [-2-jπ , 2-jπ ]. It turns out that the spaces can be defined by sinc functions of various frequencies, indeed, θ(t)= (sin πt)/ πt.

• Spline approximations: the provide smooth approximations with fast decay. The maths is a little more complex.

Scaling function øSuppose that we have been given the function θ(t). We use it, rather we use its FT, to define the scaling function ø(t) whose dilations and translations form a basis for the sequence of approximation spaces Vj.

{ } jZnnj

V of basis lorthonormaan is shown that becan It 22

)(ˆ)(ˆ Define

−∞=

⎟⎠⎞

⎜⎝⎛ −

For piecewise constant and Shannon approximations, ø=θ. However, this is rare …

Approximation using øWith this basis, the approximation of the signal f(t) over Vj is given by:

∑∞

−∞=

njnjffPj ,,,)( φφV

Denote the inner products by aj[n] since they provide an approximation of f at the scale 2j Given the scaling function ø, the approximation can be rewritten as a convolution:

21)(][

dtnttfna

⎟⎟⎠

⎞⎜⎜⎝

⎛ −= ∫

∞−

2 intervalsat sampled ,)( of version filtered pass-low a is

][ion approximat that theso ,],[in edconcentrat typicallyis )(ˆ FT theofenergy The ππφ −

Low pass filter h

)(ˆ)(ˆ2

1)2(ˆ :domainFourier In the

1][ where

)(][22

:for basis a is )( Since22

1 that and )(hat Remember t

ntnhtnt

>−⎟⎠⎞

⎜⎝⎛=<

−=⎟⎠⎞

⎜⎝⎛

⊆∈⎟⎠⎞

⎜⎝⎛∈

∑∞

−∞=

This relates a dilation of ø by 2 to its integer translations. The sequence h[n] can be regarded as a discrete filter.

Examples of hPiecewise constant approximations (eg. Haar)

⎪⎩

⎪⎨⎧ ==

>−⎟⎠⎞

⎜⎝⎛=<=

otherwise0

thatfollowsit ,)(,22

1][ sinceThen .]1,0[

nttnh φφφ 1

Polynomial cubic splines

cos212

)(ˆ)2(ˆ2)(ˆ

⎥⎥⎥

⎢⎢⎢

Orthogonal complements

0, and

:satisfy , on to sprojection its ,function any for Then,

: of complement orthogonal

thedefine now We., scaleeach at that know We

fPfPfPfPf

Wavelet bases & the fast WT1. Function spaces, orthonormal bases of functions, linear

approximation & projection 2. An approach is developed to multiresolution

approximations of a signal. This uses a sequence of subspaces of .

3. We show how approximation can be summarised by a single function θ.

4. This leads to the development of a scaling function ø(t) from which everything else is derived!

5. ø(t) yields a low-pass filter h(t), from which a high-pass filter g(t) is developed. Mallat calls the pair g,h a conjugate mirror pair.

6. finally the wavelet ψ(t) is derived from ø,g,h and the scheme leads directly to the Fast Wavelet Transform.

{ }Zjj ∈

V )(2 ℜL

Low pass filter h

)(ˆ)(ˆ2

1)2(ˆ :domainFourier In the

1][ where

)(][22

:for basis a is )( Since22

1 that and )(hat Remember t

ntnhtnt

>−⎟⎠⎞

⎜⎝⎛=<

−=⎟⎠⎞

⎜⎝⎛

⊆∈⎟⎠⎞

⎜⎝⎛∈

∑∞

−∞=

This relates a dilation of ø by 2 to its integer translations. The sequence h[n] can be regarded as a discrete filter.

High pass filter g

passlow is since pass, high is Evidently,]1[)1(][

:show thattorward"straightfo" isIt 1

hgnhng n −−= −

The high pass filter g is constructed directly from the low pass filter hfrom its FT.

)(*ˆ)(ˆ :domain Fourier thein )( define We π+= − uheugtg ju

Example of g, h

Solid line is the FT of h on [-π, π] and the dotted line is the FT of g. Note how h is approximately low pass and g is high pass. These are the filters from a cubic spline multiresolution wavelet.

wavelet ψ

{ } jZnnj

jnjntt

W of basis lorthonorma an is that proved be canIt 22

21)(ˆ

:by )( function thedefine ely)(suggestivNext

⎟⎟⎠

⎞⎜⎜⎝

⎛ −=

⎟⎠⎞

⎜⎝⎛

⎟⎠⎞

⎜⎝⎛=

High pass filter g and wavelet ψ

>−⎟⎠⎞

⎜⎝⎛=<

−=⎟⎠⎞

⎜⎝⎛ ∑

−∞=

)(][22

][ :tscoefficien thedefinecanwe together,all thisGathering

ntngtng

Schematic of the fast wavelet transform (Mallat)

g is the high pass filter constructed from the mother wavelet ω and the scaling function υ. h is the corresponding low-pass filter

Representation of the Wavelet Transform

sΨNote how the sampling in time scales with frequency: as frequency decreases, temporal sampling increases, and conversely

Shannon Fourier

Gabor / STFT Wavelet

Heisenberg “boxes”

Zoom Zoom Zoom

Debauchies’ wavelets: compact support and orthonormal

Debauchies’ wavelets are approximately fractal, hence their use in texture analysis

Segmenting an aerial image on the basis of texture: db4

Xie and Brady, 1994

Coifman’s wavelets

Almost symmetric, good frequency and space localisation, compact support, orthogonal. Excellent for picking out sharp, spatially localised features

Anisotropic Diffusion

Wavelet Analysis

Gaussian Scale-Space

Calcifications & vessel are much better preserved

Recall Phase Congruency• We used the Hilbert Transform to define local

phase• From that we defined phase congruency at each

point f(t) of a signal• We extended the definition to phase congruency

at each point I(x,y) of an image, by computing the directional phase congruency in a set of directions θi and then combining the results

The problems are (a) that this is intrinsically inefficient; and (b) is prone to interpolation errors in forming the directional signals.

Can we compute Phase Congruency of a 2- or 3-dimensional image without having to resort to one dimensional signal-like slices?

Monogenic signalPhase congruency

[ ])(),( from phase

Localbandpass. is )( where)(*)()(

signal thesmooth practice, Inamplitude phase, local Define

)(),( signal Analytic)( ansformHilbert tr

tbtbtftf

tftftf

( ) ( )[ ]

image. the in norientatio local phase local

(energy), amplitude local compute we),(),,(),,(

from then),,(is image smoothed bandpass theIf

).,(),,(domain space theinly equivalent

),,(),,( kernels, nconvolutiomoreor twouses transformRiesz

yxIhyxIhyxIyxI

yxhyxh

vuHvuH

∗∗

Monogenic signal

sinea is

and domain, Fourier thein cosinea is that Note

2),(ly equivalent,),(

2),(ly equivalent ,),( :where

),(),(

222222

221221

yyxhvu

xyxhvu

vuHvuH

⎥⎦

⎤⎢⎣

Computing the monogenic signal),( Image yxI

),(),(),(filter bandpass

yxbyxIyxI b ∗=

),(),(1 yxIyxh b∗ ),(),(2 yxIyxh b∗

phase local )()(),(tan

IhIhyx

∗∗

),(),(),( yxbyxIyxI b ∗=

energyor amplitude local )()(),( 2

bbb IhIhIyxE ∗+∗+=

image thein norientatio local )()(

),(tan2

1 bb IhIhIyx

∗+∗=φ

Monogenic triple of filters

Bandpass filter is a difference of Gaussians

h1*B h2*B

Though this generally works well, the difference of Gaussians is not the only bandpass filter that can be used …

Mammogram segmentation

50 100 150 200 250 300 350

Original mammogram image (log scale)

50 100 150 200 250 300 350

Local phase from log Gabor filters and monogenic signal

50 100 150 200 250 300 350

orientation

50 100 150 200 250 300 350

500 -3

50 100 150 200 250 300 350

Local energy, with breast edge marked

50 100 150 200 250 300 350

Phase < 90 degrees

50 100 150 200 250 300 350

Phase > 90 degrees

Motion at points in the heart

Displacement errors for two successive ultrasound images of the heart, prior to registration (left) and post registration (right). Near “features”, the error is less than a pixel.

Affine invariance

An image with the interesting points added: blue = high symmetry; red = high antisymmetry

Affine transformed house image with interest points superimposed (circles) and matched (stars).

Local symmetry

Points of local symmetry Locally “interesting” points

Blue = high symmetry; red = high antisymmetry

Advanced Transform Methods

Documents