GLOBAL MOTION ESTIMATION OF SEA ICE USING SYNTHETIC APERTURE RADAR IMAGERY Mani V. Thomas.

GLOBAL MOTION ESTIMATION OF SEA GLOBAL MOTION ESTIMATION OF SEA ICE USING SYNTHETIC ICE USING SYNTHETIC

APERTURE RADAR IMAGERYAPERTURE RADAR IMAGERY

Mani V. ThomasMani V. Thomas

Problem StatementProblem Statement Sea-Ice dynamics is composed ofSea-Ice dynamics is composed of

Large global translationLarge global translation Small local non-rigid dynamicsSmall local non-rigid dynamics

Robust estimation of global motion provides a Robust estimation of global motion provides a base for processing of non-rigid componentsbase for processing of non-rigid components

““Given a pair of ERS – 1 SAR images, this Given a pair of ERS – 1 SAR images, this thesis presents a method of estimating the thesis presents a method of estimating the global motion occurring between the pair global motion occurring between the pair robustly”robustly”

IntroductionIntroduction Investigation into the robust estimation of the global Investigation into the robust estimation of the global

motion of sea ice as captured by the European Remote motion of sea ice as captured by the European Remote Sensing Satellite (ERS) imagery.Sensing Satellite (ERS) imagery.

Reasons for estimation complexityReasons for estimation complexity

Differences in the swaths of the satellite and the rotation of the Differences in the swaths of the satellite and the rotation of the earthearth

the local sea-ice dynamics is over shadowed by the large the local sea-ice dynamics is over shadowed by the large magnitudes of the global translationmagnitudes of the global translation

Time difference between the adjacent frames (typically three Time difference between the adjacent frames (typically three days due to polar orbit constraints)days due to polar orbit constraints)

Influence of fast moving storms

Significant non-linear changes in the discontinuities occur at temporal scales much lesser than 3 days

Motion Estimation ProblemMotion Estimation Problem

““Optic Flow is computed as an approximation of Optic Flow is computed as an approximation of the image motion defined as the projection of the the image motion defined as the projection of the velocities of 3-D surface points onto the imaging velocities of 3-D surface points onto the imaging plane” plane” [Beauchemin, 1995][Beauchemin, 1995]

Image Brightness Constancy assumptionImage Brightness Constancy assumption

Apparent brightness of a moving object remains Apparent brightness of a moving object remains constant [Horn, 1986]constant [Horn, 1986]

Under the assumption of extremely small temporal resolution Under the assumption of extremely small temporal resolution the optic flow equation is considered validthe optic flow equation is considered valid

0

,,0

t

E

dt

dy

y

E

dt

dx

x

E

dt

ttytxdE

dt

dE

Motion Estimation ProblemMotion Estimation Problem Estimation techniques can be classified into Estimation techniques can be classified into

three main categories [Kruger, 1996]three main categories [Kruger, 1996]

Differential methods [Horn, 1981] [Robbins, 1983]Differential methods [Horn, 1981] [Robbins, 1983]

Image intensity is assumed to be an analytical function in the Image intensity is assumed to be an analytical function in the spatio-temporal domainspatio-temporal domain

Iteratively calculates the displacement using the gradient Iteratively calculates the displacement using the gradient functional of the imagefunctional of the image

work well for sub-pixel shifts but they fail for large motionswork well for sub-pixel shifts but they fail for large motions

extremely noise sensitive due numerical differentiationextremely noise sensitive due numerical differentiation

convergence in these methods can be extremely slowconvergence in these methods can be extremely slow

Motion Estimation ProblemMotion Estimation Problem Area based methods [Jain, 1981], [Cheung, 1998]Area based methods [Jain, 1981], [Cheung, 1998]

The simplest way in terms of both hardware and software The simplest way in terms of both hardware and software complexity complexity

Implemented in most present day video compression Implemented in most present day video compression algorithms [ISO/IEC 14496-2, 1998; ITU-T/SG15, 1995]algorithms [ISO/IEC 14496-2, 1998; ITU-T/SG15, 1995]

Estimation is performed by minimizing an error criterion such Estimation is performed by minimizing an error criterion such as “Sum of Squared Difference”as “Sum of Squared Difference”

Not satisfied completely since motion in real life can be Not satisfied completely since motion in real life can be considered a collage of various types of motionsconsidered a collage of various types of motions

i

nInISSD iix

vxx 21,,

Motion Estimation ProblemMotion Estimation Problem Feature based methodsFeature based methods

Identify particular features in the sceneIdentify particular features in the scene

computes the “feature points” between the two images using computes the “feature points” between the two images using corner detectors [Harris, 1998; Tomasi, 1991]corner detectors [Harris, 1998; Tomasi, 1991]

Deducing the motion parameters by matching the extracted Deducing the motion parameters by matching the extracted featuresfeatures

Matching the detected feature between the two images using Matching the detected feature between the two images using robust schemes such as RANSAC [Fischler, 1981]robust schemes such as RANSAC [Fischler, 1981]

Full optic flow is known at every measurement positionFull optic flow is known at every measurement position

Only a sparse set of measurements is availableOnly a sparse set of measurements is available

Reduction of the amount of information being processedReduction of the amount of information being processed

Fourier TheoryFourier Theory

Fourier Transform of Aperiodic signalsFourier Transform of Aperiodic signals

Fourier Analysis equationFourier Analysis equation

Fourier Synthesis equationFourier Synthesis equation

Fast Fourier Transform [Cooley, 1965]Fast Fourier Transform [Cooley, 1965]

Reduces computation from toReduces computation from to

Fourier shift TheoremFourier shift Theorem

Delay in the time domain of the signal equivalent to a rotation Delay in the time domain of the signal equivalent to a rotation of phase in the Fourier domainof phase in the Fourier domain

dxexfsF xsj 2

dsesFxf xsj 2

2n nn log

sFedxf dj

IFFT

FFT 2

Fourier TheoryFourier Theory Phase CorrelationPhase Correlation

Given cross correlation equation in Fourier DomainGiven cross correlation equation in Fourier Domain

Inverse Fourier Transform of the product of the individual forward Inverse Fourier Transform of the product of the individual forward Fourier TransformsFourier Transforms

By the Fourier Shift Theorem in 2DBy the Fourier Shift Theorem in 2D

Sharpening the cross correlation using and Sharpening the cross correlation using and [Manduchi, 1993][Manduchi, 1993]

Inverse Fourier Transform provide a Dirac delta function Inverse Fourier Transform provide a Dirac delta function centered at the translation parameterscentered at the translation parameters

uuxxx GFgf 11

duuuuux TjeFFGF 211

1

1

ux

FH 1

2

ux

GH

duduuu

uuuuuu

Tj

GF

GFGHFH 2exp11

211

Global Motion EstimationGlobal Motion Estimation Generalized Aperture ProblemGeneralized Aperture Problem

Uncertainty principle in image analysisUncertainty principle in image analysis

Smaller the analysis window, greater the number of possible Smaller the analysis window, greater the number of possible candidate estimatescandidate estimates

Larger the analysis window size, the greater is the probability Larger the analysis window size, the greater is the probability that the analysis window has a combination of various that the analysis window has a combination of various motionsmotions

Handle the motion estimation at multiple resolutionsHandle the motion estimation at multiple resolutions

Information percolation from coarser resolution to finer Information percolation from coarser resolution to finer resolution in a computationally efficient fashion. resolution in a computationally efficient fashion.

Motion smaller than the degree of decimation is lostMotion smaller than the degree of decimation is lost

Global Motion EstimationGlobal Motion Estimation Global translations, in ERS-1images, are on the order of Global translations, in ERS-1images, are on the order of

100 to 200 pixels100 to 200 pixels

““Normalized Cross Correlation” (NCC) or “Sum of Normalized Cross Correlation” (NCC) or “Sum of Squared Distance” (SSD) require large support windows Squared Distance” (SSD) require large support windows to capture the large translationto capture the large translation

Large support windows encompass a combination of various Large support windows encompass a combination of various motionsmotions

Images have varying degrees of illumination due to the degree Images have varying degrees of illumination due to the degree of back scatterof back scatter

SSD is extremely sensitive to the illumination variation SSD is extremely sensitive to the illumination variation though computationally tractablethough computationally tractable

NCC is invariant to illumination but is computationally NCC is invariant to illumination but is computationally ineffectiveineffective

Global Motion EstimationGlobal Motion Estimation Phase correlation is illumination invariant [Thomas, Phase correlation is illumination invariant [Thomas,

1987]1987]

Characterized by their insensitivity to correlated and frequency-Characterized by their insensitivity to correlated and frequency-dependent noisedependent noise

Calculations can be performed with much lower Calculations can be performed with much lower computational complexity with 2-D FFTcomputational complexity with 2-D FFT

It can be used robustly to estimate the large motions It can be used robustly to estimate the large motions [Vernon 2001] [Reddy, 1996] [Lucchese, 2001][Vernon 2001] [Reddy, 1996] [Lucchese, 2001]

Separation of affine parameters from the translation components Separation of affine parameters from the translation components [De Castro 1987] [Lucchese 2001] [Reddy 1996][De Castro 1987] [Lucchese 2001] [Reddy 1996]

Main disadvantage is applicability only under well-Main disadvantage is applicability only under well-defined transformationsdefined transformations

Global Motion EstimationGlobal Motion Estimation Phase Correlation v/s NCCPhase Correlation v/s NCC

Uni-modal Motion distribution Uni-modal Motion distribution within the search windowwithin the search window

Phase correlation and NCC Phase correlation and NCC have maxima at the same have maxima at the same positionposition

Multi modal motion distribution Multi modal motion distribution within search windowwithin search window

NCC produces a number of NCC produces a number of local maxima local maxima

Phase correlation produces Phase correlation produces reduced number of possible reduced number of possible candidatescandidates

Remark: Basis for support in both methods have been maintained at 96 pixels window

Global Motion EstimationGlobal Motion Estimation Histogram Equalization by Mid-Tone modificationHistogram Equalization by Mid-Tone modification

Image enhancement and histogram equalization performed over Image enhancement and histogram equalization performed over “visually significant regions” as against the entire image“visually significant regions” as against the entire image

Simple histogram equalization suffers from speckle noise and false Simple histogram equalization suffers from speckle noise and false contouring [Bhukhanwala, 1994]contouring [Bhukhanwala, 1994]

Experiments indicate that estimated motion field had the smallest error Experiments indicate that estimated motion field had the smallest error variance under mid tone modificationvariance under mid tone modification

Global Motion EstimationGlobal Motion Estimation Creation of Image Hierarchy by Median Filtering Creation of Image Hierarchy by Median Filtering

Multi-resolution image hierarchy by decimation in the spatial scale [Burt, Multi-resolution image hierarchy by decimation in the spatial scale [Burt, 1983]1983]

Aliasing due to the signal decimationAliasing due to the signal decimation

Reduced using Median filteringReduced using Median filtering

Small motions tend to get masked during the process of image decimation Small motions tend to get masked during the process of image decimation

Masking is advantageous for global motion estimationMasking is advantageous for global motion estimation

Motion Estimation in Image HierarchyMotion Estimation in Image Hierarchy

Motion estimated at the coarsest level of the pyramidMotion estimated at the coarsest level of the pyramid

Estimate is percolated to the finer levels in the pyramid by warping the Estimate is percolated to the finer levels in the pyramid by warping the images towards one anotherimages towards one another

Process iterated until the finest level of the pyramidProcess iterated until the finest level of the pyramid

Reduces the computational burden since the coarse estimate is performed Reduces the computational burden since the coarse estimate is performed on smaller imageson smaller images

Global Motion EstimationGlobal Motion Estimation Histogram based global motion EstimationHistogram based global motion Estimation

Images divided into a tessellation of blocks, each block centered Images divided into a tessellation of blocks, each block centered within a predefined window. within a predefined window.

Window size, Block size and pyramid levels obtained as a Window size, Block size and pyramid levels obtained as a parameter from the end userparameter from the end user

Motion estimated at each block using phase correlationMotion estimated at each block using phase correlation

Potential candidates are selected such that their magnitudes are Potential candidates are selected such that their magnitudes are higher than a thresholdhigher than a threshold

The best possible estimate obtained from the potential The best possible estimate obtained from the potential candidates using the “Lorentzian estimator” [Black, 1992]candidates using the “Lorentzian estimator” [Black, 1992]

The global motion at a level of pyramid is obtained as the mode The global motion at a level of pyramid is obtained as the mode of the motion vectors at that levelof the motion vectors at that level

Global Motion EstimationGlobal Motion Estimation Due to the periodic nature of the Due to the periodic nature of the

Discrete Fourier Transform, the Discrete Fourier Transform, the maximum measurable estimate using maximum measurable estimate using the Fourier Transform of a signal within the Fourier Transform of a signal within a window of size W is W/2. a window of size W is W/2.

To capture translations of magnitude (u, To capture translations of magnitude (u, v), the W should be >= 2*max(u,v)v), the W should be >= 2*max(u,v)

For the ERS-1 experimentation, the For the ERS-1 experimentation, the block size was taken as 32X32 and the block size was taken as 32X32 and the window size was taken as 128X128. window size was taken as 128X128.

The sizes of the window and the block The sizes of the window and the block are maintained a constant throughout are maintained a constant throughout the entire pyramid hierarchythe entire pyramid hierarchy

Amplification of the estimates at the Amplification of the estimates at the finer level of the pyramidfiner level of the pyramid

Functional Description of ModulesFunctional Description of Modules The first level image processing related The first level image processing related

functional units. functional units.

The image reader reads the image into The image reader reads the image into buffersbuffers

The image modifier that performs The image modifier that performs histogram equalization histogram equalization

Create image hierarchyCreate image hierarchy

The second level performs the global The second level performs the global motion estimationmotion estimation

Performs phase correlation on the image Performs phase correlation on the image pyramidpyramid

analyzer functional module performs analyzer functional module performs histogram analysis of the motion datahistogram analysis of the motion data

The final level performs local motion The final level performs local motion estimationestimation

Affine components of the local non rigid Affine components of the local non rigid deformations or a higher order parametric deformations or a higher order parametric modelmodel

Data SetsData Sets The European Space Agency’s ERS – 1 and The European Space Agency’s ERS – 1 and

ERS – 2 C-band (5.3 GHz) Active Microwave ERS – 2 C-band (5.3 GHz) Active Microwave Instrument generate RADAR images of the Instrument generate RADAR images of the Southern Ocean sea-ice cover in Antarctica, in Southern Ocean sea-ice cover in Antarctica, in particular the Weddell Seaparticular the Weddell Sea

Weather independent (day or night)Weather independent (day or night)

Frequent repeatFrequent repeat

High resolution 100 km swathHigh resolution 100 km swath

The 5 month Ice Station Weddell (ISW) 1992 The 5 month Ice Station Weddell (ISW) 1992 was the only winter field experiment performed was the only winter field experiment performed on the Western Weddell Sea. on the Western Weddell Sea.

The orbit phasing of the ERS – 1 was fixed in The orbit phasing of the ERS – 1 was fixed in the 3-day exact repeating orbit called the ice-the 3-day exact repeating orbit called the ice-phase orbit phase orbit

Uninterrupted SAR imagery of 100 x 100 kmUninterrupted SAR imagery of 100 x 100 km2 spatial coverage of during the entire duration of spatial coverage of during the entire duration of the experimentthe experiment

Courtesy: http://www.ldeo.columbia.edu/res/fac/physocean/proj_ISW.html

Data SetsData Sets SAR images obtained from ERS -1 are projected onto the SSM/I gridSAR images obtained from ERS -1 are projected onto the SSM/I grid

For the SAR imagery in the Southern Hemisphere, the tangent plane was moved For the SAR imagery in the Southern Hemisphere, the tangent plane was moved to 70to 70o

S and the reference longitude chosen at 0S and the reference longitude chosen at 0o

Values are transformed to X-Y grid coordinates using polar stereographic Values are transformed to X-Y grid coordinates using polar stereographic formulaeformulae

The digital images are speckle filtered to a spatial resolution of 100mThe digital images are speckle filtered to a spatial resolution of 100m

Images with dimensions of 1536 pixels in the horizontal and vertical direction Images with dimensions of 1536 pixels in the horizontal and vertical direction

Specified using a concatenation of orbit number and the frame numberSpecified using a concatenation of orbit number and the frame number

Courtesy: http://nsidc.org/data/psq/grids/ps_grid.html

Data SetsData Sets

Validation Motion Vectors (Ground Truth Validation Motion Vectors (Ground Truth JPL Motion Vectors)JPL Motion Vectors)

Motion vectors for each 100x100 km2 SAR images were resolved using a nested cross-correlation procedure [Drinkwater, 1998] to characterize 5x5 km2 spatial patterns.

A total of 12 such image pairs exist from this processing with an RMSE of less than 0.5 cm/s

Results and AnalysisResults and Analysis The code for performing the motion field estimation has been written The code for performing the motion field estimation has been written

C (VC++ 6.0) with the validation prototype written in Matlab 6.1 C (VC++ 6.0) with the validation prototype written in Matlab 6.1 (R12).(R12).

Window size is chosen a power of 2 Window size is chosen a power of 2

Maximize the throughput of the FFT modules, Maximize the throughput of the FFT modules,

The block size adjusted at 8x8, 16x16 or 32x32 depending on the The block size adjusted at 8x8, 16x16 or 32x32 depending on the spatial resolution spatial resolution

Output motion field at 0.8 km, 1.6 km or 3.2 km resolution.Output motion field at 0.8 km, 1.6 km or 3.2 km resolution.

Estimated motion field in the images below have been computed Estimated motion field in the images below have been computed using a 32x32 block size and a 128x128 window sizeusing a 32x32 block size and a 128x128 window size

These are overlaid on the JPL vectors on a 5km grid in the SSM/I These are overlaid on the JPL vectors on a 5km grid in the SSM/I coordinates, using linearly interpolationcoordinates, using linearly interpolation

Results and AnalysisResults and Analysis

Two statistical measures of the similarity have Two statistical measures of the similarity have been computed for the magnitude and the been computed for the magnitude and the directiondirection

Root Mean Square Error Root Mean Square Error

Index of agreement [Willmott,1985]Index of agreement [Willmott,1985]

where pwhere pk are the estimated samples, o are the estimated samples, ok are the observed are the observed samples (ground truth vectors), wsamples (ground truth vectors), wk are the weight functions are the weight functions

K

kkk op

KRMSE

1

21

1

11

1

K

kkkk

K

kkkk ooopopd


Comparison of 34025103 and 34125693: estimated vectors v/s JPL vectors











Results and AnalysisResults and Analysis Motion estimation between Motion estimation between

the 31975693 and 32305103the 31975693 and 32305103 Block resolution of 4x4Block resolution of 4x4

Observation of a turbulent field Observation of a turbulent field using higher resolution of using higher resolution of analysis windowanalysis window

Cluster map using a Quad-Cluster map using a Quad-tree modeltree model Based on the variance of the Based on the variance of the

magnitude and direction of the magnitude and direction of the motion fieldmotion field


Comparison of 30585103 and 30685693: Turbulent zone

Results and AnalysisResults and Analysis Local Motion AnalysisLocal Motion Analysis

Simplest model of local Simplest model of local motionmotion

Piecewise linear Piecewise linear approximation of the non approximation of the non rigid motion using phase rigid motion using phase correlationcorrelation

Differential motion overlaid Differential motion overlaid upon the correlation map of upon the correlation map of the goodness of the estimatethe goodness of the estimate

Regions of low correlation Regions of low correlation provide the positions of provide the positions of discontinuities in the ice discontinuities in the ice motionmotion

Results and AnalysisResults and Analysis False discontinuities due to projection of the non linear components, of the False discontinuities due to projection of the non linear components, of the

higher order motion, onto a linear motion space via phase correlationhigher order motion, onto a linear motion space via phase correlation

Abrupt changes in the frequency components cause abrupt variations in the Abrupt changes in the frequency components cause abrupt variations in the estimated vector fieldestimated vector field

Sub-pixel motion interpolation using a cubic spline. Sub-pixel motion interpolation using a cubic spline.

Within a window around the result of the local phase correlation, a cubic spline Within a window around the result of the local phase correlation, a cubic spline was fit and the peak of the spline so estimated was used as the sub-pixel motion was fit and the peak of the spline so estimated was used as the sub-pixel motion estimate.estimate.

This procedure reduced the bands of discontinuities within the motion fieldThis procedure reduced the bands of discontinuities within the motion field

The main disadvantage is the computational burden of fitting a cubic slpineThe main disadvantage is the computational burden of fitting a cubic slpine

ConclusionConclusion Robust calculation of the motion occurring between two ERS-1 SAR sea-ice Robust calculation of the motion occurring between two ERS-1 SAR sea-ice

imagesimages

Under the assumption that the net motion is composed of a large global Under the assumption that the net motion is composed of a large global motion component and small local deformationsmotion component and small local deformations

Phase Correlation provides a robust method to capture the large global motion Phase Correlation provides a robust method to capture the large global motion componentcomponent

Inherent robustness to illumination variationInherent robustness to illumination variation

Reduced computational burden due to FFTReduced computational burden due to FFT

Having eliminated the global motion, estimate the local deformation using a higher Having eliminated the global motion, estimate the local deformation using a higher order motion model such as an affine or a quadratic.order motion model such as an affine or a quadratic.

Subsequent stage to the current researchSubsequent stage to the current research

Improvement of local estimation from a simple piecewise linear approximation to Improvement of local estimation from a simple piecewise linear approximation to using a robust higher order motion modelusing a robust higher order motion model

Feature-based approaches to improve the overall robustness of the global motion Feature-based approaches to improve the overall robustness of the global motion estimatesestimates

Thank youThank you

Date post:	20-Dec-2015
Category:	Documents
View:	217 times
Download:	1 times

GLOBAL MOTION ESTIMATION OF SEA ICE USING SYNTHETIC APERTURE RADAR IMAGERY Mani V. Thomas.

Documents