1
Mirza Muhammad Waqar
GEOREFERENCING OF IMAGES BY EXPLOITING GEOMETRIC DISTORTIONS IN
STEREO IMAGES OF UK DMC
Final Defense
2
Advisor Dr. Rafia Mumtaz (SEECS, NUST)
Guidance & Examination Committee Dr. Ejaz Hussain (IGIS, NUST) Dr. Rizwan Bulbul (IGIS, NUST) Muhammad Hussan (IGIS, NUST)
3
Contents
Overview Novel Contribution Objectives & Scope Research Novelty Literature Review Thermo elastic Model Model Inversion Results Conclusions
Georeferencing It is the process of assigning
geographic coordinates to a digital image.
It is a process for correcting spatial location and orientation of a satellite image
Types of Georeferencing Direct Georeferencing Indirect Georeferencing
Required for Geospatial Analysis
Change Detection Urban Planning
Map update
Overview4
5
Overview Traditional Method of Georeferencing
Require Ground Control Points (GCPs) GCPs are acquired manually and hence an expensive task Regions like deserts which lack salient features and have
homogeneous texture, selection of GCP is difficult A large number are required for complex terrain with varied surface
elevation. Accuracy of exterior orientation depends on the accuracy of GCP Not suitable for push broom imagery as every scanned line
possesses a different set of exterior orientation parameters Proposed method is based on Direct Georeferencing
Does not require Ground Control Points (GCPs) Utilizes
Satellite position/velocity data Attitude data Sensor Configuration to determine the pixel’s geo location.
6
Objectives
The primary goal of this research is to provide accurate georeferenced imagery using no GCPs by mitigating thermo elastic effect.
This will be accomplished by the following objectives
Modeling thermo elastic effect as a transformation matrix
Model inversion in order to extract the thermal deformation from the image offsets
Find geodetic coordinates by correcting pointing error Validate the complete model by testing it on UK-DMC
imagery
7
Novel Contributions
Modeled the thermo elastic effect as transformation matrix by exploiting the inter image offsets present between the pair of images.
Inverted the model to extract the thermal deformation knowledge to remove the pointing error.
Developed a new direct georeferencing method capable of modeling the pointing errors.
8
UK DMC – Sensor GeometryLaunch by Surrey Space Center, UK
in 2003 operated by DMC international imaging
Sensor: Push broom Made up of 6 CCD Channels Making 2 banks of 3 (Port and Star-
Board Array) Separate by an angle β across track Separate by an angle α along track Both Port & Star-board Array
have10000 linear CCD detectors 19500 pixel image width with 500
pixels overlap
GSD = 32 m Spectral Bands: Green, Red, NIR Ground Swath = 640 km
DMC Multi Spectral Imager (MSI)
Star-Boar
dArra
y
Port Array
9
Direct Georeferencing Direct Georeferencing
Transforms the image coordinate in camera frame to geodetic frame. Satellite onboard attitude data Satellite position and velocity data
One crucial point Accuracy of estimated geo-locations is
directly dependent on the accuracy of onboard sensor measurements
Main Advantage Requires no GCPs Becoming more robust and accurate
every year with the ongoing development in GPS and inertial equipment.
Can be applied with aerial camera, hyperspectral scanner, Synthetic Aperture Radar, LIDAR
10
Thermo-elastic Effect
Due to thermo-elasticity Satellite cools and heat up
periodically which causes it to contract and expand.
This introduces changes in the relative orientations of the attitude sensors (e.g. star tracker) and the imager.
(Attitude of Imager)
(From Attitude Sensor)
11
Literature Review on Thermo elastic Effect
In 2003, work on “Active pixel array devices in space missions“ has been done and it has been found that
The X-ray Telescope for NASA’s Swift mission incorporates a Telescope Alignment Monitor (TAM) to measure thermo-elastic misalignments between the telescope and the spacecraft star tracker.
In 2005, research on “Pleiades-HR Image System Products, Quality And Geometric Accuracy” has been done.
It has been found that attitude sensors are mechanically fixed on the telescope support to minimized the thermo elastic distortions.
12
Radhadevi et al in “In-flight geometric calibration of fore and aft cameras of CARTOSAT-1”devised a method for in-flight geometric calibration for Cartosat-1 The objective of this study is to ensure the best
absolute 3D pointing accuracy and relative location accuracy of the cameras.
It is concluded that accuracy of direct orientation observation could be brought down to better than 100 m with the inclusion of in-flight calibrated parameters in to the adjustment model.
Literature Review on Thermo elastic Effect
13
In 2008, a research is conducted on, “Attitude Performance Requirements and Budgeting for RASAT Satellite” In this study various sources of errors are
identified for RASAT satellite. These includes Star Tracker Errors Controller Errors Thermo elastic Error
Finally these errors are combined together in an error budgeting tool for analysis.
Literature Review on Thermo elastic Effect
14
Research Novelty In direct geo-referencing the major
source of error Thermo-elastic effect
Creates misalignment between the imaging sensor and the attitude sensor onboard.
Measured and realized orientations will be different.
Mechanical design changes to reduce the distortion by mounting the
imager/star-tracker assembly together on compliant mounts
But this solution will be costly and require changes in the physical mounting of the sensors.
Towards such ends we proposed to model the thermo elastic effect as a transformation matrix by using the inter-image offsets present
between the pair of images.
(Attitude of Imager)
(From Attitude Sensor)
15
Modeling Thermo-elastic Effect
The thermal deformation could be a rotation, scaling or some sort shearing in the pixels.T = Trotation + Tscale + Tshear Rotation
It has the major effect on the accuracy of geo-locations. Small deviations from the true orientation cause a large
displacement on the ground. Scaling
Due temperature changes the focal length of the imager contracts and expands.
This effects the scaling in the height (Z-direction) Shearing
Non parallel projection of imager results in shearing of pixels.
16
Modeling Thermo-Elastic Effect as a Rotation
This can be determined by
Rotation of Imager Determined by inter-image
offsets present between stereo pair.
Mathematical model has been developed
The inter image offsets equations are the function of senor configuration angles and attitude components.
(From Attitude Sensor)
(Attitude of Imager)
17 Let the port and starboard pixel in body frame be and Let T be the matrix that represents the thermo elastic
deformation.
Let A be the attitude matrix
Next step is to find the equations for the inter-image offsets.
Modeling Thermo-Elastic Effect as a Rotation
Column Shift Determine the corresponding Pixels Measure the distance of port and starboard pixel
End of their respective array Difference in the distances gives the column shift
Row Shift Time separation Delay for the corresponding
point to appear in the second image
18
Where
Where
Modeling Thermo-Elastic Effect as a Rotation
19
Direction in which satellite is moving
Δr
Δc
Δr be the row shiftΔc be the column shift
Star Board Array
Port Array
Overlapping region
Max Row ShiftMin Column Shift
Min Row ShiftMin Column Shift
Modeling Thermo-Elastic Effect as a Rotation
Inter-Image Offsets20
Row Shift Column Shift
At Nominal Attitude and nominal thermo-elastic effect
T =
Offset Modeling as Parabola & Straight Line
21
Offset Modeling Column shift represents
parabola Row shift represents
straight line Column shift as Parabola
Express parabola parameters in terms of attitude components.
H (peak value of parabola)
xo (x-intercept for peak value of parabola)
a (shape of parabola)
Attitude matrix
Where
Thermo elastic matrix
Para
bola
Offset Modeling as Parabola & Straight Line
22
Row shift as straight line
Express slope m and intercept c in terms of attitude parameters
Stra
ight
Lin
e
23
Effect of Attitude on the Offset Parameters
Image offset Para
meters
Roll Pitch Yaw
c 0.19 0.37 0.18m 2.66 0.04 0γp 0 0.01 0.002H 0 0 2.70
Model Inversion - Modeling Thermo elastic Effect as Rotation
Attitude is determined Estimating the unknown
components of attitude matrix by solving the offsets equations linearly. Use properties of attitude matrix
Mapping the estimated components to general attitude matrix
Determine the attitude by using the standard equations
Properties of Attitude Matrix
24
Attitude matrix
25
Modeling Thermo-elastic Effect as Scaling
Temperature changes effects the focal length of the imager Introduces height changes along the Zaxis
Change in scale (along z-axis) effects ground separation of the port and starboard imaging planes.
This will effect the row shift magnitude. Hence row shift constant equation will be used to
extract
26
The cofactor of the Tscaling matrix can be written as,
Where
The cofactor terms appear in expression of row and column offset parameters.
Modeling Thermo-elastic Effect as Scaling
27
Modeling Thermo-elastic Effect as Scaling
Column offset Equations
Where
28
Modeling Thermo-elastic Effect as Scaling
Row Offset Equations Similarly using cofactors, the expression for b1,
b2, b3, b4, b5, b6, b7 and b8 can be reduced to
Using above values, the row shift parameters can be written as
29
Simulations – Sensitivity of Scaling w.r.t offset parameters
30
Scale Extraction from row offset
With scaling matrix the row offset equation will take the form
From the above equation can be found as
31
Modeling Thermo-elastic Effect as Shearing
Shearing slides one edge of an image along the X or Y axis, creating a parallelogram.
The amount of the shear is controlled by a shear angle
32
The cofactors of shear matrix can be written as
The cofactor terms appear in the expression of row and column offset parameters.
Hence
Modeling Thermo-elastic Effect as Shearing
33
Column Offsets Equations
Row Offset Equations
Modeling Thermo-elastic Effect as Shearing
Where
34
Simulations – Sensitivity of Shearing w.r.t Offset
Parameters
35
Shear Extraction from column offset
Shear pushes the pixels across the track thus effecting the column offset. Therefore column offset will be used to find the shear factor
where
36
Thermo-elastic Matrix
Thermo-elastic matrix will be the sum of rotation, scale and shear extracted from the image offsets
Now inserting this matrix between the body and orbital frame will mitigate the misalignment between the imager and the attitude sensor.
37
Direct Georeferencing using onboard Attitude Sensor
Study Area: UK Date: May 23, 2004 Rows: 16250 Columns: 10000 GPS Positions
X = 4500717.0 m Y = -55020.32 m Z = 5445201.0 m
Attitude Values Roll = 32 centidegree Pitch = -25
centidegree Yaw = 47 centidegree
38
Attitude & GPS data Provided by SSTL
GPS Data
Attitude Data
39
Direct Georeferencing using exterior orientation data
By using the onboard exterior orientation data, the accuracy of 10-15Km has been achieved.
40
Row Shift Column Shift
Offset estimation using Window based Scheme
41
Measured Image offsets over the entire Scene length
42
Stereo Angle Estimation
Prior to model inversion, the sensor configuration angles of UK DMC must be determined which are α and β.
The β is being determined by SpaceMetric β = 12.6448o.
However the magnitude of α can be found from Rearranging equation of row offset’s constant
For UK DMC band 3 image pair, the value of α was found to be 0.04560.
43
Direct Georeferencing using Imager Attitude
By using imager attitude, the accuracy of 7-10Km has been achieved.
44
Thermo elastic Rotation
45
Direct Georeferencing using thermo elastic Rotation
By applying the thermo elastic Rotation
The accuracy of 1-5Km has been achieved.
46
Thermo elastic Scale
47
Direct Georeferencing using thermo elastic Rotation and Scale
By applying the thermo elastic Rotation Scale
The accuracy of 1-3Km has been achieved.
48
Thermo elastic Shear
49
Georeferencing using thermo elastic Rotation, Scale and Shear
By applying the thermo elastic Rotation Scale Shear
The accuracy of 1-1.5Km has been achieved.
50
Potential Benefits No GCPs required:
The major benefit of this approach is that it does not based on GCPs. Collection of GCPs is time consuming and expensive Difficult to collect GCPs having homogeneous texture Large number of GCPs are required for complex terrain Unsuitable for pushbroom imagery
Thermo elastic correction To date, modeling of thermo elastic effect as a transformation matrix has
not been explored. The thermo elastic effect is deemed as major source of error in this work Not only provide cost effective solution but also mitigate pointing error
No additional hardware Low cost and low mass system for obtaining geolocation Can work with conventional EO cameras with no additional hardware
Frequent Attitude Observations Due to small baseline or angular separation between the sensors, the
registration time for the corresponding pixels is very small which results in rapid attitude observations.
51
Conclusions A novel method for measuring geolocation without GCPs has been
developed. The error in geolocations estimate has been addressed by modeling thermo
elastic effect as a deformation matrix. Mathematical model has been developed by exploiting inter image offset
between pair of images. The possible deformations that has been considered are
Rotation, Scaling & Shearing These deformations have been modeled individually and has been
summed at the end to represent the entire thermo elastic deformation. The row and column offset parameters have been simulated individually
to determine The best candidate for extracting these individual deformation.
Developed mathematical model has been validated on UK DMC imagery. Accuracy of 1-1.5Km is achieved using newly developed georeferencing
method.
52
Future Work
Developing Generic Model for thermo elastic effect
Can be applied on other celestial bodies (e.g. Moon)
53
1. Rafia, M, P.L.Palmer,. Waqar, M.M. Georeferencing of images without Ground Control Points by Exploiting Geometric Distortions in Stereo Earth Images. Journal of Remote sensing of the Environment (Manuscript Submitted) Impact Factor ~ 3.95
2. Waqar, M.M., Johum, F.M., Rafia, M., Ejaz, H. (2012) Development of New Indices for extraction of Built-up area & Bare Soil from Landsat Data. Journal of Geophysics & Remote Sensing. (Manuscript Accepted).
3. Waqar, M.M., Rafia, M., Sufyan, N., Mustafa, M. (2012) Accuracy Assessment of Geo-locations using Multi-Resolution Interpolated DEMs. 2012 4th International Conference on Digital Image Processing. Kuala Lumpur, Malaysia. Published by SPIE.
Research Publications
54
Acknowledgement
55
Questions - Discussion