AUTOMATED MOTION DETECTION FROM SPACE IN

SEA SURVEILLIANCE Elisavet Charalambous(a)(b), Junichi Takaku(c),Pantelis Michalis (d), Ian Dowman(e),

Vasiliki Charalampopoulou(f)

(a)ADITESS Ltd , Nicosia, Cyprus, (b)Department of Electrical and Computing Engineering, University of Cyprus, Nicosia, Cyprus, (c)Remote Sensing Technology Center of Japan, (d)Center for Security Studies (KEMEA), (e)Department of Civil,

Environmental and Geomatic Engineering, University College London, (f)Geosystems Hellas

Motivation

The objective in computer vision is to "understand" a scene or features in an image

• Humans are good at deriving pictorial information due to visual and mental abilities.

• Image processing of satellite images allows the extraction of geospatial (static) information.• Images are static snapshots in time

• The extraction of geometric characteristics along with the speed of movement and direction of vessels is of great interest in critical infrastructure protection and the security domain, in general.

• Automated motion detection allows the combination of information contained in periodically captured images reveals information on the dynamic content of a scene.

PRISM from ALOS

• The Panchromatic Remote-sensing Instrument for Stereo Mapping (PRISM) carried by the Advanced Land-Observing Satellite (ALOS) • designed to generate worldwide topographic data with its high-resolution and stereoscopic observation

• PRISM performs along-track (AT) triplet stereo observations using independent forward (FWD), nadir (NDR), and backward (BWD) sensors• The FWD and BWD sensors are arranged at an inclination of ±23.8◦ from NDR

• Time interval of 45secs between images

• The panchromatic optical line sensors have a ground resolution at 2.5m in swaths 35 km wide.

• Test case: the two large commercial ports in Cyprus (Larnaca, Limassol).

Registration of stereo images

• The PRISM triplet stereo images need to be geometrically registered before any processing

• Images are registered with the relative orientation technique• tie-points among the triplet images of level-1B1 are

automatically generated.

• Approximately 100 tie-points were generated for each image triplet• Used for estimating orientation errors

• The RMSE of x-parallaxes of the viewing vectors (FWD/NDR & BWD/NDR) is minimized • Ranging 0.76m to 1.50m corresponding to the relative

registration accuracy of each image triplet. • The absolute errors of the relative orientation models remain

almost in vertical offset due to instability of pitch angles of the FWD/BWD sensors;

Data Considerations

• Satellite images are essentially raster information • The cell (pixel) size determines the resolution at which the data is represented

• Imposes limits on the level of approximation

• Each pixel of ALOS PRISM is translated into an area of 6.25m2

• each pixel side corresponds to a distance of 2.5m

• around 360 million pixels exist in each image

• Sampling interval equal to twice the highest specimen spatial frequency is required in order to accurately preserve its spatial resolution • a pixel detail level of 2.5m may only adequately depict artifacts larger than 5m.

• critical limitation especially for small sized boats, close to each other or to the coastline

The objective

The computationally efficient automatic motion detection of vessels with sets of PRISM triplet images captured at time intervals of 45secs between images.

• Calculate the speed of movement

• Detect changes in orientation

• The process should be automatic and unbiased towards any dataset

• Through well established image segmentation and morphological operation techniques

METHODOLOGY

Pattern Extraction

• Image segmentation was accomplished by applying global thresholding to each image and serves as a computationally efficient method even for very large images• Limits the bias over any triplet of images

• The next phase involved outlining the areas occupied by vessels• One pixel gaps between extracted edges in the binary image were filled

• The connected edges were then filled and their boundaries were extracted

• Removal of the coastline took place after boundary detection

• Generation of pattern descriptors for each identified pattern for the reliable and robust tracking, over discontinuous snapshots

• Pattern Centroids are expressed as geo coordinates • The result of combining Geo-information and pictorial

coordinates

Metric Description

Area Actual number of pixels in the region

Perimeter

Length in pixels around the boundary of the region;

calculated as the distance between each adjoining pair of

pixels around the border of the region

RoundnessA ratio function to the area and perimeter of the region

(dimensionless ratio)

ElongationMinor axis width to major axis length ratio (dimensionless

ratio)

Bounding

box area

The area of the box completely containing the object

calculated as the product of the length of the major and

minor axes (dimensionless ratio)

EccentricityThe ratio of the distance between the foci of the ellipse

and its major axis length (dimensionless ratio)

Centroid Specifies the center of mass of the region

OrientationCalculated as the angle of the line segment connecting

two points to the equator

Solidity

Proportion of the pixels in the convex hull that are also in

the region. Computed as area/convex area (dimensionless

ratio)

METHODOLOGY

Pattern Identification

• Motion detection is performed with proximity searching for every two successive images (BWD/NDR & NDR/FWD)• Radius of searching: a function of the elapsed time between the timestamp of the images, the speed of moving and

ground resolution • The distance between the two points is calculated with the Harvesine distance metric

• Matches are signified when proximity search yields pairs of similar patterns• Matching Descriptors: roundness, elongation, eccentricity and solidity

• Area and perimeter used in multiple matching

• Calculation of speed as a function of the distance between the identified patterns and the elapsed time of movement and is reported as nautical mph

• Orientation of movement is calculated based on the angle between the equator and centroids of pattern in two consecutive images and is expressed in degrees• Avoids miscalculations for square shapes

• Provides direction of movement information

METHODOLOGY

Motion Detection

RESULTS

Test Case

• Test case: the two large commercial ports in Cyprus • Four triplets: two from Larnaka port & another two from Limassol port

• Experiment performed for proximity searches 230m & 460m corresponding to speeds of 10nmph & 20nmph respectively.

• Moving ships are captured at different snapshots where when combined prove their track, on the same lines, the traces of not moving ships fall on top of each other.

RESULTS

Example Patterns

Example patterns of successful motion detection and tracking

The analysis of triplets of the Larnaca port revealed patterns from aircrafts whose detection could be accomplished when the proximity search was performed for a large enough radius.

RESULTS

Limitations

• Undetected ships result from instances where the ship either relies outside the overlapping region or its trace changes significantly from one capture to another.

• Most misses occurred upon the detection of small vessels (with areas less than 10px), which the eye could not identify with certainty.

• The success rate varied from triplet to triplet: the method did not consider important image characteristics like image contrast

Scene ID (NDR) Detected Missed Error %

ALPSMN231682900 28 4 12.5

ALPSMN271942900 33 5 13.1

ALPSMN254292905 47 4 7.8

ALPSMN267712905 35 3 7.8

Conclusions

• The combination and further analysis of the results obtained by the individual snapshots allows the automatic motion detection of the overlapping scenes

• The more times a pattern is identified the higher the chance of generating reliable and plausible conclusions on the trajectory and route of a ship

• Dimensionless metrics serve relatively well the objective of pattern identification when the depiction of an object is not accurate

• The impact of limitations inherent by satellite images may be significantly decreased when the analysis expands outside the use of single triplets • Previously contained information may be used for increasing the reliability of already produced results

• Most vessels bear some common shape and operational properties which may be used as means of increasing robustness in presence of noise caused due to weather conditions.• Vessels are commonly print as elongated patterns• Acceleration and deceleration

Beyond identification….

• Along track stereoscopic view in cooperation with an effective automatic motion detection methodology can provide the position and the velocity of moving vessels

• The automatic detection of moving vessels and the ability to obtain operational properties gives rise to critical security information leading to:• Identification of irregular trajectories -> identification of abnormal behaviour

• ALOS PRISM along with any other Satellite system cannot provide near real time data, an airboard platform is needed for real time surveillance• Remote Piloted Aircraft Systems (RPAS) may serve as a solution