A Framework of Ontology-Based Knowledge Information Processing for Change Detection in Remote Sensing
Data
Shutaro Hashimoto1, Takeo Tadono1,2, Masahiko Onosato1, Masahiro Hori1,2, and Takashi Moriyama1,2
1Graduate School of Information Science and Technology, Hokkaido University2Earth Observation Research Center, Japan Aerospace Exploration Agency
July 28, 2011 IGARSS 2011 TH4.T09.2 1
Background
July 28, 2011 IGARSS 2011 TH4.T09.2 2
• Needs for automatic image interpretation– especially change detection– to handle large amount of data
Mudslides
Floods
?
• Humanlike interpretation requires:– high cognitive
ability– versatility
July 28, 2011 IGARSS 2011 TH4.T09.2 3
Solution
• Emulating manual interpretation using knowledge information processing
• We propose a framework for change detection – using ontology-based knowledge to recognize
and understand targets– input data: optical multispectral data
Knowledge
Mudslide
July 28, 2011 IGARSS 2011 TH4.T09.2 4
Framework for Change Detection
Day 1
Satellite Data
Inference Results
Day 2
Auxiliary Datae.g. DSM
Bayesian NetworkQuery for Targete.g. “mudslide”
Information ExtractionInformation Extraction
InferenceInference
Analysis of Target
Analysis of Target
Pixel-Based/Object-Based
Image Analysis
KnowledgeBased on Ontology
BayesianInference
Evidences
Synthesis of KnowledgeSynthesis of Knowledge
July 28, 2011 IGARSS 2011 TH4.T09.2 5
Requirements for Knowledge Representation
“Vegetation has high NDVI values”“Roads are long and narrow”
“Buildings are usually located along Road”“Artificial Forests are often located along River”
“Mountains are often covered by Forest”
Knowledge representation requires:•uncertainty•modularity and scalability •implicit structural definition of concepts
KnowledgeBased on Ontology
Remote Sensing Ontology
July 28, 2011 IGARSS 2011 TH4.T09.2 6
Heavyweightontology inremote sensing
– 420 concepts
Definition Structures•Inheritance (B is-a A)
• Slot (B part-of / attribute-of A)
A B
soilsoil
p/o 1..
leafleaf
chlorophyllcomponent
substancesubstance
Any
waterwater
chlorophyllchlorophyll
a/o 1
p/o 1..
density density
clustercluster
Anycomponentriverriver
fieldfield
slopeslope
continuant
entityp/o 1..
a/o 1
geographical objectgeographical object
component
structure structural attr.
Any
contextual changecontextual change
p/o 1.. subevent change event
p/o 0..
p/o 0.. Anybefore
Anyafter
superficial changesuperficial change
p/o 1.. component Any
geographical featuregeographical feature
p/o 1..
soil layersoil layer
soilcomponent
p/o 1..
water layerwater layer
watercomponent
woodwood
p/o 1..
trunktrunk
woodcomponent
p/o 1..
p/o 1..
treetree
leafcomponent
trunkcomponent
seasea
mountainmountainoccurrent
p/o 1
soil appearancesoil appearance
soil layerafter
water appearancewater appearance
p/o 1 water layerafter
a/o 1
p/o 1
location slope
mudslidemudslide
subevent soil appearance
substrate
p/o 1..
forestforest
treecomponent
change event
Main Categories
p/o 1 Slot 1 B
Slot 2 Ca/o 1
A
July 28, 2011 IGARSS 2011 TH4.T09.2 7
Knowledge Based on Ontology
• Describing relations among some concepts
• Using Bayesian probability to express uncertainty
(1) Concept-Slot Relation
(2) Concept-Evidence Relation
(3) Co-Occurrence2 Concepts 3 Concepts
BC
A
(2)
July 28, 2011 IGARSS 2011 TH4.T09.2 8
Analysis of Target & Synthesis of Knowledge
p/o 0
p/o 1 soil appearance
mudslide
soil layerbefore
a/o 1 location slope
p/o 1
p/o 1soil layerafter
p/o 1
subevent
soilcomponent
soilcomponent
Ontology
slope angle
slope
soil layer
soilsoil layer
soil appearance
soil layer
soil appearance
mudslide
slope
hue
soil
saturation
soil
value
soil
NDVI
soil
Knowledge
Bayesian Network
(1)
Day 2soil appearance
mudslide
slope angle
slope
Day 1 Auxiliary Data
soil layer
soil
huevalue
NDVI
saturation
soil layer
soil
huevalue
NDVI
saturation
(3)
July 28, 2011 IGARSS 2011 TH4.T09.2 9
Change Detection
Soil LayerImage ObjectSoil
HueValueSaturationNDVI
Satellite Image
Day 2soil appearance
mudslide
slope angle
slope
Day 1 Auxiliary Datasoil layer
soil
huevalue
NDVI
saturation
soil layer
soil
huevalue
NDVI
saturationSoil Layer
Soil Appearance
Day 2
Day 1
Calculate posterior probability of target using Bayesian network
Inference of Substance Inference of Object
Inference of Change
July 28, 2011 IGARSS 2011 TH4.T09.2 10
Experiment
To validate cognitive ability & versatility
Applying to three cases of practical change detection without tuning
Bi-temporal data• observed by AVNIR-2 onboard ALOS
3 visible + 1 near-infrared 10 m spatial resolution
• applied image registration with geometric errors of less than 0.5 pixel
July 28, 2011 IGARSS 2011 TH4.T09.2 11
Case 1: Detection of Mudslides in Yamaguchi City, Japan
Day 1 (14 June, 2009) Day 2 (30 July, 2009)
©JAXA ©JAXA
Mudslides caused by heavy rain in 19-26 July, 2009
July 28, 2011 IGARSS 2011 TH4.T09.2 12
Case 1: Detection of Mudslides in Yamaguchi City, Japan
- Inference Results -
Day 1 (14 June, 2009)
Day 2 (30 July, 2009)
©JAXA
©JAXA
soil on day 1
soil appearance mudslide
slope
Definition of mudslide
soil on day 2
p/o 0
p/o 1soil appearance
mudslide
soil layerbefore
a/o 1 location slope
p/o 1
p/o 1soil layerafter
p/o 1
subevent
soilcomponent
soilcomponent
July 28, 2011 IGARSS 2011 TH4.T09.2 13
Case 1: Detection of Mudslides in Yamaguchi City, Japan
- Comparison with Human’s Result -
small changes=> more sensitive than human’s result
changes in the flat area=> our definition of mudslide doesn’t include changes in flat area
July 28, 2011 IGARSS 2011 TH4.T09.2 14
Case 1: Mudslide Detection in Yamaguchi City, Japan
- Comparison with Survey Data -
Mudslides in Our ResultCollapsed SlopesMudflow TracesDebris
in Survey Data(investigated by Yamaguchi Pref.)
July 28, 2011 IGARSS 2011 TH4.T09.2 15
Case 2: Detection of Flooded Areas in Myanmar
©JAXA
©JAXA
Day 1 (4 May, 2008)
Day 2 (19 June, 2008)
Water (Day 1)
Water DisappearanceWater (Day 2)
Floods caused by Cyclone in 2-3 May, 2008
July 28, 2011 IGARSS 2011 TH4.T09.2 16
Case 2: Detection of Flooded Areas in Myanmar- Comparison with Human’s Result -
Our result not correctly detected due to the existence of clouds
Human’s result misdetected edges of clouds
July 28, 2011 IGARSS 2011 TH4.T09.2 17
Case 3: Detection of Flooded Areas in Pakistan
©JAXA
©JAXA
Day 1 (14 Oct., 2009)
Day 2 (1 Sept., 2010)
Water (Day 1)
Water AppearanceWater (Day 2)
Floods caused by heavy rain since late July, 2010
July 28, 2011 IGARSS 2011 TH4.T09.2 18
Discussion
• About 90% accuracy in mudslide detection• Our results were better than human’s results
due to using knowledge specialized on targets
• Fairly good results in all cases without tuning due to analyzing essential characteristics of
each targets using heavyweight ontology
• Possible to understand and recognize targets as humans do using rich ontology-based knowledge
July 28, 2011 IGARSS 2011 TH4.T09.2 19
Conclusions
• We proposed a framework for change detection – using ontology-based knowledge to recognize
and understand targets
• The experiment showed:– accuracy was about 90 % in mudslide detection– results were better than human’s results without
tuning
• More improvements are ongoing– to extract various information from data, such
as spatial information– to describe more expressive knowledge
July 28, 2011 IGARSS 2011 TH4.T09.2 20
Thank you!
