Rem
ote
Sen
sin
g T
ech
nolo
gy I
nsti
tute
KIM Validation for EO Archived Data
Exploitation Support (KIMV)
Mihai DatcuDLR Oberpfaffenhofen
Rem
ote
Sen
sin
g T
ech
nolo
gy I
nsti
tute
1997 first tests (ETH Zurich) what’s that? ¼ scene
1999 MMDEMO (ETH Zurich) I2M exists and works! 10 scenes
2002 KIM (ESA, DLR, ETHZ, NREC, EUSC) system, appl. Use 20 GB
2003/4 use KIM in projects: SMART, PRESENCE
2003/4 Information Theory for evaluation.
2005 KIMV: operational system, bugs, enhancements accuracy, application scenarios…1 TB, 1m – 1Km, optical and SAR….
….more TB…more users…more sensors TerraSAR
Rem
ote
Sen
sin
g T
ech
nolo
gy I
nsti
tute
KIM/KES system concept
•Knowledge-driven Information Mining (KIM)
•Knowledge enabled services (KES)
•KIM and KES are based on Human Centred Concepts
•Implements improved feature extraction
search on a semantic levelavailability of collected knowledgeinteractive knowledge discoveryshare knowledgenew visual user interfaces
Rem
ote
Sen
sin
g T
ech
nolo
gy I
nsti
tute
KIM and KES systems
1. A library of algorithms which is used to extract the primitive features
2. A machine learning (Bayesian network) algorithm to generate interactively image classifications
3. A data base management system for the image content information catalogue and semantics and knowledge
The systems are helping the user in his analytical task to extract the information; the system records the knowledge and can reuse or communicate it. In addition, KIM and KES adapt to the user conjecture and are designed to operate very fast on large image volumes.
Rem
ote
Sen
sin
g T
ech
nolo
gy I
nsti
tute
System complexity
KIM/KES integrate
natural languagetextnumerical recordsGIS spatial data representationdatabase and visual capabilitiesanalysis of multidimensional pictorial
structurescomputer vision pattern recognition relational data modelsknowledge representation and bases
Rem
ote
Sen
sin
g T
ech
nolo
gy I
nsti
tute
System complexity
Important complexity factors
is the unbalanced ratio between the huge information volume of EO data (i.e. enormous image archives) and the sequential, mainly linguistic, and limited capacity of people to access information
perception of information as ``signals-signs-symbols'' is generally not dependent on the form in which the information is presented but rather on the context in which it is perceived, i.e. upon the intentions and expectations of the perceiver.
Rem
ote
Sen
sin
g T
ech
nolo
gy I
nsti
tute
Validation procedure
objective evaluation of system performance
relevance in real applications with users in the loop, i.e. validation from the “subjective” perspective of the users interested in specific data and applications.
Rem
ote
Sen
sin
g T
ech
nolo
gy I
nsti
tute
The expert evaluators
KIM/KES systems respond to existing and new requirements of a very broad range of applications
aerospace agencies (ESA, CNES, NASA, DLR) satellite centres (EUSC, ARCS) universities and research unitsindustry data providers
Rem
ote
Sen
sin
g T
ech
nolo
gy I
nsti
tute
The tasks
access to information in large EO archivesimage interpretationunderstanding phenomenatarget or objects detectioninformation mining and knowledge discovery
Rem
ote
Sen
sin
g T
ech
nolo
gy I
nsti
tute
The data sets
Sensor Collection TilesERS ers_GEC 571ERS + LANDSAT Mozambique 448HYPERSPECTRAL Presence 4HYPERSPECTRAL/R
Smart 24IKONOS Ikonos 207IKONOS ikonos_geo 651LANDSAT Switzerland 184LANDSAT 5/7 land_IT 468LANDSAT 5/7 ls_urbex 468LANDSAT 7 landsat7 168MERIS meris_120 1099MERIS MerSelectFrame 1039MIXED Nepal 188SPOT 5 cnes_eval 423SPOT 5 cnes_spot50 45SPOT 5 cnes_spotM 45SPOT 5 geo_spot 9SPOT 5 mihai_spot 9
Rem
ote
Sen
sin
g T
ech
nolo
gy I
nsti
tute
The operation modes
Content Based Image Retrieval (CBIR)
CBIR is based on utilization of semantic queriesCBIR enables an operator to “see” into a large volume
Data/Information mining
explore the information content of the images probabilistic image retrieval integrated with interactive learning and image classification
Scene understanding
derive knowledge, interpret or understand the structures and objects
Rem
ote
Sen
sin
g T
ech
nolo
gy I
nsti
tute
The questionnaire
evaluation for information retrieval systems, manmachine communication and image classification
rank the user satisfaction on scale with 4 qualitative
values (very good, good, acceptable, uncertain)
semantic differentials for questionnaire-based system
validation (for characterization of the task, searchprocess, retrieved result and system behaviour)
evaluation of the man-machine communication ( extent
of system functionalities, effects on the user, specific
system like items, and general score)
guideline for a general assessment of the validation
results and suggestions.
Rem
ote
Sen
sin
g T
ech
nolo
gy I
nsti
tute
CBIR results analysis
28%
40%
18%
14%
Very Good
Good
Acceptable
Uncertain
Rem
ote
Sen
sin
g T
ech
nolo
gy I
nsti
tute
I2M results analysis
12%
23%
31%
34%
Very Good
Good
Acceptable
Uncertain
Rem
ote
Sen
sin
g T
ech
nolo
gy I
nsti
tute
SU/Classification results analysis
41%
32%
18%
9%
Very Good
Good
Acceptable
Uncertain
Rem
ote
Sen
sin
g T
ech
nolo
gy I
nsti
tute
Man-Machine Communication
21%
54%
22%
3%
Very Good
Good
Acceptable
Uncertain
Rem
ote
Sen
sin
g T
ech
nolo
gy I
nsti
tute
MERIS: the classification
Rem
ote
Sen
sin
g T
ech
nolo
gy I
nsti
tute
The method
Rem
ote
Sen
sin
g T
ech
nolo
gy I
nsti
tute
Results
l2_flags classification
cloud water land
cloud 96% 0,6% 3,4%
water
0,1% 99,9% 0%
land 20,3% 0,5% 79,2%
Rem
ote
Sen
sin
g T
ech
nolo
gy I
nsti
tute
Results
cloud_type
cloud water
cloud
98% 2%
water
0% 100%
Rem
ote
Sen
sin
g T
ech
nolo
gy I
nsti
tute
Results
Meris Level 2 Product
Cloud Water Land
Cloud 97,3% 1,2% 4,3%
Water 3,8% 96,2% 0%
Land 19,1% 0,3% 80,7%
Rem
ote
Sen
sin
g T
ech
nolo
gy I
nsti
tute
Conclusions cloud and water the classification given by
training the system is more than 90% similar to level 2 product in most of the cases.
land classification is not as similar as for cloud
and water, this is due to two facts: land could be covered by cloud
land is a very general concept ice classification a big difference is detected.
level 2 product classification is considering ice over the water and for as it is a classification of snow.
snow classification: level 2 product is including the snow in cloud class, meanwhile KIM can separate snow and cloud as two different classes
Rem
ote
Sen
sin
g T
ech
nolo
gy I
nsti
tute
Feature constancy (data models)
Gemetry (HR vs. LR)
SPOT (CNES) data quality
# semantic labels grows with higherresolution
MERIS Landsat ERS SPOT IKONOS
10 100 10 k100 1000