Melbourne 2002
Heiko Schröder
Srikanthan ThambipillaiIan McLoughlin
Bertil SchmidtWu Jigang
Imrich VrtoOndrej Sykora
Tanja Vladimirova
Fault tolerant high performance computing on-board small satellites
Melbourne 2002
1000 nmRGB 2500 nm
Multispectral
Melbourne 2002
CHRIS
Multispectral
Melbourne 2002
Hyperspectral
Melbourne 2002
Hyperspectral
Melbourne 2002
Hyperspectral
Melbourne 2002
Fault tolerant On-board computing
10 km/s 1 image/s 100 Mbit/image 4000 s/orbit 400 Gbit/orbit download: 4 Gbit/orbit
On-board image analysis andcompression
800 km
Singapore100 x output if useful/useless<=1/100
100 x value
Melbourne 2002
Methods currently used
shadow-processorsmajorityvoting
Byzantine systems ASTRIUM, deep space
Melbourne 2002
1 CAN2 CANs •Industrial spec.
•mil-spec.•radiation tolerant•radiation hardened
386 is modern
Melbourne 2002
Our aim: High performance via COTS16 processors (+ spares) off-the-shelfconnected via afault tolerant reconfigurable network
In X-SAT restricted to image processing
Mesh/torus
Melbourne 2002
processorsfault
tolerantmesh
on-board
Melbourne 2002
switch
current communication
FPGA
ctrlh/vo/er/w
Instructionsto PEs
link to PE
Melbourne 2002
spares
C3 -- torus
spares
Replacement algorithm exists for up to 4 faults.Reconfiguration software runs on FPGA.Could be repaired within << 1sec.
Melbourne 2002
ctrlh/vo/er/w
Instructionsto PEs
Diagnosticset switches
Melbourne 2002
Available data (320 images) – search task
Oil slicks, forest fires, red tide, settlements, …Efficiency of the system: useful output / useful input <=1
Randomselection
E=Q=1/64
Output
Algorithms:•Compression•Classification•Segmentation
Melbourne 2002
Compressionratio (CR=4loss-less)
Segmentation gain (SG=16, 1/16 of a useful image is useful)
Classification gain(CG=5, 1 in 5 images contain useful information)
E=1/16Q*CR
E=1/64Q
E=5/16Q*CR*CG
E=5/64Q*CG
E=5/4Q*CG*SG
The satellite efficiency cube
Not likely
LOSSY=60E=5/2
E=5Q*CR*CG*SG
Melbourne 2002
1 2 3 4 5 6 7 8
9 10 11 12 13 14 15 16
17 18 19 20 21 22 23 24
25 26 27 28 29 30 31 32
33 34 35 36 37 38 39 40
41 42 43 44 45 46 47 48
49 50 51 52 53 54 55 56
57 58 59 60 61 62 63 64
LL1+2+3+4
HL1+3-2-4
LH1+2-3-4
HH1+4-2-3
1 2
3 4
LL HL
LH HH
LL HL
LH HH
LL HL
LH HH
LL HL
LH HH
LL HL
LH HH
LL HL
LH HH
LL HL
LH HH
LL HL
LH HH
LL HL
LH HH
LL HL
LH HH
LL HL
LH HH
LL HL
LH HH
LL HL
LH HH
LL HL
LH HH
LL HL
LH HH
LL HL
LH HH
LL LL
LL LL
LL LL
LL LL
LL LL
LL LL
LL LL
LL LL
LH LH
LH LH
LH LH
LH LH
LH LH
LH LH
LH LH
LH LH
HL HL
HL HL
HL HL
HL HL
HL HL
HL HL
HL HL
HL HL
HH HH
HH HH
HH HH
HH HH
HH HH
HH HH
HH HH
HH HH
1 2 3 4 5 6 7 89 10 11 12 13 14 15 16
17 18 19 20 21 22 23 2425 26 27 28 29 30 31 3233 34 35 36 37 38 39 4041 42 43 44 45 46 47 4849 50 51 52 53 54 55 5657 58 59 60 61 62 63 64
L1+2
H1-2
L3+4
H3-4
Invertible!+ /2 - /2
Melbourne 2002
33+34+41+4249+50+57+58
-35+36+43+4451+52+59+60
HL1 HL1 HL1 HL1
HL1 HL1 HL1 HL1
HL1 HL1 HL1 HL1
HL1 HL1 HL1 HL1
HH1HH1HH1HH1
HH1HH1HH1HH1
HH1HH1HH1HH1
HH1HH1HH1HH1
LH1 LH1 LH1 LH1
LH1 LH1 LH1 LH1
LH1 LH1 LH1 LH1
LH1 LH1 LH1 LH1
LL1 LL1 LL1 LL1
LL1 LL1 LL1 LL1
LL1 LL1 LL1 LL1
LL1 LL1 LL1 LL1
7+8+15+16
23+24+31+32
38+40+47+48
55+56+63+64
5+6+13+14
21+22+29+30
37+38+24+46
53+54+61+62
3+4+11+12
19+20+27+28
35+36+43+44
51+52+59+60
1+2+9+10
17+18+25+26
33+34+41+42
49+50+57+58
1 2 3 4 5 6 7 89 10 11 12 13 14 15 16
17 18 19 20 21 22 23 2425 26 27 28 29 30 31 3233 34 35 36 37 38 39 4041 42 43 44 45 46 47 4849 50 51 52 53 54 55 5657 58 59 60 61 62 63 64
LL2
LH2
LH2
LH2
LH2
HL2
HL2
HH2
HH2
HL2
HL2
HH2
HH2
LL2
LL2 LL2LH3
HL3
HH3
LL3
33+42-34-41
35+44-36-43
49+58-50-57
51+60-52-59
LL1+2+3+4
HL1+3-2-4
LH1+2-3-4
HH1+4-2-3
1 2
3 4
L1+2
H1-2
L3+4
H3-4
1+…+64
1..4,9..12,17-20,25-2833-36,41-44,49-52,57-60
-5-8,13-16,21-24,29-32
37-40,45-48,53-56,61-64
1+…+32-
33+…+64
1-4,9-12,17-20,25-2837-40,45-48,53-56,61-64
-5-8,13-16,21-24,29-32
33-36,41-44,49-52,57-60
Melbourne 2002
1+2+9+103+4+11+12
17+18+25+2619+20+27+28
5+6+13+147+8+15+16
21+22+29+3023+24+31+32
33+34+41+4235+36+43+4449+50+57+5851+52+59+60
37+38+45+4638+40+47+4853+54+61+6255+56+63+64
1+2+9+103+4+11+12
-17+18+25+2619+20+27+28
5+6+13+147+8+15+16
-21+22+29+3023+24+31+32
33+34+41+4235+36+43+44
-49+50+57+5851+52+59+60
37+38+24+4638+40+47+48
-53+54+61+6255+56+63+64
1+2+9+10+17+18+25+26
-3+4+11+12
19+20+27+28
5+6+13+1421+22+29+30
-7+8+15+16
23+24+31+32
33+34+41+4249+50+57+58
-35+36+43+4451+52+59+60
37+38+24+4653+54+61+62
-38+40+47+4855+56+63+64
1+2+9+1017+18+25+26
-3+4+11+12
19+20+27+28
5+6+13+1421+22+29+30
-7+8+15+16
23+24+31+32
33+34+41+4249+50+57+58
-35+36+43+4451+52+59+60
37+38+24+4653+54+61+62
-38+40+47+4855+56+63+64
LL2 HL2
LH2 HH2
Zero-tree
Melbourne 2002
Main ideas of zero tree encoding:•When the parent is small the children are small•If a root of a tree is smaller than a given threshold,
and all descendants are too,then only the root needs to be encoded
•Many values of the result of the wavelet transform are small,as they are differences of neighbors.
Melbourne 2002
Melbourne 2002
How to find areas of interestImage classificationIn real-time
?
Melbourne 2002
Thresholding
Melbourne 2002
Mathematical morphologyMathematical morphology
erosion
dilation
erosion
edge detection, thinning, noise removal, enlarging
Structural elementreference point
Melbourne 2002
ThresholdingMM-segmentation
Melbourne 2002
skeletonsskeletons
Histograms
Melbourne 2002
Red square skeletonRed square skeleton
1
1
0
3
0
0
0
1
6
Melbourne 2002
new = min{W,NW,N}+1
one-sweep algorithm to produce the red square skeleton:
Melbourne 2002
Rough SegmentationRough Segmentation
• Threshold
• Noise removal
• Red square
• frame
Melbourne 2002
MM-Hough TransformMM-Hough Transform
reference pointerosion
m d
d
m
a dot leads to one addition if there is a matching point
Melbourne 2002
• Higher contrast
• More flexibility– Lines of given thickness
– Dashed lines
– Lines of given length
– Lines of given orientation
– Other curves
Lines at 90 degrees
Melbourne 2002
Melbourne 2002
60 sec420km
1min420km
1min maneuver420km
30sec210km
Single image19 bands
Image sequence3 bands
130sec 19 bands910km
Image sequence19 bands
60 sec420km
Investigative mode
Melbourne 2002
60 sec420km
60 sec420 km
45 sec315 km
7 min2900 km3 bands
5 min1800 km19 bands
Follow coast3 bands
?Change band selection
Search mode
Melbourne 2002
2000 km
High-performance Computer network
Real-time image analysis•Classification•Segmentation•compression
Intelligent search
Maximize the efficiency of the satellite!
200 km
1 min
??
??
Thank you!
Melbourne 2002
… an arrary of SHARCs to provides throughput 160 Mb/s.… 2.5 billion floating point operations per second. … first demonstration of real-time image processing in space.
image cube froma 30 km wide swath of Korea’s coastline.(Launch: 2001?)
Nemo