Date post: | 21-Dec-2015 |
Category: |
Documents |
View: | 215 times |
Download: | 1 times |
Focused Reducts
Janusz A. Starzyk and Dale Nelson
What Do We Know?Major Assumption
ASSUMPTION:This is ALL we know
RealWorld
Model
19 18 16 84 38 124 69 60 55 36 10918 27 184 63 28 13 16 68 67 29 5911 30 47 185 25 31 31 29 52 101 426 19 76 151 34 64 50 26 19 83 465 37 44 223 12 53 237 28 51 36 27
15 20 80 153 11 48 254 90 97 88 3725 14 51 107 27 72 79 78 82 36 4221 30 211 134 36 109 159 110 48 68 9230 30 175 91 35 128 68 45 47 102 4715 8 66 95 45 175 116 48 142 114 4822 11 151 78 30 79 20 78 54 100 10115 8 138 103 40 67 57 32 53 24 6826 24 50 71 50 145 73 196 13 52 29
Sampled Data
Problem Size Dilemma
250 125 169 89 43 100 33 190251 255 110 200 50 83 56 150217 250 98 141 48 66 44 232108 78 105 181 34 33 5 141119 255 244 241 65 33 19 5078 222 212 58 109 38 86 25592 124 68 144 67 64 55 218
…1024
...1602
Rough Set Tutorial
• Difference between rough sets and fuzzy sets
• Labeling data
• Remove duplicates/ambiguities
• What is a core?
• What is a reduct?
Rough Sets vs Fuzzy Sets
Fuzzy Sets - How gray is the pixel
Rough Sets - How big is the pixel
ExampleSample HRR Data
Signal Target ID Range Bin 1 Range Bin 2 Range Bin 3 Range Bin 41 1 .680 .127 .121 .5162 1 .948 .272 .022 .4403 1 .821 .189 .139 .4804 2 .396 .680 .279 .2395 2 .512 .851 .184 .2906 2 .394 .281 .338 .5647 2 .775 .507 .006 .6178 2 .281 .359 .582 .7739 3 .113 .097 .451 .45010 3 .896 .327 .122 .927
ExampleLabel Data
Signal Target ID Range Bin 1 Range Bin 2 Range Bin 3 Range Bin 41 1 3 1 1 22 1 3 1 1 23 1 3 1 1 24 2 2 3 1 15 2 2 3 1 16 2 2 1 2 27 2 3 2 1 38 2 1 2 2 39 3 1 1 2 210 3 3 2 1 3
Label 1 < .25 .25 >= Label 2 <=.45 Label 3 > .45
Labeling can be different for different columns/attributes
Ranges can be different for different columns/attributes
Signal Target ID Range Bin 1 Range Bin 2 Range Bin 3 Range Bin 41 1 3 1 1 22 1 3 1 1 23 1 3 1 1 24 2 2 3 1 15 2 2 3 1 16 2 2 1 2 27 2 3 2 1 38 2 1 2 2 39 3 1 1 2 210 3 3 2 1 3
Remove Ambiguities & Duplicates
Equivalence Classes
E1={1, 2, 3} E2={4, 5} E3={6} E4={7} E5={8}
Signal Target ID Range Bin 1 Range Bin 2 Range Bin 3 Range Bin 41 1 3 1 1 22 1 3 1 1 23 1 3 1 1 24 2 2 3 1 15 2 2 3 1 16 2 2 1 2 27 2 1 2 2 38 3 1 1 2 2
Definitions
• Reduct - A reduct is a reduction of an information system which results in no loss of information (classification ability) by removing attributes (range bins). There may be one or many for a given information system)
• Core - A core is the set of attributes (range bins) which are common to all reducts.
Compute Core
Signal Target ID Range Bin 1 Range Bin 2 Range Bin 3 Range Bin 41 1 3 1 1 22 1 3 1 1 23 1 3 1 1 24 2 2 3 1 15 2 2 3 1 16 2 2 1 2 27 2 1 2 2 38 3 1 1 2 2
Signals 6 and 8 are ambiguous upon removal of Range Bin 1.Therefore, Range Bin 1 is part of core.
Core - The range bins common to ALL reducts - The most essential range bins without which signals cannot be classified
Compute Core
Signal Target ID Range Bin 1 Range Bin 2 Range Bin 3 Range Bin 41 1 3 1 1 22 1 3 1 1 23 1 3 1 1 24 2 2 3 1 15 2 2 3 1 16 2 2 1 2 27 2 1 2 2 38 3 1 1 2 2
No ambiguous signals therefore, Range Bin 2 is NOT part of core.
Compute Core
Signal Target ID Range Bin 1 Range Bin 2 Range Bin 3 Range Bin 41 1 3 1 1 22 1 3 1 1 23 1 3 1 1 24 2 2 3 1 15 2 2 3 1 16 2 2 1 2 27 2 1 2 2 38 3 1 1 2 2
No ambiguous signals therefore, Range Bin 3 is NOT part of core.
Compute Core
Signal Target ID Range Bin 1 Range Bin 2 Range Bin 3 Range Bin 41 1 3 1 1 22 1 3 1 1 23 1 3 1 1 24 2 2 3 1 15 2 2 3 1 16 2 2 1 2 27 2 1 2 2 38 3 1 1 2 2
No ambiguous signals therefore, Range Bin 4 is NOT part of core.
Signal Target ID Range Bin 1 Range Bin 2 Range Bin 3 Range Bin 41 1 3 1 1 22 1 3 1 1 23 1 3 1 1 24 2 2 3 1 15 2 2 3 1 16 2 2 1 2 27 2 1 2 2 38 3 1 1 2 2
Compute ReductsRange Bin 1 + Range Bin 2
Range Bin 1 and Range Bin 2 classify therefore, they belong to a reduct
Compute ReductsRange Bin 1 + Range Bin 3
Range Bin 1 and Range Bin 3 do not classify therefore, they do NOT belong to a reduct
Signal Target ID Range Bin 1 Range Bin 2 Range Bin 3 Range Bin 41 1 3 1 1 22 1 3 1 1 23 1 3 1 1 24 2 2 3 1 15 2 2 3 1 16 2 2 1 2 27 2 1 2 2 38 3 1 1 2 2
Compute ReductsRange Bin 1 + Range Bin 4
Range Bin 1 and Range Bin 4 classify therefore, they belong to a reduct
Signal Target ID Range Bin 1 Range Bin 2 Range Bin 3 Range Bin 41 1 3 1 1 22 1 3 1 1 23 1 3 1 1 24 2 2 3 1 15 2 2 3 1 16 2 2 1 2 27 2 1 2 2 38 3 1 1 2 2
Reduct Summary
• Range bins 1 and 2 are a reduct
– Sufficient to classify all signals
• Range bins 1 and 4 are a reduct
– Sufficient to classify all signals
• Range bins 1 and 3 are NOT a reduct
– Cannot distinguish target classes 2 and 3
• No need to try
– Range bins 1, 2, 3
– Range bins 1, 2, 4
Did You Notice?
• Calculating a reduct is time consuming!
• n = 29 value = 536,870,911
• We are interested in n 50
• This is a BIG NUMBER requiring a lot of time to compute reduct which is a f (# signals), too
n
k knk
n
1 )!(!
!
Why Haven’t Rough Sets Been Used Before?
The Procedure
• Normalize signal• Partition signal
– Block– Interleave
• Wavelet transform• Binary multi-class entropy labeling• Entropy based range bin selection• Determine minimal reducts• Fuse marginal reducts for classification
Data
• Synthetic generated by XPATCH
• Six targets– 1071 Signals per target– 128 Range bins/signal
– Azimuth -25o to +25o
– Elevation -20o to 0o
Normalize the Data
• Ensures all data is range normalized
• Use the 2 Norm
• Divide each signal bin value by N
2
1
2
iiyN
Partition the Signal
1 128
64 651 128
64 651 12832 33 96 97
64 651 12832 33 96 9716 17 48 48 80 81 112 113
1
21
1 2 3 4
4321 5 6 7 8
Block Partitioning
Partition the Signal
1 128
1 128
1 128
1 128
1
Interleave Partitioning
1st 2nd 3rd 4th 5th 6th 7th 8th
1 Piece
2 Pieces
4 Pieces
8 Pieces
Why Use a Wavelet Transform?
0 200 400 600 800 1000 12000
5
10
15
20
25
30
35
40
45
50Feature and Maximum Cluster Sizes
Feature Index
Cluster
Size
Original Signal
Best-20/60 signalsClassified
BestWavelet50/60 SignalsClassified!!
Many features are better than the best from original signal
HRR Signal and Its Haar Transform
Multi-Class Information Entropy
},,{ 1 nt xxu
}|{ tiixt uxxxC
t
xtxt u
CP
ixi
xjxii P
PPP
x
ixi
xjxii P
PPP
x
)1)(1(
1)1()1(6 kji
6
1
6
1
loglogj
jxjx
jjxjx PPPPE
36
1
36
1
loglogl
ll
lll xxxx
PPPP
Let xi be range bin values across all signals for a target class
Define
Without assuming any particular distribution we can define the probability as:
Using this definition we define two other probabilities
where
Then multi-class entropy is defined as:
Binary Multi-Class Labeling
-0.25 -0.2 -0.15 -0.1 -0.05 0 0.05 0.1 0.15 0.2 0.250
2
4
6
8
10
12
14
16
18
Range Bin Value
Target Gaussians Bin 910
target 1target 2target 3target 4target 5target 6
-0.1 -0.05 0 0.05 0.1 0.150
2
4
6
8
10
12
14
Range Bin Value
Target Gaussians Bin 1000
target 1target 2target 3target 4target 5target 6
0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 0.50
1
2
3
4
5
6
7
8
9
10
Range Bin Value
Target Gaussians Bin 54
target 1target 2target 3target 4target 5target 6
0.004 0.006 0.008 0.01 0.012 0.014 0.016 0.018 0.020
50
100
150
200
250
Range Bin Value
Target Gaussians Bin 130
target 1target 2target 3target 4target 5target 6
Range Bin Selection
• Total range bins available depends on partition size
• We chose 50 bins per reduct– Time considerations
– Implications
• Based on maximum relative entropy
Signal Size
Transformed Range Bins
128 102464 44832 19216 80
Signal Size
Bins in Classifier
128 5064 10032 20016 400
Compute Core
• Computation of core is easy and fast– Eliminate one range bin at a time and see if the
training set is ambiguous - only that range bin can discriminate between the ambiguous signals
– Accumulate the bins resulting in ambiguous data - that is the core
• These range bins MUST be in every reduct
• O(n) process
Compute Minimal Reducts
• To the core add one range bin at a time and compute the number of ambiguities
• Select the range bin(s) with the fewest ambiguities-there may be several-save these as we will use them to compute the reduct
• Add that range bin to the core and repeat previous step until there are no ambiguities - this is a reduct
• Calculate reducts for all bins with equivalent number of ambiguities-yields multiple reducts
• O(n2) process
Time Complexity
Training Set Size 50 to 400 Attributes (Range Bins) 1602 SignalsTest Set Size 4823 Signals
5 10 15 20 25 30 35 40 45 50
1000
2000
3000
4000
5000
6000
7000
8000
9000
10000Run Time
Number of Bins
Tim
e Need 50
Fuzzy Rough Set Classification
• Test signals may have a range bin value very close to labeling division point
• If this happens we define a distance where this is considered a “don’t care” region
• Classification process proceeds without the “don’t care” range bin
))max(,)min(min(* diid xxxxb
Weighting FormulaRequirements
• We desire the following for combining classifications– All Pcc(s) = 0 weight = 0– All Pcc(s) = 1 weight = 1– Several low Pcc(s) weight higher than
any of the Pcc(s)– One high Pcc and several low Pcc(s)
weight higher than the highest Pcc
Weighting Formula
n
i i
n
ii
t
Pcc
PccPccPcc
W
1
max1
max
11
11
1
Fusing Marginal Reducts
• Each signal is marked with the classification by each reduct along with the reduct’s performance (Pcc) on the training set
• A weight is computed for each target class for each signal
• A signal is assigned the target class with the highest weight
Results - Training
Results Testing
Conjectures
• Robust in the presence of noise– Due to binary labeling– Due to fuzzification
• Robust to signal registration– Due to binary labeling– Due to averaging effect of wavelets on interleaved
partitions– Due to fuzzification
5 10 15 20 25 30 35 40 45 50
1000
2000
3000
4000
5000
6000
7000
8000
9000
10000Run Time
Number of Bins
Tim
e
Rough Set Theoretic HRR ATR - Summary
0 1 TIME
METHOD
-Normalize Signal
-Partition Signal
- Block
- Interleave
-Wavelet Transform
-Binary Multi-class Entropy Labeling
-Entropy based Range Bin Selection
-Determine Minimal Reducts
-Fuse marginal reducts for classification
BREAKTHROUGHS-Reduct (classifier) generation time from exponential to quadratic !-Fusion of marginal (poor performing) reducts-Wavelet Transform Aiding-Multi partition to increase number of range bins considered-Use of binary multi-class entropy labeling-Entropy based range bin selection-Performance within 1% of theoretic best-Max problem size increased by 2 orders of magnitude
APPLICATIONS
-1-D Signals
-HRR
-LADAR vibration
-Sonar
-Medical
-Stock market
-Data Mining
Quadratic
Exponential
Future Directions
• Fuzz factor sensitivity study
• Sensitivity to signal alignment
• Sensitivity to noise
• Iterated wavelet transform performance study
• Effectiveness on air to ground targets
• Other application areas