Distancesmarmakoide.org/download/teaching/dm/dm-distance.pdf · UNIVERSITY OF SCIENCE AND...

UNIVERSITY OF SCIENCE AND TECHNOLOGY OF CHINA SCHOOL OF SOFTWARE ENGINEERING OF USTC

DistancesMeasuring similarities

Devert AlexandreSchool of Software Engineering of USTC

December 7, 2012 — Slide 1/62


Table of Contents

1 Introduction

2 Strings

3 Data semantic

4 Perceptive models

Devert Alexandre (School of Software Engineering of USTC) — Distances — Slide 2/62


DistancesMany data-mining algorithms, like k-means, rely on adistance measure



Distances

The distance measure express how 2 data of the datasetrelates to each other



Distances

So far, we considered point in Rn and Euclidean distance.



Distances

But what looks like a distance measure for

• Text documents ?

• Sounds ?

• Shapes ?



Distances

Is the Euclidean good enough for all cases ?



Table of Contents

1 Introduction

2 Strings

3 Data semantic

4 Perceptive models



Binary strings

Let’s consider binary strings of fixed length

001010011101010100111100011111



Hamming distance

A convenient distance measure for strings is theHamming distance

s1 = 001010011101010100111100011111

s2 = 001010011101010000111100011111

d(s1, s2) = 1

Distance is the number of different digits



Hamming distance

A convenient distance measure for strings is theHamming distance

s1 = 001010011101010100111100011111

s2 = 001010001101000100110100011111

d(s1, s2) = 3

Distance is the number of different digits



Hamming distance

A way to compute it quickly, if you use integers

• Exclusive Or of the 2 chains

• Count the ones in the resulting chain



Fixed size strings

Hamming distance for other alphabets than {0, 1}

s1 = GATEAU

s2 = BATEAU

d(s1, s2) = 1



Fixed size strings

Hamming distance for other alphabets than {0, 1}

s1 = BIGLOTRON

s2 = BAFFOTRON

d(s1, s2) = 3



Variable length strings

But for many practical applications, we need to comparestrings with different lengths

CYCLOTRON

SYNCHROTRON

SYNCHROPHASOTRON

BIGLOTRON



Levenshtein distance

The Levenshtein distance is the minimum number ofedits needed to transform one string into the other

1 insertion of a character

2 deletion of a character

3 substitution of a character




Insertion of a character

FAT ⇒ FAST




Deletion of a character

FART ⇒ FAT




Substitution of a character

FAT ⇒ CAT




CYCLOTRON

SYCLOTRON substitution S ⇒ C

SYNCLOTRON insertion YC ⇒ YNC

SYNCHOTRON substitution L ⇒ H

SYNCHROTRON insertion HO ⇒ HRO

distance is 4, because of the 4 steps to turnCYCLOTRON in SYNCHROTRON



Computing Levenshtein distance

Initial step for s1 = CAAT and s2 = CAT

C A A T0 1 2 3 4

C 1A 2T 3

We will fill step by step a matrix, with the distance ofeach prefix of s1 and s2




Let’s fill the matrix

C A A T0 1 2 3 4

C 1 0A 2T 3

Prefixes C and C are the same, distance is 0




Let’s fill the matrix

C A A T0 1 2 3 4

C 1 0 1 2 3A 2T 3

Prefixes C and CA, CAA, CAAT have distance 1, 2, 3




C A A T0 1 2 3 4

C 1 0 1 2 3A 2 1T 3 2

Prefixes C and CA, CAT have distance 1, 2




C A A T0 1 2 3 4

C 1 0 1 2 3A 2 1 0 1 2T 3 2 1




C A A T0 1 2 3 4

C 1 0 1 2 3A 2 1 0 1 2T 3 2 1 1 1




C A A T0 1 2 3 4

C 1 0 1 2 3A 2 1 0 1 2T 3 2 1 1 1

Distance is 1 !




CUT and CAT

C U T0 1 2 3

C 1A 2T 3




CUT and CAT

C U T0 1 2 3

C 1 0 1 2A 2 1T 3 2




CUT and CAT

C U T0 1 2 3

C 1 0 1 2A 2 1 1 2T 3 2 1




CUT and CAT

C U T0 1 2 3

C 1 0 1 2A 2 1 1 2T 3 2 1 1




CUT and CAT

C U T0 1 2 3

C 1 0 1 2A 2 1 1 2T 3 2 1 1

Distance is 1 !




The general idea

x1 x2 . . . xn0 1 2 . . . n

y1 1y2 2...

... di ,jym m

di ,j is the distance between {x1, x2, . . . , xi} and{y1, y2, . . . , yj}




The general idea

x1 x2 . . . xn0 1 2 . . . n

y1 1y2 2 di−1,j−1...

... di ,jym m

If xi = yjdi ,j = di−1,j−1




The general idea

x1 x2 . . . xn0 1 2 . . . n

y1 1y2 2 di−1,j−1 di ,j−1...

... di−1,j di ,jym m

If xi 6= yj

di ,j = 1 + min(di ,j−1, di−1,j , di−1,j−1)




C Y C L O T R O N0 1 2 3 4 5 6 7 8 9

S 1Y 2N 3C 4H 5R 6O 7T 8R 9O 10N 11




C Y C L O T R O N0 1 2 3 4 5 6 7 8 9

S 1 1 2 3 4 5 6 7 8 9Y 2 2N 3 3C 4 3H 5 4R 6 5O 7 6T 8 7R 9 8O 10 9N 11 10




C Y C L O T R O N0 1 2 3 4 5 6 7 8 9

S 1 1 2 3 4 5 6 7 8 9Y 2 2 1 2 3 4 5 6 7 8N 3 3 2C 4 3 3H 5 4 4R 6 5 5O 7 6 6T 8 7 7R 9 8 8O 10 9 9N 11 10 10




C Y C L O T R O N0 1 2 3 4 5 6 7 8 9

S 1 1 2 3 4 5 6 7 8 9Y 2 2 1 2 3 4 5 6 7 8N 3 3 2 2 3 4 5 6 7 7C 4 3 3 2H 5 4 4 3R 6 5 5 4O 7 6 6 5T 8 7 7 6R 9 8 8 7O 10 9 9 8N 11 10 10 9




C Y C L O T R O N0 1 2 3 4 5 6 7 8 9

S 1 1 2 3 4 5 6 7 8 9Y 2 2 1 2 3 4 5 6 7 8N 3 3 2 2 3 4 5 6 7 7C 4 3 3 2 3 4 5 6 7 8H 5 4 4 3 3R 6 5 5 4 4O 7 6 6 5 5T 8 7 7 6 6R 9 8 8 7 7O 10 9 9 8 8N 11 10 10 9 9




C Y C L O T R O N0 1 2 3 4 5 6 7 8 9

S 1 1 2 3 4 5 6 7 8 9Y 2 2 1 2 3 4 5 6 7 8N 3 3 2 2 3 4 5 6 7 7C 4 3 3 2 3 4 5 6 7 8H 5 4 4 3 3 4 5 6 7 8R 6 5 5 4 4 4O 7 6 6 5 5 4T 8 7 7 6 6 5R 9 8 8 7 7 6O 10 9 9 8 8 7N 11 10 10 9 9 8




C Y C L O T R O N0 1 2 3 4 5 6 7 8 9

S 1 1 2 3 4 5 6 7 8 9Y 2 2 1 2 3 4 5 6 7 8N 3 3 2 2 3 4 5 6 7 7C 4 3 3 2 3 4 5 6 7 8H 5 4 4 3 3 4 5 6 7 8R 6 5 5 4 4 4 5 5 6 7O 7 6 6 5 5 4 5T 8 7 7 6 6 5 4R 9 8 8 7 7 6 5O 10 9 9 8 8 7 6N 11 10 10 9 9 8 7




C Y C L O T R O N0 1 2 3 4 5 6 7 8 9

S 1 1 2 3 4 5 6 7 8 9Y 2 2 1 2 3 4 5 6 7 8N 3 3 2 2 3 4 5 6 7 7C 4 3 3 2 3 4 5 6 7 8H 5 4 4 3 3 4 5 6 7 8R 6 5 5 4 4 4 5 5 6 7O 7 6 6 5 5 4 5 6 5 6T 8 7 7 6 6 5 4 5R 9 8 8 7 7 6 5 4O 10 9 9 8 8 7 6 5N 11 10 10 9 9 8 7 6




C Y C L O T R O N0 1 2 3 4 5 6 7 8 9

S 1 1 2 3 4 5 6 7 8 9Y 2 2 1 2 3 4 5 6 7 8N 3 3 2 2 3 4 5 6 7 7C 4 3 3 2 3 4 5 6 7 8H 5 4 4 3 3 4 5 6 7 8R 6 5 5 4 4 4 5 5 6 7O 7 6 6 5 5 4 5 6 5 6T 8 7 7 6 6 5 4 5 6 6R 9 8 8 7 7 6 5 4 5O 10 9 9 8 8 7 6 5 4N 11 10 10 9 9 8 7 6 5




C Y C L O T R O N0 1 2 3 4 5 6 7 8 9

S 1 1 2 3 4 5 6 7 8 9Y 2 2 1 2 3 4 5 6 7 8N 3 3 2 2 3 4 5 6 7 7C 4 3 3 2 3 4 5 6 7 8H 5 4 4 3 3 4 5 6 7 8R 6 5 5 4 4 4 5 5 6 7O 7 6 6 5 5 4 5 6 5 6T 8 7 7 6 6 5 4 5 6 6R 9 8 8 7 7 6 5 4 5 6O 10 9 9 8 8 7 6 5 4 5N 11 10 10 9 9 8 7 6 5 4




Recursive & lazy style

de f l e v ( a , b ) :i f not a :

r e t u r n l e n ( b )

i f not b :r e t u r n l e n ( a )

r e t u r n min ( l e v ( a [ 1 : ] , b [ 1 : ] ) + ( a [ 0 ] != b [ 0 ] ) , # s u b s t i t u t i o nl e v ( a [ 1 : ] , b ) + 1 , # d e l e t i o nl e v ( a , b [ 1 : ] ) + 1) # i n s e r t i o n

Most programming languages do not deal well with suchcode




Imperative & eager style

de f l e v ( s , t ) :s , t , d = ’ ’ + s , ’ ’ + t , { }

f o r i i n x range ( l e n ( s ) ) :d [ i , 0 ] = i

f o r j i n x range ( l e n ( t ) ) :d [ 0 , j ] = j

f o r j i n x range (1 , l e n ( t ) ) :f o r i i n range (1 , l e n ( s ) ) :

i f s [ i ] == t [ j ] :d [ i , j ] = d [ i −1, j −1]

e l s e :d [ i , j ] = min ( d [ i −1, j ] , d [ i , j −1] , d [ i −1, j −1]) + 1

r e t u r n d [ l e n ( s )−1, l e n ( t )−1]

Can be better, no need to store the complete matrix



Damerau–Levenshtein distance

The Damerau–Levenshtein distance is like theLevenshtein distance, with one more edit operation

1 insertion of a character

2 deletion of a character

3 substitution of a character

4 transposition of 2 adjacent characters



Computing Damerau–Levenshtein

distance

It works almost like the Levenshtein distance

x1 x2 . . . xn0 0 1 2 . . . n

0 0 1 2 . . . n

y1 1 1y2 2 2...

...... di ,j

ym m m

di ,j is the distance between {x1, x2, . . . , xi} and{y1, y2, . . . , yj}




distance

It works almost like the Levenshtein distance

x1 x2 . . . xn0 0 1 2 . . . n

0 0 1 2 . . . n

y1 1 1y2 2 2...

...... di ,j

ym m m

If xi = yjdi ,j = min(di−2,j−2, di−1,j−1)




distanceIt works almost like the Levenshtein distance

x1 x2 . . . xn0 0 1 2 . . . n

0 0 1 2 . . . n

y1 1 1y2 2 2...

...... di ,j

ym m m

If xi 6= yj

di ,j = 1 + min(di−2,j−2, di ,j−1, di−1,j , di−1,j−1)



Robustness

Damerau–Levenshtein distance will differentiates thefollowing strings

THOMAS ⇒ TOHMAS

THOMAS ⇒ THOMASS

THOMAS ⇒ TOMAS

They have a distance = 1 with the string THOMAS



Robustness

Those differences are likely due to some typing error !

THOMAS

TOHMAS

THOMASS

TOMAS

The Damerau–Levenshtein distance might make ouralgorithms sensible to noise, if used for typed things



Jaro distance

A distance specialized for names records : Jaro distance



Jaro distance

d(s1, s2) =1

3

(

m

|s1|+

m

|s2|+

m − t

m

)

• d(s1, s2) = 1 means exact match

• d(s1, s2) = 0 means no similarity

• m is the number of matching characters

• t is half the number of transpositions



Jaro distance

2 characters from s1 and s2 are matching if they areequal and not farther than

⌊

max(|s1|, |s2|)

2

⌋

− 1



Jaro distance

THOMAS

TOHMAS

6 matching characters, m = 6



Jaro distance

2 characters from s1 and s2 are transposed if they are not

equal and not farther than

⌊

max(|s1|, |s2|)

2

⌋

− 1

t is equal to half the number of transposed characters



Jaro distance

THOMAS

TOHMAS

Mismatched characters H/O and O/H , t = 22= 1



Jaro distance

Jaro distance for THOMAS and TOHMAS is13

(

66+ 6

6+ 6−1

6

)

= 0.944



Jaro distance

Computes which character are matching and notmatching

D I X O ND 1 0 0 0 0I 0 1 0 0 0C 0 0 0 0 0K 0 0 0 0 0S 0 0 0 0 0O 0 0 0 1 0N 0 0 0 0 1X 0 0 0 0 0

• |s1| = 5, |s2| = 8

• match window is 3

• m = 4, t = 0

• distance is13

(

m|s1|

+ m|s2|

+ m−tm

)

= 0.822



Jaro distance

Jaro distance is an example of how to deal with noise indata ⇒ distance that considers identical 2 strings likelyto be the same thing but with some typing errors



Distance between texts

A simple and popular way to measure distance betweenlarge texts for data-mining ⇒ bag of words

• Find a large list of common words L likely to be intext A and B .

• Build vectors X (A) and X (B), where X (A)i is thenumber of occurence of words Li .

• Distance is X (A).X (B)

Used for spam-filtering for instance



Table of Contents

1 Introduction

2 Strings

3 Data semantic

4 Perceptive models



UFO sightings data

Let’s consider geographical data : UFO sightings



UFO sightings data

The raw data looks like this

description ufolocation Austin, Texas, USAsight date 20020804shape circle

InfoChimp UFO dataset ⇒ 60000 entries !



UFO sightings data

Using string distances distance for the date string wouldbe silly

d(20020804, 20050804) = 1

d(20020804, 20020704) = 1

d(20020804, 20020814) = 1

One year, one month or ten days are not the same thing !



UFO sightings data

Using number difference for the date string would be silly

d(20020804, 20050321) = 18020483

Dates are not a single base 10 number !



UFO sightings data

We need to convert dates into single numbers

description ufolocation Austin, Texas, USAsight date 1028419200shape circle

We can convert them to UTC time (beware, many UFOsightings before 1970)



UFO sightings data

Using string distance for location names would be silly

d(SUZHOU , FUZHOU) = 1

d(HEFEI ,HEBEI ) = 1



UFO sightings data

We need to convert locations into positions

description ufolocation latitude 30.25location longitude 97.75sight date 1028419200shape circle

Geo–location services can convert this to geo–coordinates



UFO sightings data

We will consider just the location and the time

location latitude 30.25location longitude 97.75sight date 1028419200

Can we use Euclidean distance now, to cluster those data?



Spherical coordinatesThe locations are spherical coordinates

λ

φr

(r,φ,λ)

A



Spherical coordinates

spherical coordinates are angles, does not work like theusual cartesian coordinates



Spherical coordinates

Distance between 2 sphere coordinates (φa, λa), (φb, λb)is not the usual Euclidean distance

rarcos(sinφa sinφb + cosφa cosφb cos(λb − λa))

Use Vincenty formula to actually compute this



Table of Contents

1 Introduction

2 Strings

3 Data semantic

4 Perceptive models



Picture segmentation

Using kmeans algorithm, one can segment a picture intosimilar areas

Colors are [r , g , b] triplets, so we can use euclideandistance, right ?



Color perception

Electronic sensors record color informations as 3 signals

red R green G blue B



Color perceptionOthers color spaces, like YUV, separate luminance andchrominance

luminance Y’ chrominance U chrominance V



Color interpolation

Let’s interpolate 2 colors Ca and Cb, using different colorspaces

Cα = Ca + α(Cb − Ca), α ∈ [0, 1]



Color interpolation

Interpolating colors in different color spaces givesdifferent results

Some color spaces introduce new colors wheninterpolating from one to an other !



Color interpolation

Interpolating colors in different color spaces givesdifferent results

Some color spaces introduce new colors wheninterpolating from one to an other !



Color perception

When you choose to represent colors in a given colorspace, you choose

• which colors are alike and which colors are verydifferent

• a color ”neighbourhood”



YUV color space

YUV color space have interesting properties

• color interpolation in YUV looks perceptually morecorrect

• human eye is much more sensitive to luminance Y’

than chrominance UV



RGB and YUV color spaces

The transformation from RGB to YUV is linear

Y ′

U

V

=

0.299 0.587 0.114−0.14713 −0.28886 0.4360.615 −0.51499 −0.10001

R

G

B

R , G and B are in the [0, 1] range



RGB and YUV color spaces

RGB and YUV color spaces represents the same thing :colors. But

• RGB is the signal as it comes out of the sensors

• YUV takes in account human perception of color

Some color spaces like LAB are even better models forhuman color perception



Human perception

There are perceptive models for

• colors

• shape

• depth

• sounds

• speech

• motion

• . . .



Human perception

A correct data-mining approach can return completelymeaningless results, without a perceptive model

In speech recognition, it’s essential, just to obtain apractically usable system


Date post:	19-Jul-2020
Category:	Documents
Upload:	others
View:	1 times
Download:	0 times

Distancesmarmakoide.org/download/teaching/dm/dm-distance.pdf · UNIVERSITY OF SCIENCE AND...

Documents