data mining slides

8/9/2019 data mining slides

1/43

Data MiningPractical Machine Learning Tools and Techniques

Slides for Chapter 3 of Data Miningby I. H. Witten !. "ran# andM. $. Hall


2/43

2Data Mining: Practical Machine Learning Tools and Techniques (Chapter 3)

Output: Knowledge representation

Tables

Linear odels

Trees

!ules

Classi"ication rules

#ssociation rules

!ules with e$ceptions

More e$pressi%e rules

&nstance'based representation

Clusters


3/43


Output: representing structural patterns

Man di""erent was o" representing patterns

Decision trees rules instance'based *

#lso called +,nowledge- representation

!epresentation deterines in"erence ethod

.nderstanding the output is the ,e to understanding theunderling learning ethods

Di""erent tpes o" output "or di""erent learning probles

(e/g/ classi"ication regression *)


4/434Data Mining: Practical Machine Learning Tools and Techniques (Chapter 3)

Tables

0iplest wa o" representing output:

.se the sae "orat as input1

Decision table "or the weather proble:

Main proble: selecting the right attributes



Linear odels

#nother siple representation

!egression odel&nputs (attribute %alues) and output are all

nueric

Output is the su o" weighted attribute %alues

The tric, is to "ind good %alues "or the weights



# linear regression "unction "or the CP.

per"orance data

PRP = 37.06 + 2.47CACH


7/43


inar classi"ication

Lineseparates the two classes

Decision boundar ' de"ines where the decision changes"ro one class %alue to the other

Prediction is ade b plugging in obser%ed %alues o" the

attributes into the e$pression

Predict one class i" output and the other class i" output 4

oundar becoes a high'diensional plane

(hyperplane) when there are ultiple attributes

Linear odels "or classi"ication


8/43


0eparating setosas "ro %ersicolors

2.0 0.5PETAL-LENGTH 0.8PETAL-WIDTH = 0


9/43


10/43


5oinal and nueric attributes

5oinal:nuber o" children usuall equal to nuber %alues

attribute won6t get tested ore than once

Other possibilit: di%ision into two subsets

5ueric:test whether %alue is greater or less than constant attribute a get tested se%eral ties

Other possibilit: three'wa split (or ulti'wa split)

&nteger: less than, equal to, greater than

!eal: below, within, above


11/43


Missing %alues

Does absence o" %alue ha%e soe signi"icance7

8es +issing- is a separate %alue

5o +issing- ust be treated in a special wa

assign instance to ost popular branch


12/43


Trees "or nueric prediction

Regression: the process o" coputing an e$pression

that predicts a nueric quantit

Regression tree: +decision tree- where each lea"

predicts a nueric quantit

Predicted %alue is a%erage %alue o" traininginstances that reach the lea"

Model tree: +regression tree- with linear regressionodels at the lea" nodes

Linear patches appro$iate continuous "unction


13/43


Linear regression "or the CP. data

PRP =

- 56.1

+ 0.049 MYCT

+ 0.015 MMIN

+ 0.006 MMAX

+ 0.630 CACH- 0.270 CHMIN

+ 1.46 CHMAX


14/43


!egression tree "or the CP. data


15/43


Model tree "or the CP. data


16/43


Classi"ication rules

Popular alternati%e to decision trees

Antecedent(pre'condition): a series o" tests (9ust li,e the

tests at the nodes o" a decision tree)Tests are usuall logicall #5Ded together (but a also

be general logical e$pressions)

Consequent(conclusion): classes set o" classes or

probabilit distribution assigned b rule

&ndi%idual rules are o"ten logicall O!ed togetherCon"licts arise i" di""erent conclusions appl


17/43


ro trees to rules

;as: con%erting a tree into a set o" rules

One rule "or each lea":

#ntecedent contains a condition "or e%er node on the path "ro the root to thelea"

Consequent is class assigned b the lea"

Produces rules that are unabiguousDoesn6t atter in which order the are e$ecuted

ut: resulting rules are unnecessaril cople$

Pruning to reo%e redundant tests


18/43


ro rules to trees

More di""icult: trans"oring a rule set into a tree

Tree cannot easil e$press dis9unction between rules

;$aple: rules which test di""erent attributes

0etr needs to be bro,en

Corresponding tree contains identical subtrees( +replicated subtree proble-)

If a and b !"n #

If $ and d !"n #


19/43


# tree "or a siple dis9unction


20/43


The e$clusi%e'or proble

If # = % and & = 0!"n $'a(( = a

If # = 0 and & = %!"n $'a(( = a

If # = 0 and & = 0

!"n $'a(( = b

If # = % and & = %!"n $'a(( = b


21/43


# tree with a replicated subtree

If # = % and & = %!"n $'a(( = a

If ) = % and * = %

!"n $'a(( = a

!",*(" $'a(( = b

+ - " , l d


22/43


+5uggets- o" ,nowledge

#re rules independent pieces o" ,nowledge7 (&t sees eas

to add a rule to an e$isting rule base/)Proble: ignores how rules are e$ecuted

Two was o" e$ecuting a rule set:

Ordered set o" rules (+decision list-)Order is iportant "or interpretation

.nordered set o" rules

!ules a o%erlap and lead to di""erent conclusions "or the sae instance

& i l


23/43


&nterpreting rules

=hat i" two or ore rules con"lict7

>i%e no conclusion at all7>o with rule that is ost popular on training data7

*

=hat i" no rule applies to a test instance7>i%e no conclusion at all7

>o with class that is ost "requent in training data7

*

0 i l b l l


24/43


0pecial case: boolean class

#ssuption: i" instance does not belong to class +es- it

belongs to class +no-Tric,: onl learn rules "or class +es- and use de"ault

rule "or +no-

If # = % and & = % !"n $'a(( = a

If ) = % and * = % !"n $'a(( = a

!",*(" $'a(( = b

# i ti l


25/43


#ssociation rules

#ssociation rules*

* can predict an attribute and cobinations o"

attributes

* are not intended to be used together as a set

Proble: iense nuber o" possible associations

Output needs to be restricted to show onl the ost

predicti%e associations

onl those with highsupport and high confidence

0 t d "id " l


26/43


0upport and con"idence o" a rule

0upport (Co%erage): nuber o" instances predicted correctlCon"idence (#ccurac) : nuber o" correct predictions as

proportion o" all instances that rule applies to

;$aple: 4cool das with noral huidit

0upport ? 4 con"idence ? 1@

5orall: iniu support and con"idence pre'speci"ied

(e/g/ 58rules with support 2 and con"idence 95@ "orweather data)

If "/",a," = $11' !"n !d& = n1,a'

&nterpreting association rules


27/43


&nterpreting association rules

&nterpretation is not ob%ious:

is notthe sae as

&t eans that the "ollowing also holds:

If *nd& = fa'(" and /'a& = n1 !"n 1'11 = (nn&

and !d& = !!

If *nd& = fa'(" and /'a& = n1 !"n 1'11 = (nn&If *nd& = fa'(" and /'a& = n1 !"n !d& = !!

If !d& = !! and *nd& = fa'(" and /'a& = n1

!"n 1'11 = (nn&



28/43



&dea: allow rules to ha%e exceptions

;$aple: rule "or iris data

5ew instance:

Modi"ied rule:

0.%

Petal

&idth

%.'

Petal

length

Iris(setosa3.)).*

TypeSepal

&idth

Sepal

length

If /"a'-'"n! 2.45 and /"a'-'"n! 4.45 !"n I,(-",($1'1,

If /"a'-'"n! 2.45 and /"a'-'"n! 4.45 !"n I,(-",($1'1,

ECEPT f /"a'-*d! %.0 !"n I,(-("1(a

# ore cople$ e$aple


29/43


# ore cople$ e$aple

;$ceptions to e$ceptions to e$ceptions *

d"fa' I,(-("1(a

"#$"/ f /"a'-'"n! 2.45 and /"a'-'"n! 5.355

and /"a'-*d! %.75

!"n I,(-",($1'1,

"#$"/ f /"a'-'"n! 4.5 and /"a'-*d! %.55

!"n I,(-,n$a

"'(" f ("/a'-'"n! 4.5 and ("/a'-*d! 2.45

!"n I,(-,n$a

"'(" f /"a'-'"n! 3.35

!"n I,(-,n$a

"#$"/ f /"a'-'"n! 4.85 and ("/a'-'"n! 5.5

!"n I,(-",($1'1,

#d%antages o" using e$ceptions


30/43


#d%antages o" using e$ceptions

!ules can be updated increentall

;as to incorporate new data

;as to incorporate doain ,nowledge

People o"ten thin, in ters o" e$ceptions

;ach conclusion can be considered 9ust in the

conte$t o" rules and e$ceptions that lead to it

Localit propert is iportant "or understandinglarge rule sets

+5oral- rule sets don6t o""er this ad%antage

More on e$ceptions


31/43


More on e$ceptions

Default...exceptif...then...

is logicall equi%alent to

if...then...else

(where the else speci"ies what the de"ault did)

ut: e$ceptions o""er a pschological ad%antage#ssuption: de"aults and tests earl on appl ore

widel than e$ceptions "urther down

;$ceptions re"lect special cases

!ules in%ol%ing relations


32/43


!ules in%ol%ing relations

0o "ar: all rules in%ol%ed coparing an attribute'%alue to a

constant (e/g/ teperature 4 45)

These rules are called +propositional- because the ha%ethe sae e$pressi%e power as propositional logic

=hat i" proble in%ol%es relationships between e$aples

(e/g/ "ail tree proble "ro abo%e)7Can6t be e$pressed with propositional rules

More e$pressi%e representation required

The shapes proble


33/43


The shapes proble

Target concept:standing up

0haded:standing.nshaded: lying

# propositional solution


34/43


# propositional solution

If *d! 3.5 and !"! 7.0!"n '&n

If !"! 3.5 !"n (andn

# relational solution


35/43


# relational solution

Coparing attributes with each other

>eneraliAes better to new data

0tandard relations: ? 4 B

ut: learning relational rules is costl

0iple solution: add e$tra attributes

(e/g/ a binar attribute is width < height?)

If *d! 9 !"! !"n '&n

If !"! 9 *d! !"n (andn

!ules with %ariables


36/43


!ules with %ariables.sing %ariables and ultiple relations:

The top o" a tower o" bloc,s is standing:

The whole tower is standing:

!ecursi%e de"inition1

If !"!:and:*d!:1f;#

and (:,"(:1f;#!"n (andn;#

&nducti%e logic prograing


37/43


&nducti%e logic prograing

!ecursi%e de"inition can be seen as logic progra

Techniques "or learning logic progras ste "ro the areao" +inducti%e logic prograing- (&LP)

ut: recursi%e de"initions are hard to learn

#lso: "ew practical probles require recursionThus: an &LP techniques are restricted to non'recursi%e

de"initions to a,e learning easier

&nstance'based representation


38/43


&nstance based representation

0iplest "or o" learning: rote learning

Training instances are searched "or instance that

ost closel resebles new instanceThe instances thesel%es represent the ,nowledge

#lso called instance-basedlearning0iilarit "unction de"ines what6s +learned-

&nstance'based learning is lazy learning

Methods: nearest-neighbor, k-nearest-neighbor,


39/43

Learning prototpes


40/43


g p p

Onl those instances in%ol%ed in a decision need

to be stored

5ois instances should be "iltered out

&dea: onl useprototypicale$aples

!ectangular generaliAations


41/43


g g

5earest'neighbor rule is used outside rectangles

!ectangles are rules1 (ut the can be ore conser%ati%ethan +noral- rules/)

5ested rectangles are rules with e$ceptions

!epresenting clusters &


42/43


p g

Simple 2-D representation Venn diagram

O%erlapping clusters

!epresenting clusters &&


43/43


p g

1 2 3

a /4 /1 /5b /1 /8 /1

c /3 /3 /4

d /1 /1 /8

e /4 /2 /4" /1 /4 /5g /7 /2 /1

h /5 /4 /1

*

Probabilistic assignment Dendrogram

5: dendron is the >ree,word "or tree

Date post:	01-Jun-2018
Category:	Documents
Upload:	abdul-samad
View:	217 times
Download:	0 times

data mining slides

Documents