Randomforest Q

8/10/2019 Randomforest Q

1/41

Random Forest

MUSA AL-HAWAMDAH / 12812900101115-10-2012


2/41

What is random

forestsAn ensem !e "!assi#er $sin%man& de"ision tree mode!s' (an e $sed for "!assi#"ation orRe%ression' A""$ra"& and )aria !eim*ortan"e information is*ro)ided +ith the res$!ts '


3/41

R,n% %.$ hi n !

%Cng c phn lp s dng nhiu mhnh cy quyt nh.

C th c s dng phn lph!y h"i quy.

#$ ch%nh &'c () *in thu$c t%nhd c cung c+p (i ,t qu-


4/41

3he A!%orithm /t th/ num*/0 12 t0!ining c!s/s */ 34 !nd th/ num*/0 12 (!0i!*l/sin th/ cl!ssi5/0 */ 6. 7h/ num*/0 m 12 input (!0i!*l/s !0/ us/d t1 d/t/0min/ th/

d/cisi1n !t ! n1d/ 12 th/ t0//8 m sh1uld */ much l/ss th!n 6.

Ch11s/ ! t0!ining s/t 210 this t0// *y ch11sing 3 tim/s 9ith0/pl!c/m/nt 201m !ll 3 !(!il!*l/ t0!ining c!s/s. :s/ th/ 0/st 12 th/c!s/s t1 /stim!t/ th/ /0010 12 th/ t0//4 *y p0/dicting th/i0 cl!ss/s.

;10 /!ch n1d/ 12 th/ t0//4 0!nd1mly ch11s/ m (!0i!*l/s 1n 9hicht1 *!s/ th/ d/cisi1n !t th!t n1d/. C!lcul!t/ th/ */st split *!s/d 1nth/s/ m (!0i!*l/s in th/ t0!ining s/t.


5/41

3h$4t 3o nCho tp hun luyn vi N trng hp , v s lng cc thuc tnh tvic phn lo i l !"

Cho # l $i%n &'u vo &c s( )*ng &+ c &-nh cc .uy%t &-nh t

n/t c01 cy, # n2n &c t h3n nhi4u so vi !"Ch=n 0! > m?u *11tst0!p t@ tAp hu+n luyBn c s n 3. D

dng phEn cFn lGi cH! c'c t0Ing h p c t%nh lJi cH! cy.

5i #6i #7u $ootstr1p, t i #6i n/t c01 cy, ng7u nhi2n ch8n cc $i%thuc tnh # &+ l# c3 s9 .uy%t &-nh 9 n/t &:" ;nh ton v phn chi1nht )


6/41

Random forest a!%orithm67o+"hart


7/41

S: ;< 3h$4t 3o n


8/41

Ad)anta%es Kt p01duc/s ! highly !ccu0!t/ cl!ssi5/0 !nd l/!0ning is 2!st

Kt 0uns /Lci/ntly 1n l!0g/ d!t! *!s/s.

Kt c!n h!ndl/ th1us!nds 12 input (!0i!*l/s 9ith1ut(!0i!*l/ d/l/ti1n.

Kt c1mput/s p01&imiti/s */t9//n p!i0s 12 c!s/s th!t

c!n */ us/d in clust/0ing4 l1c!ting 1utli/0s 10 M*y sc!lingNgi(/ int/0/sting (i/9s 12 th/ d!t!.

Kt 1O/0s !n /&p/0im/nt!l m/th1d 210 d/t/cting(!0i!*l/ int/0!cti1ns.


9/41

=$ ;i>m 7G1 0! m$t phn lp ch%nh &'c c!1 () h=c tAp l) nh!nh

ChGy hiBu qu- t0Pn cQ sR dS liBu ln.

C th & lT h)ng ng)n *in sU Eu ()1 m) ,hng &! *in.

7%nh tQng qu!n giS! c'c cVp t0Ing h p c th c sdng t01ng g1m nhm4 c lng gi tr-! 0! qu!n im thW( cH! dS liBu.

Cung c+p m$t phQng ph'p thXc nghiBm ph't hiBn tQngt'c *in.


10/41

Y Z!nd1m ;10/st is ! cl!ssi5/0 c1nsisting 12 !c1ll/cti1n 12 t0//[st0uctu0/d cl!ssi5/0s \hM&4 ? , N4 , ] ^...._9h/0/ th/ ? , !0/ind/p/nd/ntly4 id/ntic!lly dist0i*ut/d 0!nd1m

t0//s !nd /!ch t0// c!sts ! unit (1t/ 210 th/5n!l cl!ssi5c!ti1n 12 input &. i,/ CYZ74Z!nd1m ;10/st us/s th/ gini ind/& 210d/t/0mining th/ 5n!l cl!ss in /!ch t0//.

7h/ 5n!l cl!ss 12 /!ch t0// is !gg0/g!t/d !nd(1t/d *y 9/ight/d (!lu/s t1 c1nst0uct th/5n!l cl!ssi5/0.


11/41

6$t 0@ng ng?u nhiPn l) m$t phn lp *!1 g"mm$t tAp h p c'c phn lp c+u t0Wc cy \h M&4`,N4 , ] ^ ...._ t01ng `, l) $c lAp4 cyng?u nhiPn () c cang phn *U &'c &u+t ()mJi cy tQng bng ch1 lp ph *in nh+t cH!Eu ()1 &.iUng nh CYZ74 Z!nd1m ;10/st s dng c'c

che sU ini &'c nh lp cuUi cang t01ng mJi

cy. C'c lp h=c cuUi cang cH! mJi cy c t ngh p () *nh ch=n *Ri gi' t0 t0=ng sU &ydXng c'c phn lp ch%nh thbc.


12/41

@ini ndeBZ!nd1m ;10/st us/s th/ gini ind/& t!,/n 201m th/CYZ7 l/!0ning syst/m t1 c1nst0uct d/cisi1n t0//s.

7h/ gini ind/& 12 n1d/ impu0ity is th/ m/!su0/ m1st

c1mm1nly ch1s/n 210 cl!ssi5c!ti1n[typ/ p01*l/ms. K2! d!t!s/t 7 c1nt!ins /&!mpl/s 201m n cl!ss/s4

gini ind/&4 iniM7N is d/5n/d !sf

9h/0/ p is th/ 0/l!ti(/ 20/qu/ncy 12 cl!ss in 7.


13/41

(hC S @iniZ@ng ng?u nhiPn s dng c'c che sU ini l+yt@ CYZ7 &y dXng cy quyt nh. iBc t%nhche sU ini cH! nWt l) *iBn ph'p ph *in nh+t c lX! ch=n ch1 (+n phn lp. 3u m$ttAp dS liBu 7 c n lp4 che sU gini4 ini M7N cnh ngh ! l)f

i p l) tEn sU tQng Ui cH! lp t01ng 7 .


14/41

K2 ! d!t!s/t 7 is split int1 t91 su*s/ts 7^ !nd 7j 9ith sik/s 3^ !nd 3j 0/sp/cti(/ly4 th/ giniind/& 12 th/ split d!t! c1nt!ins /&!mpl/s 201mn cl!ss/s4 th/ gini ind/& M7N is d/5n/d !sf

7h/ !tt0i*ut/ (!lu/ th!t p01(id/s th/ sm!ll/st D K7 ini M7N isch1s/n t1 split th/ n1d/.


15/41

3u m$t tAp dS liBu 7 c chi! th)nh h!i tApc1n 7^ () 7j c ,%ch thc 3^ () 3j tQngbng4 che sU ini cH! dS liBu chi! chb! n lp 4che sU gini M7N c nh ngh ! l)f

i' t0 thu$c t%nh cung c+p c'c D K7 ini nh nh+t M7N c ch=n phn chi! c'c nWt.


16/41

E*eration of Random

Forest 7h/ 910,ing 12 0!nd1m 210/st !lg10ithm is !s 21ll19s.^. Y 0!nd1m s//d is ch1s/n 9hich pulls 1ut !t

0!nd1m ! c1ll/cti1n 12 s!mpl/s 201m th/ t0!iningd!t!s/t 9hil/ m!int!ining th/ cl!ss dist0i*uti1n.

j. oith this s/l/ct/d d!t! s/t4 ! 0!nd1m s/t 12!tt0i*ut/s 201m th/ 10igin!l d!t! s/t is ch1s/n *!s/d1n us/0 d/5n/d (!lu/s. Yll th/ input (!0i!*l/s !0/n1t c1nsid/0/d */c!us/ 12 /n10m1us c1mput!ti1n!nd high ch!nc/s 12 1(/0 5tting.


17/41

; t tGnh ( a R,n% n%.$ nhi nC'c Vc t%nh cH! thuAt t1'n 0@ng ng?u nhiPn l) nhs!u.

^. 7hu$c t%nh phn lp c ch=n ng?u nhiPn t@ c'cm?u c l+y t@ tAp hu+n luyBn t01ng ,hi (?n duy t0sX phn lp

j. i m?u ch=n n)y4 m$t tAp h p ng?u nhiPn c'cthu$c t%nh t@ tAp dS liBu *!n Eu c lX! ch=n dX!t0Pn ngIi dang &'c nh gi' t0. 7+t c- c'c *in Eu()1 ,hng cEn &/m & t *Ri ( t%nh t1'n 0+t ln () cQh$i c!1 hQn pha h p.


18/41

r. Kn ! d!t!s/t 9h/0/ 6 is th/ t1t!l num*/0 12input !tt0i*ut/s in th/ d!t!s/t4 1nly Z!tt0i*ut/s !0/ ch1s/n !t 0!nd1m 210 /!ch t0//9h/0/ Z 6.

. 7h/ !tt0i*ut/s 201m this s/t c0/!t/s th/ */stp1ssi*l/ split using th/ gini ind/& t1 d/(/l1p !

d/cisi1n t0// m1d/l. 7h/ p01c/ss 0/p/!ts 210/!ch 12 th/ *0!nch/s until th/ t/0min!ti1nc1nditi1n st!ting th!t l/!(/s !0/ th/ n1d/sth!t !0/ t11 sm!ll t1 split.


19/41

r. 701ng m$t tAp dS liBu4 6 l) t ng sU Eu ()1thu$c t%nh t01ng c'c tAp dS liBu4 che c c'cthu$c t%nh Z c ch=n ng?u nhiPn ch1 mJicy4 t01ng Z 6.

. C'c thu$c t%nh t@ tAp n)y4 c tG1 0! sXphn chi! tUt nh+t c th * ng c'ch s dngche sU gini ph't t0in m$t m hnh cyquyt nh. vu' t0nh n)y lVp i lVp lGi ch1mJi nh'nh ch1 n ,hi c'c iu ,iBn ch+m dbt,hi c'c nWt qu' nh chi!.


20/41


21/41


22/41


23/41

i- s 0 ng c'c thu$c t%nh Eu tiPn c chi! l)thu$c t%nh w6


24/41


25/41

hn (ang s!u ,hi chi! nh phn t0Pn

w6


26/41


27/41


28/41

701ng *c tip th/14 c'c dS liBu Vt Rw6


29/41


30/41

lAp *-ng gi' t0 che sU gini ch1 thu$c t%nhw6


31/41

t!*ul!t/s th/ gini ind/& (!lu/ 210 th/w6


32/41

Kn Z!nd1m ;10/st4 th/ split !t 9hichth/ gini ind/& is l19/st is ch1s/n !t

th/ split (!lu/.

19/(/04 sinc/ th/ (!lu/s 12 th/w6


33/41

701ng Z!nd1m ;10/st4 imphn chi! l) im c gi' t0

phn chi! l) che sU gini l) th+pnh+t. 7uy nhiPn4 nu gi' t0 cH! c'cthu$c t%nh w6


34/41


35/41

7hH tc n)y c lVp i lVp lGi ch1 c'c thu$ct%nh cFn lGi t01ng tAp dS liBu.

701ng (% d n)y4 c'c gi' t0 che sU ini cH!3 thu$c t%nh thb h!i c t%nh t1'n.

i' t0 th+p nh+t cH! che sU gini c ch=n l)chi! tUt nh+t ch1 c'c thu$c t%nh.

C'c cy quyt nh cuUi cang th hiBn f


36/41


37/41

7h/ d/cisi1n 0ul/s 210 th/ d/cisi1n t0//

illust0!t/d !0/f

K2 w6


38/41

C'c luAt ch1 cy quyt nh minh h=! l)f

K2 w6


39/41

7his is ! singl/ t0// c1nst0ucti1n using th/CYZ7 !lg10ithm. Z!nd1m 210/s 21ll19s thiss!m/ m/th1d1l1gy !nd c1nst0ucts multipl/t0//s 210 th/ 210/st using diO/0/nt s/ts 12!tt0i*ut/s. Z!nd1m 210/st h!s us/s ! p!0t 12th/ t0!ining s/t t1 c!lcul!t/ th/ m1d/l /00100!t/ *y !n in*uilt /0010 /stim!t/4 th/ 1ut[12[*!g MwwN /0010 /stim!t/.


40/41

#y l) m$t &y dXng cy Qn s dng thuAtt1'n CYZ7. Z@ng ng?u nhiPn giUng nh m$tphQng ph'p ph%! s!u () &y dXng ch1 nhiucy s dng thu$c t%nh ,h'c nh!u. Z@ng ng?unhiPn s dng m$t phEn cH! tAp hu+nluyBn lAp t%nh t1'n t lB lJi cH! m hnhc t%nh lJi in*uilt4 1ut[12[tWi MwwN c t%nhlJi.


41/41

Date post:	02-Jun-2018
Category:	Documents
Upload:	edogawaconan90
View:	217 times
Download:	0 times

Randomforest Q

Documents