Chapter 8-Learning (2)

8/12/2019 Chapter 8-Learning (2)

http://slidepdf.com/reader/full/chapter-8-learning-2 1/46

Chapter 8: Learning

By, Safa Hamdare



Learning

• Learning is essential for unknown environments,– i.e., when designer lacks omniscience

• Learning is useful as a system constructionmethod,– i.e., expose the agent to reality rather than trying to

write it down

• Learning modifies the agent's decisionmechanisms to improve performance



Learning agents



Learning Agent• Four Components

1. Performance Element: collection of knowledgeand procedures to decide on the next action.

E.g. walking, turning, drawing, etc.

2. Learning Element: takes in feedback from thecritic and modifies the performance elementaccordingly.

3. Critic: provides the learning element withinformation on how well the agent is doing based ona fixed performance standard.

E.g. the audience

4. Problem Generator: provides the performance

element with suggestions on new actions to take.



Designing Learning element

• Design of a learning element is affected by4 major issues:1. Which components of the performance

element to improve2. The representation of those

components3. Available feedback

4. Prior knowledge





Types of Inductive Learning1. Supervised learning: Inputs and Outputs

available. For every input, the learner is provided witha target; that is, the environment tells the learner what itsresponse should

2. Unsupervised learning: no hint of correctoutcome. the learner receives no feedback from the

world at all– just examples (e.g. – the same figures asabove , without the labels)

3. Reinforcement learning: evaluation of action. thelearner receives feedback about the appropriateness of itsresponse. i.e. occasional rewards

M M MF F F





Inductive Learning

• Key idea:– To use specific examples to reach

general conclusions

• Given a set of examples, the systemtries to approximate the evaluationfunction.

• Also called Pure Inductive Inference .



Recognizing Handwritten Digits

Learning Agent

Training Examples



Bi



Bias• Bias: Any preference for one hypothesis

over another, beyond mere consistencywith the examples.

• Since there are almost always a large

number of possible consistent hypotheses,all learning algorithms exhibit some sortof bias.

• Example:



Formal Definition for Inductive Learning

• Simplest form: learn a function from

examples• Example: a pair (x, f(x)), where

– x is the input,

– f(x) is the output of the function /target functionapplied to x.

• hypothesis: a function “h” that

approximates f, given a set of examples.• Task of induction: Given a set of examples,

find a function h that approximates the true

evaluation function f.



Inductive learning - Example 1

• f (x) is the target function

• An example is a pair [x, f (x)]

• Learning task: find a hypothesis h such that h(x) f (x) given a

training set of examples D = {[xi, f (xi) ]}, i = 1,2,…, N

1)(,

0

0

1

0

10

1

1

1

xx f

1)(,

0

0

1

1

10

0

1

1

xx f

1)(,

0

1

0

0

10

1

1

1

xx f

Etc...



Inductive learning method

• Construct/adjust h to agree with f on training set• (h is consistent if it agrees with f on all examples

• E.g., curve fitting:




• Construct/adjust h to agree with f on training set

• (h is consistent if it agrees with f on all examples)





• Construct/adjust h to agree with f on training set

• (h is consistent if it agrees with f on all examples)






Inductive learning method• Construct/adjust h to agree with f on training set

• (h is consistent if it agrees with f on all examples)• E.g., curve fitting:

How do we choose from among multiple consistenthypothesis?

• Ockham’s razor: prefer the simplest hypothesis

consistent with data



Th R t t D i



The Restaurant Domain Attributes Goal

Example Fri Hun Pat Price Rain Res Type Est WillWait

X1 No Yes Some $$$ No Yes French 0-10 YesX2 No Yes Full $ No No Thai 30-60 No

X3 No No Some $ No No Burger 0-10 Yes

X4 Yes Yes Full $ No No Thai 10-30 Yes

X5 Yes No Full $$$ No Yes French >60 NoX6 No Yes Some $$ Yes Yes Italian 0-10 Yes

X7 No No None $ Yes No Burger 0-10 No

X8 No Yes Some $$ Yes Yes Thai 0-10 Yes

X9 Yes No Full $ Yes No Burger >60 No

X10 Yes Yes Full $$$ No Yes Italian 10-30 No

X11 No No None $ No No Thai 0-10 No

X12 Yes Yes Full $ No No Burger 30-60 Yes

Will we wait, or not?



S li i E l b T i A ib ( ’ )



Splitting Examples by Testing on Attributes (con’t)

+ X1, X3, X4, X6, X8, X12 (Positive examples) - X2, X5,

X7, X9, X10, X11 (Negative examples)

+ X1, X3, X4, X6, X8, X12 (Positive ex)

- X2, X5, X7, X9, X10, X11 (Negative ex)



Splitting Examples by Testing onAttributes (con’t)

+ X1, X3, X4, X6, X8, X12 (Positive examples) - X2, X5,

X7, X9, X10, X11 (Negative examples)

Patrons?

+- X7, X11

nonesome

full

+X1, X3, X6, X8- +X4, X12- X2, X5, X9, X10

No Yes

l E l b b ( ’ )



Splitting Examples by Testing on Attributes (con’t)

+ X1, X3, X4, X6, X8, X12 (Positive examples) - X2, X5,X7, X9, X10, X11 (Negative examples)

Yes No



Decision tree learning example

Induced tree (from examples)

D l l




True tree

Goal Predicate:

Will wait for a table?



Patrons?

WaitEst?

Hungry?

Yes

nonesome

full

>6030-60 10-30

0-10

noyes

Logical Representation of a Path

r [Patrons(r, full) Wait_Estimate(r, 10-30) Hungry(r, yes)] Will_Wait(r)

Ch i ib



Choosing an attribute• Idea: a good attribute splits the examples into

subsets that are (ideally) "all positive" or "allnegative"

• Patrons? is a better choice

Wh M k G d A ib ?



Patrons?

+

- X7, X11

nonesome

full

+X1, X3, X6, X8

-

+X4, X12

- X2, X5, X9, X10

Type?

+ X1

- X5

French

Italian Thai

+X6

- X10

+X3, X12

- X7, X9

+ X4,X8

- X2, X11

Burger

What Makes a Good Attribute?

Better

Attribute

Not As

Good AnAttribute






16

3ln6

36

3ln6

312

6

63ln

63

63ln

63

12

6Entropy

Alternate?

3 T, 3 F 3 T, 3 F

Yes No

Entropy decrease for Alternate= 1 – 1= 0






98.07

3ln7

37

4ln7

412

7

53ln

53

52ln

52

12

5Entropy

Fri

2 T, 3 F 4 T, 3 F

Yes No

Entropy decrease for Fri/Sat= 1 – 0.98 = 0.02




804.05

4ln5

45

1ln5

112

5

72ln

72

75ln

75

12

7Entropy

Hungry?

5 T, 2 F 1 T, 4 F

Yes No

Entropy decrease for Hungry= 1 – 0.804 = 0.19




18

4ln8

48

4ln8

412

8

42ln

42

42ln

42

12

4Entropy

Raining?

2 T, 2 F 4 T, 4 F

Yes No

Entropy decrease for Raining= 1 – 1 = 0




978.07

4ln7

47

3ln7

312

7

52ln

52

53ln

53

12

5Entropy

Reservation?

3 T, 2 F 3 T, 4 F

Yes No

Entropy decrease for Reservation = 1 – 0.978 = 0.02




456.06

4ln6

46

2ln6

212

6

40ln

40

44ln

44

12

4

22ln

22

20ln

20

12

2Entropy

Patrons?

2 F

4 T

None Full

Entropy decrease for Patrons= 1 – 0.456 = 0.543

2 T, 4 F

Some




77.04

3ln4

34

1ln4

112

4

20ln

20

22ln

22

12

2

63ln

63

63ln

63

12

6Entropy

Price

3 T, 3 F

2 T

$ $$$

Entropy decrease for Price = 1 – 0.77 = 0.23

1 T, 3 F

$$




14

2ln4

24

2ln4

212

4

42ln

42

42ln

42

12

42

1ln

2

1

2

1ln

2

1

12

2

2

1ln

2

1

2

1ln

2

1

12

2Entropy

Type

1 T, 1 F

1 T, 1 F

French Burger

Entropy decrease for Type= 1 – 1 = 0

2 T, 2 FItalian

2 T, 2 F

Thai




792.02

2ln2

22

0ln2

012

2

21ln

21

21ln

21

12

22

1ln

2

1

2

1ln

2

1

12

2

6

2ln

6

2

6

4ln

6

4

12

6Entropy

Est. waitingtime

4 T, 2 F

1 T, 1 F

0-10 > 60

Entropy decrease for Est = 1 – 0.792 = 0.21

2 F10-30

1 T, 1 F

30-60



Entropy for each AttributeEntropy decrease for Alternate= 1 – 1= 0

Entropy decrease for Bar= 1 – 1= 0

Entropy decrease for Fri/Sat= 1 – 0.98 = 0.02

Entropy decrease for Hungry= 1 – 0.804 = 0.19

Entropy decrease for Raining= 1 – 1 = 0

Entropy decrease for Reservation = 1 – 0.978 = 0.02

Entropy decrease for Patrons= 1 – 0.456 = 0.543

Entropy decrease for Price = 1 – 0.77 = 0.23

Entropy decrease for Est = 1 – 0.792 = 0.21




Patrons?

2 F

4 T

None Full

Largest entropy decrease (0.543)achieved by splitting on Patrons.

2 T, 4 F

Some

X? Continue like this, making new splits,

always purifying nodes.

Next step



Next stepGiven Patrons as root node, the next attribute chosen is

Hungry?

33.02

2ln2

22

0ln2

012

24

2ln4

24

2ln4

212

4Entropy

Entropy decrease for Hungry= 1 – 0.33= 0.666

Decision tree learning



Decision tree learning• Aim: find a small tree consistent with the training

examples

• Idea: (recursively) choose "most significant" attributeas root of (sub)tree.



Date post:	03-Jun-2018
Category:	Documents
Upload:	divya-shetty
View:	225 times
Download:	0 times

Chapter 8-Learning (2)

Documents