Post on 27-Dec-2015
transcript
Lectures 5,6
MACHINE LEARNING
EXPERT SYSTEMS
Contents
Machine learning Knowledge representation Expert systems
INDUCTION OF DECISION TREES FROM DATA
Sunny Rain
Overcast
Outlook
Humidity Wind
High Normal Strong Weak
Outlook Temp. Humid. Wind Sport?
1 Sunny Hot High Weak No
2 Sunny Hot High Strong No
3 Overcast Hot High Weak Yes
4 Rain Mild High Weak Yes
5 Rain Cold Normal Weak Yes
6 Rain Cold Normal Strong No
7 Overcast Cold Normal Strong Yes
8 Sunny Mild High Weak No
9 Sunny Cold Normal Weak Yes
10 Rain Mild Normal Weak Yes
11 Sunny Mild Normal Strong Yes
12 Overcast Mild High Strong Yes
13 Overcast Hot Normal Weak Yes
14 Rain Mild High Strong No
Decision trees
Sport=No Sport=Yes
Data from credit history of loan applications
A simplified tree… But how to do it?
The induction algorithm ID3
Partially constructed decision trees
STEP 1
STEP 2
A heuristic problem
HOW TO SELECT THE BEST PROPERTY?
Approximate trees
High Normal
Humidity
85%Outlook
75%100%
Sunny Not Sunny
Outlook Temp. Humid. Wind Sport?
1 Sunny Hot High Weak No
2 Sunny Hot High Strong No
3 Overcast Hot High Weak Yes
4 Rain Mild High Weak Yes
5 Rain Cold Normal Weak Yes
6 Rain Cold Normal Strong No
7 Overcast Cold Normal Strong Yes
8 Sunny Mild High Weak No
9 Sunny Cold Normal Weak Yes
10 Rain Mild Normal Weak Yes
11 Sunny Mild Normal Strong Yes
12 Overcast Mild High Strong Yes
13 Overcast Hot Normal Weak Yes
14 Rain Mild High Strong No
CLASSIFICATION SYSTEMS
A full classification system
Pattern recognition
Patterns:– images, personal records, driving habits, etc.
Representation:– vector of features (inputs to a neural network)
Pattern classification: – Classify a pattern to one of the given classes
> classifier < Marks
> classifier < not Marks
> classifier < not Marks
> classifier < not Marks
> classifier < Marks
> classifier < not Marks
Classifier training
Classifier application
> Classifier > Marks
Note: The test image does not appear in the training data
LEARNING IN GENERAL
The data and the goals
We begin with a collection of positive (and usually negative) examples of a target class (a concept to be learnt)
The goal is to infer a general definition that will allow the learner to recognize future instances of the class
Knowledge representation
Positive and negative examples can be represented, e.g., in predicate calculus
Two positive instances of the concept of “ball” can be expressed as follows:size(obj1,small) color(obj1,red) shape(obj1,round) size(obj2,large) color(obj2,red) shape(obj2,round)
The general concept of “ball” could be defined by:
size(X,Y) color(X,Z) shape(X,round)
where any sentence that unifies with this general definition represents a ball
A general model of the learning
process
A set of operations
Given a set of training instances, the learner must construct a generalization, heuristic rule or plan that satisfies its goals
The concept space
Representation language and the operations define a space of potential concept definitions
The learner must search this space to find the desired concept
Heuristic search
Learning programs must commit to a direction and order of search, as well as…
…to the use of available data and heuristics to search efficiently
PATRICK WINSTON’S PROGRAM ON LEARNING CONCEPTS
Examples and near misses for the concept “arch”
Generalization of descriptions to include multiple examples (I)
Generalization of descriptions to include multiple examples (II)
Specialization of a description to exclude a near miss
so that this can’t match
Starting with the original
we add special constraints
A BRIEF HISTORY OF AI REPRESENTATIONAL SCHEMES
Semantic network developed by Collins & Quillian
in their research on human information
storage and response times
Network representation of properties of…
…snow and ice
…three definitions of the word “plant”
Three planes representing…
Intersection path between “cry” and “comfort” (Quillian 1967)
Case frame representation of the sentence “Sarah fixed thechair with glue.”
Conceptual dependency theory of four primitive conceptualizations
For example, all actions are assumed to reduce to one or more of the primitive ACTs listed below:
“John ate the egg”
“John prevented Mary from giving a book to
Bill”
Restaurant script
(Schank and Abelson 1977)
Restaurant script (continued)
FRAMES
A frame includes:
Frame identification information Its relationship to other frames Descriptors of requirements Procedural information on use
of the structure described Frame default information New instance information
Relationship to other frames
For instance, the “hotel phone” might be a special instance of “phone”, which might be an instance of a “communication device”
Descriptors of requirements
For instance, a chair has its seat between 20 and 40 cm from the floor, its back higher than 60 cm, etc.
These requirements may be used to determine when new objects fir the stereotype defined by the frame
Procedural information
An important feature of frames is the ability to attach procedural code to a slot
Frame default information
These are slot values that are taken to be true when no evidence to the contrary has been found
For instance, chairs have four legs, telephones are pushbutton, hotel beds are made by the staff
New instance information
Many frame slots may be left unspecified until given a value for a particular instance or when they are needed for some aspect of problem solving
For instance, the color of the bedspread may be left unspecified
Part of a frame description of a hotel room
“Specialization” indicates a pointer to a superclass
Spatial frame for viewing a cube (Minsky 1975)
CONCEPTUAL GRAPHS: A NETWORK LANGUAGE
Conceptual relations of
different arities
“Mary gave John the book”
“A dog named emma is brown”
Examples of restriction…
…join, and simplify operations
Inheritance in conceptual graphs
“Tom believes that Jane likes pizza”
This example shows the use of a propositional concept
RULE BASED EXPERT SYSTEMS
Architecture of a typical expert system for a particular problem
domain
Guidelines to determine whether a problem is appropriate for expert system solution (1)
The need for the solution justifies the cost and effort of building an expert system
Human expertise is not available in all situations where it is needed
The problem may be solved using symbolic reasoning
Guidelines to determine whether a problem is appropriate for expert system solution (2)
The problem domain is well structured and does not require commonsense reasoning
The problem may not be solved using traditional computing methods
Cooperative and articulate experts exist The problem is of proper size and scope
Reasoning with a typical expert system
The role of mental or conceptual models in problem solving
A small system for analysis of automotive problems
The and/or graph searched in the car diagnosis example
The production system at the start of a consultation in the car diagnostic example
Imagine that we want to get
information about spark plugs
The production system after Rule 1 has fired
The system after Rule 4 has fired
Note the stack-based approach to goal reduction
The following dialogue begins with the computer asking the user about the goals present in working memory
CASE BASED REASONING
Case based reasoners
Share a common structure For each new problem they
– Retrieve appropriate cases from memory– Modify a retrieved case so that it will apply
to the current situation– Apply the transformed case– Save the solution, with a record of success
or failure, for future use
Possible preference heuristics to help organize the storage and retrieval of cases (1) Goal-directed: Organize cases, at least in
part, by goal descriptions; Retrieve cases that have the same goal as the current situation
Salient-feature: Prefer cases that match the most important features or those matching the largest number of important features
Recency: Prefer cases used most recently
Possible preference heuristics to help organize the storage and retrieval of cases (2) Specify: Look for as exact as possible
matches of features before considering more general matches
Frequency: Check first the most frequently matched cases
Ease of adaptation: Use first cases most easily adapted to the current situation
Transformational analogy
Advantages of rule based approach
Disadvantages
Advantages of case based reasoning
Disadvantages
Combination of rule based and case based systems
3 Robotic Planning
ROBOTIC PLANNING
The blocks world
Predicate based representation
A number of truth relations or rules for robot’s performance
Portion of the search space
Goal state for the blocks world
Descriptor triples
Preconditions (P): Conditions the world must meet for an operator to be applied
Add List (A): Additions to the state description that are a result of applying the operator
Delete List (D): Items that are removed from a state description to create the new state when the operator is applied
Operators as triples of descriptions
A simple tree showing condition action rules
THANK YOU