Chapter 4 Chapter 4 Informed Search and Informed Search and Exploration Exploration
Transcript
Chapter 4 Informed Search and Exploration
Outline
Informed (Heuristic) search strategies
(Greedy) Best-first search
A* search
(Admissible) Heuristic Functions
Relaxed problem
Subproblem
Local search algorithms
Hill-climbing search
Simulated anneal search
Local beam search
Genetic algorithms
Online search *
Online local search
learning in online search
Informed search strategies
Informed search
uses problem-specific knowledge beyond the problem
definition
finds solution more efficiently than the uninformed search
Best-first search
uses an evaluation function f ( n ) for each node
e.g., Measures distance to the goal lowest evaluation
Implementation :
fringe is a queue sorted in increasing order of f -values.
Can we really expand the best node first?
No! only the one that appears to be best based on f ( n ).
heuristic function h ( n )
estimated cost of the cheapest path from node n to a goal
node
Specific algorithms
greedy best-first search
A* search
Greedy best-first search
expand the node that is closest to the goal
: Straight line distance heuristic
Greedy best-first search example
Properties of Greedy best-first search
Complete ?
Optimal ?
Time ?
Space ?
No No can get stuck in loops, e.g., Iasi > Neamt > Iasi >
Neamt Yes complete in finite states with repeated-state checking ,
but a good heuristic function can give dramatic improvement keeps
all nodes in memory
A* search
evaluation function f ( n ) = g ( n ) + h ( n )
g ( n ) = cost to reach the node
h ( n ) = estimated cost to the goal from n
f ( n ) = estimated total cost of path through n to the
goal
an admissible (optimistic) heuristic
never overestimates the cost to reach the goal
estimates the cost of solving the problem is less than it
actually is
e.g., never overestimates the actual road distances
A* using Tree-Search is optimal if h ( n ) is admissible
could get suboptimal solutions using Graph-Search
might discard the optimal path to a repeated state if it is not
the first one generated
a simple solution is to discard the more expensive of any two
paths found to the same node (extra memory)
: Straight line distance heuristic
A* search example
Optimality of A*
Consistency (monotonicity)
n is any successor of n, general triangle inequality ( n , n ,
and the goal)
consistent heuristic is also admissible
A* using Graph-Search is optimal if h ( n ) is consistent
the values of f ( n ) along any path are nondecreasing
Properties of A*
Suppose C * is the cost of the optimal solution path
A* expands all nodes with f ( n ) < C *
A* might expand some of nodes with f ( n ) = C * on the goal
contour
A* will expand no nodes with f ( n ) > C *, which are pruned
!
Pruning : eliminating possibilities from consideration without
examination
A* is optimally efficient for any given heuristic function
no other optimal algorithm is guaranteed to expand fewer nodes
than A*
an algorithm might miss the optimal solution if it does not
expand all nodes with f ( n ) < C *
A* is complete
Time complexity
exponential number of nodes within the goal contour
Space complexity
keeps all generated nodes in memory
runs out of space long before runs out of time
Memory-bounded heuristic search
Iterative-deepening A* (IDA*)
uses f -value ( g + h ) as the cutoff
Recursive best-first search (RBFS)
replaces the f -value of each node along the path with the best
f -value of its children
remembers the f -value of the best leaf in the forgotten
subtree so that it can reexpand it later if necessary
is efficient than IDA* but generates excessive nodes
changes mind : go back to pick up the second-best path due to
the extension ( f -value increased) of current best path
optimal if h ( n ) is admissible
space complexity is O ( bd )
time complexity depends on the accuracy of h ( n ) and how
often the current best path is changed
Exponential time complexity of Both IDA* and RBFS
cannot check repeated states other than those on the current
path when search on Graphs Should have used more memory (to store
the nodes visited)!
: Straight line distance heuristic
RBFS example
Memory-bounded heuristic search (contd)
SMA* Simplified MA* (Memory-bounded A*)
expands the best leaf node until memory is full
then drops the worst leaf node the one has the highest f
-value
regenerates the subtree only when all other paths have been
shown to look worse than the path it has forgotten
complete and optimal if there is a solution reachable
might be the best general-purpose algorithm for finding optimal
solutions
If there is no way to balance the trade off between time an
memory, drop the optimality requirement !
(Admissible) Heuristic Functions h 1 ? h 2 ? = the number of
misplaced tiles = total Manhattan (city block) distance = 7 tiles
are out of position = 4+0+3+3+1+0+2+1 = 14
Effect of heuristic accuracy
Effective branching factor b*
total # of nodes generated by A* is N , the solution depth is
d
b* is b that a uniform tree of depth d containing N +1 nodes
would have
well-designed heuristic would have a value close to 1
h 2 is better than h 1 based on the b*
Domination
h 2 dominates h 1 if for any node n
A* using h 2 will never expand more nodes than A* using h
1
every node n with will be expanded
the larger the better, as long as it does not
overestimate!
Inventing admissible heuristic functions
h 1 and h 2 are solutions to relaxed (simplified) version of
the puzzle.
If the rules of the 8-puzze are relaxed so that a tie can move
anywhere , then h 1 gives the shortest solution
If the rules are relaxed so that a tile can move to any
adjacent square , then h 2 gives the shortest solution
Relaxed problem : A problem with fewer restrictions on the
actions
Admissible heuristics for the original problem can be derived
from the optimal (exact) solution to a relaxed problem
Key point : the optimal solution cost of a relaxed problem is
no greater than the optimal solution cost of the original
problem
Which should we choose if none of the h 1 h m dominates any of
the others?
We can have the best of all worlds, i.e., use whichever
function is most accurate on the current node
Subproblem *
Admissible heuristics for the original problem can also be
derived from the solution cost of the subproblem.
Learning from experience *
Local search algorithms and optimization
Systematic search algorithms
to find (or given) the goal and to find the path to that
goal
Local search algorithms
the path to the goal is irrelevant , e.g., n -queens
problem
state space = set of complete configurations
keep a single current state and try to improve it, e.g., move
to its neighbors
Key advantages:
use very little (constant) memory
find reasonable solutions in large or infinite (continuous)
state spaces
(pure) Optimization problem:
to find the best state (optimal configuration ) based on an
objective function , e.g. reproductive fitness Darwinian, no goal
test and path cost
Local search example
Local search state space landscape
elevation = the value of the objective function or heuristic
cost function
global minimum heuristic cost function
A complete local search algorithm finds a solution if one
exists
A optimal algorithm finds a global minimum or maximum
moves in the direction of increasing value until a peak
current node data structure only records the state and its
objective function
neither remember the history nor look beyond the immediate
neighbors
like climbing Mount Everest in thick fog with amnesia
Hill-climbing search
complete-state formulation for 8-queens
successor function returns all possible states generated by
moving a single queen to another square in the same column (8 x 7 =
56 successors for each state)
the heuristic cost function h is the number of pairs of queens
that are attacking each other
Hill-climbing search - example best moves reduce h = 17 to h = 12
local minimum with h = 1
Hill-climbing search greedy local search
Hill climbing, the greedy local search, often gets stuck
Local maxima : a peak that is higher than each of its
neighboring states, but lower than the global maximum
Ridges : a sequence of local maxima that is difficult to
navigate
Plateau : a flat area of the state space landscape
a flat local maximum : no uphill exit exists
a shoulder : possible to make progress
can only solve 14% of 8-queen instance but fast (4 steps to S
and 3 to F )
Hill-climbing search improvement
Allows sideways move : with hope that the plateau is a
shoulder
could stuck in an infinite loop when it reaches a flat local
maximum
limits the number of consecutive sideways moves
can solve 94% of 8-queen instances but slow (21 steps to S and
64 to F )
Variations
stochastic hill climbing
chooses at random ; probability of selection depends on the
steepness
first choice hill climbing
randomly generates successors to find a better one
All the hill climbing algorithms discussed so far are
incomplete
fail to find a goal when one exists because they get stuck on
local maxima
Random-restart hill climbing
conducts a series of hill-climbing searches; randomly generated
initial states
Have to give up the global optimality
landscape consists of a large amount of porcupines on a flat
floor
NP-hard problems
Simulated annealing search
combine hill climbing ( efficiency ) with random walk (
completeness )
annealing : harden metals by heating metals to a high
temperature and gradually cooling them
getting a ping-pong ball into the deepest crevice in a humpy
surface
shake the surface to get the ball out of the local minima
not too hard to dislodge it from the global minimum
simulated annealing :
start by shaking hard (at a high temperature) and then
gradually reduce the intensity of the shaking (lower the
temperature)
escape the local minima by allowing some bad moves
but gradually reduce their size and frequency
Simulated annealing search - Implementation
Always accept the good moves
The probability to accept a bad move
decreases exponentially with the badness of the move
decreases exponentially with the temperature T
(decreasing)
finds a global optimum with probability approaching 1 if the
schedule lowers T slowly enough
Local beam search
Local beam search : keeps track of k states rather than just
one
generates all the successors of all k states
selects the k best successors from the complete list and
repeats
quickly abandons unfruitful searches and moves to the space
where the most progress is being made
Come over here, the grass is greener!
lack of diversity among the k states
stochastic beam search : chooses k successors at random , with
the probability of choosing a given successor having an increasing
value
natural selection: the successors (offspring) if a state
(organism) populate the next generation according to is value
(fitness).
Genetic algorithms
Genetic Algorithms (GA): successor states are generated by
combining two parents states.
population : s set of k randomly generated states
each state, called individual , is represented as a string over
a finite alphabet, e.g. a string of 0s and 1s; 8-queens : 24 bits
or 8 digits for their positions
fitness (evaluation) function : return higher values for better
states,
e.g., the number of nonattacking pairs of queens
randomly choosing two pairs for reproducing based on the
probability ; proportional to fitness score; not choosing the
similar ones too early
Genetic algorithms (contd)
schema : a substring in which some of the positions can be left
unspecified
instances : strings that match the schema
GA works best when schemas correspond to meaningful components
of a solution.
a crossover point is randomly chosen from the positions in the
string
larger steps in the state space early and smaller steps
later
each location is subject to random mutation with a small
independent probability