Date post: | 05-Jan-2016 |
Category: |
Documents |
Upload: | evelyn-marshall |
View: | 212 times |
Download: | 0 times |
Local search• In many optimisation problems we are interested in
the solution rather than the path to it.• Previously the state space was the paths to goal• Now the state space is "complete" configurations• The approach is to find one or more configurations
satisfying constraints then use a local search algorithm and try to improve it
• So local search algorithms need only keep a single "current" state rather than a current path, thus being very memory efficient even in very large problems.
Example problems
• Scheduling
• Layout
• Evolution
• Travelling salesman
• N-queens
Example – N-queens
• Given a n×n chess board, place n queens without any being able to take another in a single chess move.
• i.e. Only a single queen in any row, column, or diagonal.
• Left example a 4-queens problem (n=4).
Objective function
• Local search problems need an objective function.
• This may be the nearness to goal or simply the negative of the “cost” of a given state.
• A high objective function denotes a good solution.
Example objective functions
• Scheduling: negative time to complete, time spend working/wasted time.
• Layout: number of items/amount of space
• Evolution: reproductive success of a species
• Travelling salesman: negative number of cities visited twice or more
• N-queens: number of non-attacking queens
State space landscape
State space landscape
Hill-climbing search• “Climbing Everest in thick fog with amnesia”• Repeatedly “climb” to the neighboring state with
highest objective function until no neighbor has higher objective function.
Local maxima/minina• Problem: depending on initial state, can
get stuck in local maxima/minina
1/ (1+H(n)) = 1/17
1/ (1+H(n)) = 1/2 Local minima
Local beam search
• keep track of k states rather than just one• start with k randomly generated states• at each iteration, all the successors of all k
states are generated• if any one is a goal state, stop; else select k best
successors from complete list and repeat.
Simulated annealing search• Idea: escape local maxima by allowing some "bad" moves but gradually
decrease their frequency and range (VSLI layout, scheduling)
QuickTime™ and aTIFF (LZW) decompressor
are needed to see this picture.
QuickTime™ and aTIFF (LZW) decompressor
are needed to see this picture.
Simulated annealing example• Point feature labelling
Genetic algorithm search
• population of k randomly generated states• a state is represented as a string over a finite
alphabet (often a string of 0s and 1s)• evaluation function (fitness function) - higher
values for better states.• produce the next generation of k states by
selection, crossover, and mutation• rates for each of these configure search
– elitism / crossover rate / mutation rate
Genetic algorithms in games
• computationally expensive so primarily used as an offline form of learning
• Cloak, Dagger & DNA (Oidian Systems)– 4 DNA strands defining opponent behaviour– between battles, opponents play each other
• Creatures (Millennium Interactive)– genetic algorithms to learning the weights in a neural
network that defines behaviour
Online search
• All previous techniques have focused on offline reasoning (think first then act).
• Now we will briefly look at online search (think, act, think, act, ...)
• Advantageous in dynamic situations or those with only partial information.
“Real-time” search concepts
• in A* the whole path is computed off-line, before the agent walks through the path
• this solution is only valid for static worlds • if the world changes in the meantime, the initial
path is no longer valid:– new obstacles appear– position of goal changes (e.g. moving target)
“Real-time” definitions
• off-line (non real-time): the solution is computed in a given amount of time before being executed
• real-time: one move computed at a time, and that move executed before computing the next
• anytime: the algorithm constantly improves its solution through time capable of providing “current best” at any time
Agent-based (online) search• for example:
– mobile robot– NPC without perfect knowledge– agent that must act now with limited information
• planning and execution are interleaved• could apply standard search techniques:
– best-first (but we know it is poor)– depth-first (has to physically back-track)– A* (but nodes in the fringe are not accessible)
LRTA*: Learning Real-time A*• augment hill-climbing with memory• store “current best estimate”• follow path based on neighbours’ estimates• update estimates based on experience• experience learning• flatten out local maxima…
LRTA*: example8 9 2 2 4 1
1 1 1 1 1
8 9 3 2 4 11 1 1 1 1
8 9 3 4 4 11 1 1 1 1
8 9 5 4 4 11 1 1 1 1
8 9 5 5 4 11 1 1 1 1
Learning real-time A*