Date post: | 18-Jan-2018 |
Category: |
Documents |
Upload: | audrey-hart |
View: | 214 times |
Download: | 0 times |
Search• We will consider the problem of designing goal-based agents in fully observable, deterministic, discrete, known environments
• Example:Start state
Goal state
Search• We will consider the problem of designing goal-based agents in fully observable, deterministic, discrete, known environments • The solution is a fixed sequence of actions• Search is the process of looking for the sequence of
actions that reaches the goal• Once the agent begins executing the search solution, it
can ignore its percepts (open-loop system)
Search problem components• Initial state• Actions• Transition model
• What is the result of performing a given action in a given state?
• Goal state• Path cost
• Assume that it is a sum of nonnegative step costs
• The optimal solution is the sequence of actions that gives the lowest path cost for reaching the goal
Initialstate
Goal state
Example: Romania
• Initial state• Arad
• Actions• Go from one city to another
• Transition model• If you go from city A to
city B, you end up in city B• Goal state
• Bucharest• Path cost
• Sum of edge costs
• On vacation in Romania; currently in Arad• Flight leaves tomorrow from Bucharest
State space• The initial state, actions, and transition model define the state
space of the problem• The set of all states reachable from initial state by any sequence of
actions• Can be represented as a directed graph where the nodes are states
and links between nodes are actions• What is the state space for the Romania problem?
Search• Given:
• Initial state• Actions• Transition model• Goal state• Path cost
• How do we find the optimal solution?• How about building the state space and then using Dijkstra’s
shortest path algorithm?• The state space may be huge!• Complexity of Dijkstra’s is O(E + V log V), where V is the size of the state
space
Tree Search• Let’s begin at the start node and expand it by making a list of all
possible successor states• Maintain a fringe or a list of unexpanded states• At each step, pick a state from the fringe to expand • Keep going until you reach the goal state• Try to expand as few states as possible
Search tree• “What if” tree of possible actions and
outcomes• The root node corresponds to the
starting state• The children of a node correspond to the
successor states of that node’s state• A path through the tree corresponds to a
sequence of actions• A solution is a path ending in the goal state
• Nodes vs. states• A state is a representation of a physical
configuration, while a node is a data structure that is part of the search tree
…………
Starting state
Successor state
Action
Goal state
Tree Search Algorithm Outline• Initialize the fringe using the starting state• While the fringe is not empty
• Choose a fringe node to expand according to search strategy
• If the node contains the goal state, return solution• Else expand the node and add its children to the fringe
Tree search example
Tree search example
Tree search exampleFringe
Search strategies• A search strategy is defined by picking the order of node
expansion• Strategies are evaluated along the following dimensions:
• Completeness: does it always find a solution if one exists?• Optimality: does it always find a least-cost solution?• Time complexity: number of nodes generated• Space complexity: maximum number of nodes in memory
• Time and space complexity are measured in terms of • b: maximum branching factor of the search tree• d: depth of the least-cost solution• m: maximum length of any path in the state space (may be infinite)
Uninformed search strategies• Uninformed search strategies use only the information
available in the problem definition
• Breadth-first search• Uniform-cost search• Depth-first search• Iterative deepening search
Review: Search problem formulation• Initial state• Actions• Transition model• Goal state• Path cost
• What is the optimal solution?• What is the state space?
Review: Tree search• Initialize the fringe using the starting state• While the fringe is not empty
• Choose a fringe node to expand according to search strategy
• If the node contains the goal state, return solution• Else expand the node and add its children to the fringe
• To handle repeated states:• Keep an explored set; add each node to the explored set
every time you expand it• Every time you add a node to the fringe, check whether it
already exists in the fringe with a higher path cost, and if yes, replace that node with the new one
Search strategies• A search strategy is defined by picking the order of node
expansion• Strategies are evaluated along the following dimensions:
• Completeness: does it always find a solution if one exists?• Optimality: does it always find a least-cost solution?• Time complexity: number of nodes generated• Space complexity: maximum number of nodes in memory
• Time and space complexity are measured in terms of • b: maximum branching factor of the search tree• d: depth of the optimal solution• m: maximum length of any path in the state space (may be infinite)
Uninformed search strategies• Uninformed search strategies use only the information
available in the problem definition
• Breadth-first search• Uniform-cost search• Depth-first search• Iterative deepening search
Search Algorithm Properties
Breadth-First Search
Breadth-first search• Expand shallowest unexpanded node• Implementation:
• fringe is a FIFO queue, i.e., new successors go at end
A
D F
B C
E G
Breadth-first search• Expand shallowest unexpanded node• Implementation:
• fringe is a FIFO queue, i.e., new successors go at end
A
D F
B C
E G
Breadth-first search• Expand shallowest unexpanded node• Implementation:
• fringe is a FIFO queue, i.e., new successors go at end
A
D F
B C
E G
Breadth-first search• Expand shallowest unexpanded node• Implementation:
• fringe is a FIFO queue, i.e., new successors go at end
A
D F
B C
E G
Breadth-first search• Expand shallowest unexpanded node• Implementation:
• fringe is a FIFO queue, i.e., new successors go at end
A
D F
B C
E G
Properties of breadth-first search• Complete?
Yes (if branching factor b is finite)• Optimal?
Yes – if cost = 1 per step• Time?
Number of nodes in a b-ary tree of depth d: O(bd)(d is the depth of the optimal solution)
• Space? O(bd)
• Space is the bigger problem (more than time)
Uniform-cost search• Expand least-cost unexpanded node• Implementation: fringe is a queue ordered by path cost (priority
queue)• Equivalent to breadth-first if step costs all equal
• Complete? Yes, if step cost is greater than some positive constant ε (we don’t want
infinite sequences of steps that have a finite total cost)• Optimal?
Yes – nodes expanded in increasing order of path cost• Time?
Number of nodes with path cost ≤ cost of optimal solution (C*), O(bC*/ ε)This can be greater than O(bd): the search can explore long paths consisting
of small steps before exploring shorter paths consisting of larger steps • Space?
O(bC*/ ε)
Depth-First Search
Depth-first search• Expand deepest unexpanded node• Implementation:
• fringe = LIFO queue, i.e., put successors at front
A
D F
B C
E G
Depth-first search• Expand deepest unexpanded node• Implementation:
• fringe = LIFO queue, i.e., put successors at front
A
D F
B C
E G
Depth-first search• Expand deepest unexpanded node• Implementation:
• fringe = LIFO queue, i.e., put successors at front
A
D F
B C
E G
Depth-first search• Expand deepest unexpanded node• Implementation:
• fringe = LIFO queue, i.e., put successors at front
A
D F
B C
E G
Depth-first search• Expand deepest unexpanded node• Implementation:
• fringe = LIFO queue, i.e., put successors at front
A
D F
B C
E G
Depth-first search• Expand deepest unexpanded node• Implementation:
• fringe = LIFO queue, i.e., put successors at front
A
D F
B C
E G
Depth-first search• Expand deepest unexpanded node• Implementation:
• fringe = LIFO queue, i.e., put successors at front
A
D F
B C
E G
Depth-first search• Expand deepest unexpanded node• Implementation:
• fringe = LIFO queue, i.e., put successors at front
A
D F
B C
E G
Depth-first search• Expand deepest unexpanded node• Implementation:
• fringe = LIFO queue, i.e., put successors at front
A
D F
B C
E G
Properties of depth-first search• Complete?
Fails in infinite-depth spaces, spaces with loopsModify to avoid repeated states along path
complete in finite spaces
• Optimal?No – returns the first solution it finds
• Time? Could be the time to reach a solution at maximum depth m: O(bm)Terrible if m is much larger than dBut if there are lots of solutions, may be much faster than BFS
• Space? O(bm), i.e., linear space!
Iterative deepening search• Use DFS as a subroutine
1. Check the root2. Do a DFS searching for a path of length 13. If there is no path of length 1, do a DFS searching for a path of
length 24. If there is no path of length 2, do a DFS searching for a path of
length 3…
Iterative deepening search
Iterative deepening search
Iterative deepening search
Iterative deepening search
Properties of iterative deepening search
• Complete?Yes
• Optimal?Yes, if step cost = 1
• Time?(d+1)b0 + d b1 + (d-1)b2 + … + bd = O(bd)
• Space?O(bd)
Informed search• So far, have assumed that no nongoal state looks better than another
• Unrealistic• Even without knowing the road structure, some locations
seem closer to the goal than others• Some states of the 8s puzzle seem closer to the goal than
others• Makes sense to expand closer-seeming nodes first
Informed search• Idea: give the algorithm “hints” about the desirability of
different states • Use an evaluation function to rank nodes and select the most
promising one for expansion
• Greedy best-first search• A* search
Heuristic function• Heuristic function h(n) estimates the cost of reaching goal
from node n• Example:
Start state
Goal state
Heuristics• Key notion: heuristic function h(n) gives an estimate of the distance from n to the goal• h(n)=0 for goal nodes
• E.g. straight-line distance for traveling problem
A
BC
FD E
3 4 4
39
2
2
start stategoal state
• Say: h(A) = 9, h(B) = 8, h(C) = 9, h(D) = 6, h(E) = 3, h(F) = 0
• We’re adding something new to the problem!
• Can use heuristic to decide which nodes to expand first
Heuristic for the Romania problem
Greedy best-first search• Expand the node that has the lowest value of the heuristic
function h(n)
Greedy best-first search example
Greedy best-first search example
Greedy best-first search example
Greedy best-first search example
Properties of greedy best-first search• Complete?
No – can get stuck in loops
startgoal
Properties of greedy best-first search• Complete?
No – can get stuck in loops• Optimal?
No
Properties of greedy best-first search• Complete?
No – can get stuck in loops• Optimal?
No• Time?
Worst case: O(bm)Best case: O(bd) – If h(n) is 100% accurate
• Space?Worst case: O(bm)
A bad example for greedy
A
B
FD E
3 4 4
7 6
start stategoal state
• Say: h(A) = 9, h(B) = 5, h(D) = 6, h(E) = 3, h(F) = 0
• Problem: greedy evaluates the promise of a node only by how far is left to go, does not take cost occurred already into account
How can we fix the greedy problem?
A* search• Idea: avoid expanding paths that are already expensive
• The evaluation function f(n) is the estimated total cost of the path through node n to the goal:
f(n) = g(n) + h(n)
g(n): cost so far to reach n (path cost)h(n): estimated cost from n to goal (heuristic)
Romania with step costs in km
A* search exampleOpen List:
Arad
We start with our initial state Arad. We make a node and add it to the open list. Since it’s the only thing on the open list, we expand the node.
Think of the open list as a priority queue (or heap) that sorts the nodes inside of it according to their g()+h() score.
A* search exampleOpen List:
Sibiu
Timisoara
Zerind
We add the three nodes we found to the open list.
We sort them according to the g()+h() calculation.
A* search exampleOpen List:
Rimricu Vicea
Fagaras
Timisoara
Zerind
Arad
Oradea
We’ve been to Arad before. Don’t list it again on the open list.
When we expand Sibiu, we run into Arad again. But we’ve already expanded this node once; so, we don’t add it to the open list again.
A* search exampleOpen List:
Rimricu Vicea
Fagaras
Timisoara
Zerind
Oradea
We see that Rimricu Vicea is at the top of the open list; so, it’s the next node we will expand.
A* search exampleOpen List:
Fagaras
Pitesti
Timisoara
Zerind
Craiova
Sibiu
Oradea
When we expand Rimricu Vicea, we run into Sibiu again.
But we’ve already expanded this node once; so, we don’t add it to the open list again.
A* search exampleOpen List:
Fagaras
Pitesti
Timisoara
Zerind
Craiova
Oradea
Fagaras will be the next node we should expand – it’s at the top of the sorted open list.
A* search exampleOpen List:
Pitesti
Timisoara
Zerind
Bucharest
Craiova
Sibiu
Oradea
When we expand Fagaras, we find Sibiu again. We don’t add it to the open list.
We also find Bucharest, but we’re not done. The algorithm doesn’t end until we “expand” the goal node – it has to be at the top of the open list.
A* search exampleOpen List:
Pitesti
Timisoara
Zerind
Bucharest
Craiova
Oradea
It looks like Pitesti is the next node we should expand.
A* search exampleOpen List:
Bucharest
Timisoara
Zerind
Craiova
Rimricu Vicea
Oradea
We just found a better value for Bucharest; so, it got moved higher in the list. We also found a worse value for Craiova – we just ignore this.
And of course, we ran into Rimricu Vicea again. Since it’s already been expanded once, we don’t re-add it to the Open List.
A* search exampleOpen List:
Bucharest
Timisoara
Zerind
Craiova
Oradea
Now it looks like Bucharest is at the top of the open list…
A* search exampleOpen List:
Bucharest
Timisoara
Zerind
Craiova
Oradea
Now we “expand” the node for Bucharest.
We’re done! (And we know the path that we’ve found is optimal.)
Admissible heuristics• A heuristic h(n) is admissible if for every node n, h(n) ≤ h*(n), where h*(n) is the true cost to reach the goal state from n
• An admissible heuristic never overestimates the cost to reach the goal, i.e., it is optimistic
• Example: straight line distance never overestimates the actual road distance
• Theorem: If h(n) is admissible, A* is optimal
Optimality of A*
• Suppose A* terminates its search at n* • It has found a path whose actual cost f(n*) = g(n*) is lower than the estimated cost f(n) of any path going through any fringe node
• Since f(n) is an optimistic estimate, there is no way n can have a successor goal state n’ with g(n’) < C*
n
n*f(n*) = C*
f(n) > C*
n' g(n') f(n) > C*
Optimality of A*• A* is optimally efficient – no other tree-based algorithm
that uses the same heuristic can expand fewer nodes and still be guaranteed to find the optimal solution• Any algorithm that does not expand all nodes with f(n) < C* risks
missing the optimal solution
Properties of A*• Complete?
Yes – unless there are infinitely many nodes with f(n) ≤ C*• Optimal?
Yes• Time?
Number of nodes for which f(n) ≤ C* (exponential)• Space?
Exponential
Uninformed search strategiesAlgorithm Complete? Optimal? Time
complexitySpace
complexity
BFS
UCS
DFS
IDS
b: maximum branching factor of the search treed: depth of the optimal solutionm: maximum length of any path in the state spaceC*: cost of optimal solution
Yes
Yes
No
Yes
If all step costs are equal
If all step costs are equal
Yes
No
O(bd)
O(bm)
O(bd)
O(bd)
O(bm)
O(bd)
Number of nodes with g(n) ≤ C*
All search strategies
Algorithm Complete? Optimal? Time complexity
Space complexity
BFS
UCS
DFS
IDS
Greedy
A*
Yes
Yes
No
Yes
If all step costs are equal
If all step costs are equal
Yes
No
O(bd)
Number of nodes with g(n) ≤ C*
O(bm)
O(bd)
O(bd)
O(bm)
O(bd)
No NoWorst case: O(bm)
Yes Yes
Best case: O(bd)
Number of nodes with g(n)+h(n) ≤ C*
Exercise• Trace from Lugoj to Bucharest
Search where the path doesn’t matter• So far, looked at problems where the path was the solution• Traveling on a graph• Eights puzzle
• However, in many problems, we just want to find a goal state• Doesn’t matter how we get there
Queens puzzle
Q
Q
Q
Q
Q
Q
Q
Q
• Place eight queens on a chessboard so that no two attack each other
Backtracking• Backtracking is a technique used to solve problems with a large search space, by systematically trying and eliminating possibilities.
• A standard example of backtracking would be going through a maze. • At some point in a maze, you might have two options
of which direction to go:
Junction
Portion A
Port
ion
B
Backtracking
Junction
Portion B
Port
ion
A
One strategy would be to try going through Portion A of the maze. If you get stuck before
you find your way out, then you "backtrack" to the junction.
At this point in time you
know that Portion A will NOT lead you out of the maze, so you then start
searching in Portion B
Backtracking• Clearly, at a single junction you could have even more than 2 choices.
• The backtracking strategy says to try each choice, one after the other, • if you ever get stuck,
"backtrack" to the junction and try the next choice.
• If you try all choices and never found a way out, then there IS no solution to the maze.
Junction
BC
A
N-queens problem• Find a configuration of n queens not attacking each other
• What is the maximum number of queens that can be placed on an n x n chessboard such that no two attack one another?
• The answer is n queens, which gives eight queens for the usual 8x8 board
Backtracking – Eight Queens Problem• Find an arrangement of 8 queens on a single chess board such that no two queens are attacking one another.
• In chess, queens can move all the way down any row, column or diagonal (so long as no pieces are in the way).
• Due to the first two restrictions, it's clear that each row and column of the board will have exactly one queen.
12 Unique Solutions8-Queens Problem
8-Queens Problem• There are 92 distinct solutions• There are 12 unique solutions discounting symmetrical answers (rotatations/reflections)
• How many solutions for 4-Queens? N-Queens?
Wikipedia.org
Problems N < 4
3
2
1
N < 4
Cannot use N Queens
Backtracking in Decision Trees•Idea: Start at the root of the decision tree and move downwards, that is, make a sequence of decisions, until you either reach a solution or you enter a situation from where no solution can be reached by any further sequence of decisions.•In the latter case, backtrack to the parent of the current node and take a different path downwards from there. If all paths from this node have already been explored, backtrack to its parent.•Continue this procedure until you find a solution or establish that no solution exists (there are no more paths to try out).
Solution: Backtracking• place a queen in the top row, then note the column and diagonals it
occupies.
• then place a queen in the next row down, taking care not to place it in the same column or diagonal. Keep track of the occupied columns and diagonals and move on to the next row.
• If no position is open in the next row, we back track to the previous row and move the queen over to the next available spot in its row and the process starts over again.
Backtracking in Decision Trees
Q
Q
Q
Q
Q
Q
Q
Q
Q
Q
Q
Q
place 1st queen
place 2nd queen
place 3rd queen
place 4th queen
empty board
• The backtracking strategy is as follows:
1) Place a queen on the first available square in row 1.
2) Move onto the next row, placing a queen on the first available square there (that doesn't conflict with the previously placed queens).
3) Continue in this fashion until either:
a) you have solved the problem, or
b) you get stuck. • When you get stuck, remove the
queens that got you there, until you get to a row where there is another valid square to try.
Backtracking – Eight Queens Problem
Animated Example:http://www.hbmeyer.de/backtrack/achtdamen/eight.htm#up
Q Q
Continue…
Exercise – now it’s your turn!
Backtracking – Eight Queens Problem
• The neat thing about coding up backtracking, is that it can be done recursively, without having to do all the bookkeeping at once.• Instead, the stack or recursive calls does most of the bookkeeping • (ie, keeping track of which queens we've placed, and which
combinations we've tried so far, etc.)
void solveItRec(int perm[], int location, struct onesquare usedList[]) {
if (location == SIZE) { printSol(perm); } for (int i=0; i<SIZE; i++) {
if (usedList[i] == false) {
if (!conflict(perm, location, i)) { perm[location] = i; usedList[i] = true; solveItRec(perm, location+1, usedList); usedList[i] = false; } } } }
perm[] - stores a valid permutation of queens from index 0 to location-1. location – the column we are placing the next queenusedList[] – keeps track of the rows in which the queens have already been placed.
Found a solution to the problem, so print it!
Loop through possible rows to place this queen.
Only try this row if it hasn’t been used
Check if this position conflicts with any previous queens on the diagonal
1) mark the queen in this row2) mark the row as used3) solve the next column location
recursively4) un-mark the row as used, so
we can get ALL possible valid solutions.
Backtracking – 8 queens problem - Analysis• Another possible brute-force algorithm is generate the permutations
of the numbers 1 through 8 (of which there are 8! = 40,320), • and uses the elements of each permutation as indices to place a queen
on each row. • Then it rejects those boards with diagonal attacking positions.
• The backtracking algorithm, is a slight improvement on the permutation method, • constructs the search tree by considering one row of the board at a time,
eliminating most non-solution board positions at a very early stage in their construction.
• Because it rejects row and diagonal attacks even on incomplete boards, it examines only 15,720 possible queen placements.
• A further improvement which examines only 5,508 possible queen placements is to combine the permutation based method with the early pruning method: • The permutations are generated depth-first, and the search space is
pruned if the partial permutation produces a diagonal attack
100
Solving Sudoku – Later Steps1 1 2 1 2 4
1 2 4 8 1 2 4 8 9
uh oh!
101
Sudoku – A Dead End• We have reached a dead end in our search
• With the current set up none of the nine digits work in the top right corner
1 2 4 8 9
CS314 102
Backing Up• When the search reaches a dead end in backs up to the previous cell it was trying to fill and goes onto to the next digit
• We would back up to the cell with a 9 and that turns out to be a dead end as well so we back up again• so the algorithm needs to remember
what digit to try next• Now in the cell with the 8. We try and 9 and move forward again.
1 2 4 8 9
1 2 4 9
Characteristics of Brute Forceand Backtracking
• Brute force algorithms are slow• The don't employ a lot of logic
• For example we know a 6 can't go in the last 3 columns of the first row, but the brute force algorithm will plow ahead any way
• But, brute force algorithms are fairly easy to implement as a first pass solution• many backtracking algorithms are brute force algorithms
Key Insights• After trying placing a digit in a cell we want to solve the
new sudoku board• Isn't that a smaller (or simpler version) of the same problem
we started with?!?!?!?• After placing a number in a cell the we need to
remember the next number to try in case things don't work out.
• We need to know if things worked out (found a solution) or they didn't, and if they didn't try the next number
• If we try all numbers and none of them work in our cell we need to report back that things didn't work
Minesweeper
CS314 Recursive Backtracking 105
Minesweeper Reveal Algorithm• Minesweeper• click a cell
• if bomb game over• if cell that has 1 or more bombs on border
then reveal the number of bombs that border cell• if a cell that has 0 bombs on border
then reveal that cell as a blank and click on the 8 surrounding cells
CS314 Recursive Backtracking 107Another Backtracking ProblemA Simple Maze
Search maze until wayout is found. If no wayout possible report that.
Mazes and Backtracking• A final example of something that can be solved using backtracking is a maze. • From your start point, you will iterate through each
possible starting move. • From there, you recursively move forward. • If you ever get stuck, the recursion takes you back to
where you were, and you try the next possible move.
• In dealing with a maze, to make sure you don't try too many possibilities, • one should mark which locations in the maze have
been visited already so that no location in the maze gets visited twice.
• (If a place has already been visited, there is no point in trying to reach the end of the maze from there again.
109
The Local View
North
EastWest
Behind me, to the South is a door leading South
Which way doI go to getout?
Modified Backtracking Algorithm for Maze
• If the current square is outside, return TRUE to indicate that a solution has been found.If the current square is marked, return FALSE to indicate that this path has been tried.Mark the current square.for (each of the four compass directions) { if ( this direction is not blocked by a wall )
{ Move one step in the indicated direction from the current square.Try to solve the maze from there by making a recursive call.If this call shows the maze to be solvable, return TRUE to indicate
that fact.}
}Unmark the current square.Return FALSE to indicate that none of the four directions led to a solution.
CS314 Recursive Backtracking 111
Backtracking in ActionThe crucial part of the algorithm is the for loop that takes us through the alternatives from the current square. Here we have moved to the North.
for (dir = North; dir <= West; dir++){ if (!WallExists(pt, dir))
{if (SolveMaze(AdjacentPoint(pt, dir)))return(TRUE);
}
CS314 Recursive Backtracking 112
Backtracking in Action
Here we have moved North again, but there isa wall to the North .East is alsoblocked, so we try South. That call discovers thatthe square is marked, so it just returns.
113
So the next move we can make is West.
Where is this leading?
CS314 Recursive Backtracking 114
This path reaches a dead end.
Time to backtrack!
Remember theprogram stack!
CS314 Recursive Backtracking 115
The recursive calls end and return until we find ourselves back here.
CS314 Recursive Backtracking 116
And now we trySouth
CS314 Recursive Backtracking 117
Path Eventually Found
118
Other Backtracking Problems• Knapsack problem / Exhaustive Search
• Filling a knapsack. Given a choice of items with various weights and a limited carrying capacity find the optimal load out. 50 lb. knapsack. items are 1 40 lb, 1 32 lb. 2 22 lbs, 1 15 lb, 1 5 lb. A greedy algorithm would choose the 40 lb item first. Then the 5 lb. Load out = 45lb. Exhaustive search 22 + 22 + 5 = 49.
Assignment 1