Warm Up A* Exercise
In groups, draw the tree that A* builds to find a path from Oradea to Bucharest marking g, h, and f for each node. Use straight-line distance as h(n)
In what order are the nodes expanded? What is the path found?
Reminder
• Project proposal : due in two weeks
• Thanks for the pre-proposal, should be graded, some have comments
• Homework 1 due next week
• Next week: guest lecture by Janith Weerasinghe, no supplemental readings
Overview
• Local Search (Hill climbing, simulated annealing, genetic algorithms)
• Constraint Reasoning (Constraint Satisfaction problems (CSP), constraint optimization (COP), Distributed COP
• Paper discussion
Local Search
• In many optimization problems, the path to the goal is irrelevant; the goal state itself is the solution
• State space = set of "complete" configurations
• Find configuration satisfying constraints, e.g., n-queens
• In such cases, we can use local search algorithms
• keep a single "current" state, try to improve it
Example: n-queens
• Put n queens on an n × n board with no two queens on the same row, column, or diagonal
Hill-climbing search 8-queens problem
• h = number of pairs of queens that are attacking each other, either directly or indirectly
• h = 17 for the above state
Annealing in Metallurgy
• Remove crystal dislocations in metal
• Heat the material - cause atoms to become unstuck from position
• Then slowly cool, atoms wander try to find configurations with lower energy
Simulate this process to escape local minima
• Replace solution with nearby solution
• How nearby depends on “temperature”
• Heat - nearly random solution
• Slowly cool - gradually improve
Simulated annealing search
• Idea: escape local maxima by allowing some "bad" moves but gradually decrease their frequency
Properties of Simulated annealing search
• One can prove: If T decreases slowly enough, then simulated annealing search will find a global optimum with probability approaching 1
• Widely used in VLSI layout, airline scheduling, etc
Local beam search
• Keep track of k states rather than just one
• Start with k randomly generated states
• At each iteration, all the successors of all k states are generated
• If any one is a goal state, stop; else select the k best successors from the complete list and repeat.
Genetic Algorithms
• A successor state is generated by combining two parent states
• Start with k randomly generated states (population)
• A state is represented as a string over a finite alphabet (often a string of 0s and 1s)
• Evaluation function (fitness function). Higher values for better states.
• Produce the next generation of states by selection, crossover, and mutation
Genetic algorithms
• Fitness function: number of non-attacking pairs of queens (min = 0, max = 8 × 7/2 = 28)
• 24/(24+23+20+11) = 31%
• 23/(24+23+20+11) = 29% etc
Constraint Satisfaction Problem (CSP)
• Variables: V1,V2,V3...with domains D1, D2, D3
• Constraints: Set of allowed value pairs
• V1 “not equal” V2 = {(red,blue),(green, blue,(green, red)}
• Solution: Assign values to variables that satisfy all constraints
• V1=red, V2=blue, V3=green
V1
V2 V3
= =
D1={red, green}
D2={blue,red} D3={green}
3SAT as CSP
• xi are variables
• values are true and false
• Constraints expressed in conjunctive normal form
• ex 1: (x1 v x2 v x3) ^ (~x1 v ~x2 v ~x3) ^ (~x1 v x2 v ~x3)
• ex 2: (x1 v ~x2) ^ (~x1 v ~x2) ^ (~x1 v ~x2)
What if no solution? Or multiple solutions?
• If no solution, minimize broken constraints?
• If multiple solutions, some solutions may be preferred
• Gives rise to the Constraint Optimization Problem
Constraint Optimization
• Constraints have degrees of violation (or degrees of satisfaction)
• Goal is to minimize violation (or maximize satisfaction)
• Significantly generalizes CSPs
Constraint Optimization Problem (COP)
• Given:
• Variables (x1,x2,...,xn)
• Finite, discrete domains (D1,D2,...,Dn)
• Foreach xi, xj:
• Valued constraint function
• fij: Di x Dj N
Constraint Optimization(COP)
• Goal: Find complete solution: an assignment A that minimizes F(A) where
• F(A) = Sum(fij(di,dj), xi di, xj dj in A
• Example:
di dj f(di,dj)
3
1
1
2
3
333
3
1 11
1 11
2F(A)=12 F(A)=6 F(A)=5
Example: Map Coloring
• Variables WA, NT, Q, NSW, V, SA, T
• Domains Di = {red,green,blue}
• Constraints: adjacent regions must have different colors
e.g., WA ≠ NT, or (WA,NT) in {(red,green),(red,blue),(green,red), (green,blue),(blue,red),(blue,green)}
Examples: Map Coloring
• Solutions are complete and consistent assignments, e.g., WA = red, NT = green,Q = red,NSW = green,V = red,SA = blue,T = green
Constraint Graph
• Binary CSP: each constraint relates two variables
• Constraint graph: nodes are variables, arcs are constraints
CSP as search
• Initial State: Empty assignment {}
• Successor Function: Assign value to an unassigned variable
• Goal Test: Complete assignment that does not violate any constraints
• Path Cost: Constant for each step
Improving efficiency
• General-purpose heuristics can give huge gains in speed:
• Which variable should be assigned next?
• In what order should its values be tried?
• Can we detect inevitable failure early?
Most Constrained Variable
• Most constrained variable:
choose the variable with the fewest legal values
• a.k.a. minimum remaining values (MRV) heuristic
Most Constraining Variable
• Tie-breaker among most constrained variables
• Most constraining variable:
• Choose the variable with the most constraints on remaining variables
Least constraining variable assignment
• Given a variable, choose the least constraining value:
• the one that rules out the fewest values in the remaining variables
• Combining these heuristics makes 1000 queens possible (as opposed to about 25)
Forward Checking
• Idea:
• Keep track of remaining legal values for unassigned variables
• Terminate search when any variable has no legal values
Forward Checking
• Idea:
• Keep track of remaining legal values for unassigned variables
• Terminate search when any variable has no legal values
Constraint Propagation
• Forward checking propagates information from assigned to unassigned variables, but doesn't provide early detection for all failures:
• NT and SA cannot both be blue!
• Constraint propagation repeatedly enforces constraints locally
Arc Consistency
• Simplest form of propagation makes each arc consistent
• X Y is consistent iff
• for every value x of X there is some allowed y
Arc Consistency
• Simplest form of propagation makes each arc consistent
• X Y is consistent iff
• for every value x of X there is some allowed y
• If X loses a value, neighbors of X must be rechecked
Arc Consistency• Simplest form of propagation makes each arc consistent
• X Y is consistent iff
• for every value x of X there is some allowed y
• If X loses a value, neighbors of X must be rechecked
• Detects failures quicker than forward checking
Local Search for CSPs
• Hill-climbing, simulated annealing typically work with "complete" states, i.e., all variables assigned
• To apply to CSPs:
• allow states with unsatisfied constraints
• operators reassign variable values
• Variable selection: randomly select any conflicted variable
• Value selection by min-conflicts heuristic:
• choose value that violates the fewest constraints
• i.e., hill-climb with h(n) = total number of violated constraints
Example: 4 Queens• States: 4 queens in 4 columns (44 = 256 states)
• Actions: move queen in column
• Goal test: no attacks
• Evaluation: h(n) = number of attacks
• Given random initial state, can solve n-queens in almost constant time for arbitrary n with high probability (e.g., n = 10,000,000)
44
Distributed Optimization Problem
“How do a set of agents optimize over a set of alternatives that have varying degrees of global quality?”
Examples
• allocating resources
• constructing schedules
• planning activities
Difficulties
• No global control/knowledge
• Localized communication
• Quality guarantees required
• Limited Time
Interesting Research Questions
• How to evaluate performance?
• Cycles, CBR, CCC, etc?
• How much distribution counts? OptAPO?
• Local approximations of optimization
• How to apply DCR to new domains?
• More flexible DCOP? multiply constrained, resource constraints?
• Privacy?45
46
• Common model for automated coordination
• Assign values to variables, subject to constraints
• Problems
• Constraints may be personal or proprietary
• Current algorithms not designed for privacy
• NP-hard!
Privacy in Constraint Optimization
Supply Chain
47
EucalyptusGrower
Transport Company
Transport Company
Corn Farmer
Shipping Company
Gas
Alcohol
Bundled
leavesBundled
corn
DistributorPackedleaves
Bundledcorn
Tanked alcohol
Packedleaves
RetailerWholesaler
Need to coordinate when to send and how much
Algs: Asynchronous Backtracking (ABT)
• Not DCOP, constraint satisfaction
• Agents form communication/priority chain
• Pass down tentative assignments (ok? messages)
• If value inconsistent with higher agents, change
• Pass up violations (nogood messages)
• nogoods causes higher agents to change value
• All happens concurrently
48
Synchronous Branch and Bound (SynchBB)
• Simulate branch and bound in distributed environment
• Messages sent in predefined, sequential order
• First agent sends a partial solution (one variable)
• Next agent adds variable, evaluates
• If < bound (best complete solution cost), send on
• If > bound, try new values
• If all values tried, backtrack
49
Synchronous Iterative Deepening (SynchID)
• Simulates iterative deepening
• First agent picks value, sends it and cost limit (initially 0) to next agent
• Next agent tries to find value under limit
• If succeed, pass on values, limit to next agent
• If fail, backtrack up and increase the limit
50
Adopt (Asynchronous Optimization)
• Agents organized in tree, not chain, constraints between ancestors/descendants not siblings
• When agents receive messages
• choose value with min cost
• send VALUE message to descendants
• send COST message to parents
• send THRESHOLD message to child
51
DPOP (Dynamic Pseudotree)
• Same tree as Adopt, but sequential
• Dynamic programming/variable elimination approach
• Each agent sends only one message up (with constraint information) and then one message down (with value information)
52
Discussion: PageRank and Eigenvalues
• Create matrix whose values aij are the number of backlinks into page i from page j divided by number of forward links of page j.
• a14 = 1 link from page 4/2 total links from page 4 = 1/2
• To get page rank find eigenvector Ax=lambda(x) where lambda =1 (eigenvalue)
53
Discussion: PageRank and Eigenvalues
• To get page rank find eigenvector Ax=lambda(x) where lambda =1 (eigenvalue)
• A [12 4 9 6] = [12 4 9 6]
• All multiples of eigenvectors are eigenvectors (normalize to fraction)
• [12/31 4/31 9/31 6/31]
54
Page Rank
• Have to add in E to fix link problems
• Vector [0.15 0.15 0.15 0.15] for democratic E
• or [x 0 0 0] for specific E
• rank now c(A + E x 1)R’ = R’ still looking for an eigenvector just more complex one
55
Iterating Rank• R0 = [0.15 0.15 0.15 0.15]
• R1 = df R0
• R1 = [0.22 0.05 0.19 0.12]
• d = 0.3 - 0.32
• R1 = R1 + [0.003 0.003 0.003 0.003]
• delta = ||[0.073 -0.097 0.043 0.027]||||v|| = sqrt(v12 + v22 + ... + vn2)
Is PageRank Admissible?
• How does the type of search talked about here connect with the other search algorithms we’ve studied?