Austin Dionne Heuristic Search Under Deadlines – 1 / 56
Master’s Thesis:Heuristic Search Under a Deadline
Austin Dionne
Department of Computer Scienceaustin.dionne at gmail.com
Acknowledgements
Introduction
Related Work
DAS
Conclusion
DDT
Austin Dionne Heuristic Search Under Deadlines – 2 / 56
Thanks to:
■ Wheeler Ruml (Advisor)
■ Jordan T. Thayer (Collaborator)
■ NSF (grant IIS-0812141)
■ DARPA CSSG program (grant N10AP20029)
Introduction
Introduction
■ Heuristic Search
■ Problem Def.
■ Thesis Statement
■ Contributions
Related Work
DAS
Conclusion
DDT
Austin Dionne Heuristic Search Under Deadlines – 3 / 56
Search Is Awesome!
Introduction
■ Heuristic Search
■ Problem Def.
■ Thesis Statement
■ Contributions
Related Work
DAS
Conclusion
DDT
Austin Dionne Heuristic Search Under Deadlines – 4 / 56
Heuristic Search
Introduction
■ Heuristic Search
■ Problem Def.
■ Thesis Statement
■ Contributions
Related Work
DAS
Conclusion
DDT
Austin Dionne Heuristic Search Under Deadlines – 5 / 56
Heuristic Search (Continued)
Introduction
■ Heuristic Search
■ Problem Def.
■ Thesis Statement
■ Contributions
Related Work
DAS
Conclusion
DDT
Austin Dionne Heuristic Search Under Deadlines – 6 / 56
s0 : starting stateexpand(s) : returns list of child states (sc, c)goal(s) : returns true if s is a goal state, false otherwiseg(s) : cost accumulated so far on path from s0 to s
h∗(s) : cost of cheapest solution under sf∗(s) = g(s) + h∗(s) : estimated cost of best solution under sd∗(s) : number of steps to cheapest solution under sh(s), f(s), d(s) : heuristic estimators of true values
d(s) : unbiased estimator of d∗
Problem Definition
Introduction
■ Heuristic Search
■ Problem Def.
■ Thesis Statement
■ Contributions
Related Work
DAS
Conclusion
DDT
Austin Dionne Heuristic Search Under Deadlines – 7 / 56
Given a problem and a limited amount of computation time,find the best solution possible before the deadline.
■ Problem which often occurs in practice
■ The current “best” methods do not directly consider thepresence of a deadline and waste effort.
■ The current “best” methods require off-line tuning foroptimal performance.
Thesis Statement
Introduction
■ Heuristic Search
■ Problem Def.
■ Thesis Statement
■ Contributions
Related Work
DAS
Conclusion
DDT
Austin Dionne Heuristic Search Under Deadlines – 8 / 56
My thesis is that a deadline-cognizant approach which attemptsto expend all available search effort towards a single finalsolution has the potential for outperforming these methodswithout off-line optimization.
Contributions
Introduction
■ Heuristic Search
■ Problem Def.
■ Thesis Statement
■ Contributions
Related Work
DAS
Conclusion
DDT
Austin Dionne Heuristic Search Under Deadlines – 9 / 56
In this thesis we have proposed:
■ Corrected single-step error model for d(s) and h(s)
■ Deadline Aware Search (DAS) which can outperformcurrent approaches
■ Extended single-step error model for calculating d∗ and h∗
distributions on-line
■ Deadline Decision Theoretic Search (DDT) which is a moreflexible and theoretically based algorithm that holds somepromise
Related Work
Introduction
Related Work
■ Related Work■ Related Work(Continued)
■ Related Work(Continued)
■ Current Approach
■ Our Motivation
■ Recap
DAS
Conclusion
DDT
Austin Dionne Heuristic Search Under Deadlines – 10 / 56
Related Work
Introduction
Related Work
■ Related Work■ Related Work(Continued)
■ Related Work(Continued)
■ Current Approach
■ Our Motivation
■ Recap
DAS
Conclusion
DDT
Austin Dionne Heuristic Search Under Deadlines – 11 / 56
We are not the first to attempt to solve this problem...
■ Time Constrained Search (Hiraishi, Ohwada, andMizoguchi 1998)
■ Contract Search (Aine, Chakrabarti, and Kumar 2010)
Neither of these methods work well in practice!
Related Work (Continued)
Introduction
Related Work
■ Related Work■ Related Work(Continued)
■ Related Work(Continued)
■ Current Approach
■ Our Motivation
■ Recap
DAS
Conclusion
DDT
Austin Dionne Heuristic Search Under Deadlines – 12 / 56
Problem with Time Constrained Search:
■ Parameters abound! (ǫupper, ǫlower, ∆w)
■ Important questions without answers:
◆ When (if ever) should we resort open list?
◆ Is a hysteresis necessary for changes in w?
I could not implement a version of this algorithm that workedwell!
Related Work (Continued)
Introduction
Related Work
■ Related Work■ Related Work(Continued)
■ Related Work(Continued)
■ Current Approach
■ Our Motivation
■ Recap
DAS
Conclusion
DDT
Austin Dionne Heuristic Search Under Deadlines – 13 / 56
Problem with Contract Search:
■ Not really applicable to domains with goals at a wide rangeof depths (tiles/gridworld/robots)
■ Takes substantial off-line effort to prepare the algorithmfor a particular domain and deadline
Jordan Thayer implemented this algorithm and it does not workwell!
Currently Accepted Approach
Introduction
Related Work
■ Related Work■ Related Work(Continued)
■ Related Work(Continued)
■ Current Approach
■ Our Motivation
■ Recap
DAS
Conclusion
DDT
Austin Dionne Heuristic Search Under Deadlines – 14 / 56
Anytime Search
■ Search for a suboptimal initial solution relatively quickly
■ Continue searching, finding sequence of improved solutions overtime
■ Eventually converge to optimal
Problems:
1. Wasted effort in finding sequence of mostly unused solutions
2. Based on bounded suboptimal search, which requires parametersettings
■ May not have time for off-line tuning
■ For some domains different deadlines require differentsettings
Our Motivation
Introduction
Related Work
■ Related Work■ Related Work(Continued)
■ Related Work(Continued)
■ Current Approach
■ Our Motivation
■ Recap
DAS
Conclusion
DDT
Austin Dionne Heuristic Search Under Deadlines – 15 / 56
Our desired deadline-aware approach should:
■ Consider the time remaining in ordering state expansion
■ Perform consistently well across a full range deadlines(fractions of a second to minutes)
■ Be parameterless and general
■ Not require significant off-line computation
Recap
Introduction
Related Work
■ Related Work■ Related Work(Continued)
■ Related Work(Continued)
■ Current Approach
■ Our Motivation
■ Recap
DAS
Conclusion
DDT
Austin Dionne Heuristic Search Under Deadlines – 16 / 56
■ Search under deadlines is a difficult and important problem
■ Previously proposed approaches don’t work
■ Currently used approaches are unsatisfying
■ We propose an algorithm (DAS) which can outperformthese methods without the use of off-line tuning
Deadline Aware Search (DAS)
Introduction
Related Work
DAS
■ Motivation
■ Algorithm (1)
■ Vacillation
■ Exp Delay
■ Calc dmax
■ Algorithm (2)
■ Results
■ Results
■ results
■ Conclusion
Conclusion
DDT
Austin Dionne Heuristic Search Under Deadlines – 17 / 56
Motivation
Introduction
Related Work
DAS
■ Motivation
■ Algorithm (1)
■ Vacillation
■ Exp Delay
■ Calc dmax
■ Algorithm (2)
■ Results
■ Results
■ results
■ Conclusion
Conclusion
DDT
Austin Dionne Heuristic Search Under Deadlines – 18 / 56
DAS pursues the best solution path which is reachable withinthe time remaining in the search.
■ Best is defined as minimal f(s)
■ Reachability is a function of an estimate distance to asolution d(s) and the current behavior of the search
DAS: High-Level Algorithm
Introduction
Related Work
DAS
■ Motivation
■ Algorithm (1)
■ Vacillation
■ Exp Delay
■ Calc dmax
■ Algorithm (2)
■ Results
■ Results
■ results
■ Conclusion
Conclusion
DDT
Austin Dionne Heuristic Search Under Deadlines – 19 / 56
While there is time remaining before the deadline:
■ Calculate maximum allowable distance dmax
■ Select node n from open list with minimal f(n)
■ If d(n) ≤ dmax (solution is reachable)
◆ Expand n, add children to open list
■ Otherwise (solution is unreachable)
◆ Add n to pruned list
Search Vacillation
Introduction
Related Work
DAS
■ Motivation
■ Algorithm (1)
■ Vacillation
■ Exp Delay
■ Calc dmax
■ Algorithm (2)
■ Results
■ Results
■ results
■ Conclusion
Conclusion
DDT
Austin Dionne Heuristic Search Under Deadlines – 20 / 56
Error in h(s) produces Search Vacillation.
Expansion Delay
Introduction
Related Work
DAS
■ Motivation
■ Algorithm (1)
■ Vacillation
■ Exp Delay
■ Calc dmax
■ Algorithm (2)
■ Results
■ Results
■ results
■ Conclusion
Conclusion
DDT
Austin Dionne Heuristic Search Under Deadlines – 21 / 56
Expansion Delay
Maintain a running expansion counter during search.
At state expansion, define expansion delay as:
∆e = (current exp counter)− (exp counter at generation)
Expansion Delay
Introduction
Related Work
DAS
■ Motivation
■ Algorithm (1)
■ Vacillation
■ Exp Delay
■ Calc dmax
■ Algorithm (2)
■ Results
■ Results
■ results
■ Conclusion
Conclusion
DDT
Austin Dionne Heuristic Search Under Deadlines – 22 / 56
Use mean expansion delay ∆e to calculate dmax:
dmax =(expansions remaining)
∆e(1)
dmax estimates the expected number of steps that will beexplored down any particular path in the search space.
DAS: High-Level Algorithm
Introduction
Related Work
DAS
■ Motivation
■ Algorithm (1)
■ Vacillation
■ Exp Delay
■ Calc dmax
■ Algorithm (2)
■ Results
■ Results
■ results
■ Conclusion
Conclusion
DDT
Austin Dionne Heuristic Search Under Deadlines – 23 / 56
While there is time remaining before the deadline:
■ Calculate maximum allowable distance dmax
■ Select node n from open list with minimal f(n)
■ If d(n) ≤ dmax (solution is reachable)
◆ Expand n, add children to open list
■ Otherwise (solution is unreachable)
◆ Add n to pruned list
■ If open list is empty
◆ Recover a set of nodes from pruned list with“reachable” solutions
◆ Reset estimate of dmax
DAS: High-Level Algorithm: Search Recovery
Introduction
Related Work
DAS
■ Motivation
■ Algorithm (1)
■ Vacillation
■ Exp Delay
■ Calc dmax
■ Algorithm (2)
■ Results
■ Results
■ results
■ Conclusion
Conclusion
DDT
Austin Dionne Heuristic Search Under Deadlines – 24 / 56
Start again with a set of nodes with “reachable” solutions:
Recap
Introduction
Related Work
DAS
■ Motivation
■ Algorithm (1)
■ Vacillation
■ Exp Delay
■ Calc dmax
■ Algorithm (2)
■ Results
■ Results
■ results
■ Conclusion
Conclusion
DDT
Austin Dionne Heuristic Search Under Deadlines – 25 / 56
■ Search under deadlines is a difficult and important problem
■ Previously proposed approaches don’t work
■ Currently used approaches are unsatisfying
■ We propose an algorithm (DAS) which can outperformthese methods without the use of off-line tuning
◆ Uses expansion delay to measure search vacillation
◆ Estimates a “reachable” solution distance and prunesnodes
Empirical Evaluation: Domains
Introduction
Related Work
DAS
■ Motivation
■ Algorithm (1)
■ Vacillation
■ Exp Delay
■ Calc dmax
■ Algorithm (2)
■ Results
■ Results
■ results
■ Conclusion
Conclusion
DDT
Austin Dionne Heuristic Search Under Deadlines – 26 / 56
Empirical Evaluation: Methodology
Introduction
Related Work
DAS
■ Motivation
■ Algorithm (1)
■ Vacillation
■ Exp Delay
■ Calc dmax
■ Algorithm (2)
■ Results
■ Results
■ results
■ Conclusion
Conclusion
DDT
Austin Dionne Heuristic Search Under Deadlines – 27 / 56
■ All algorithms run “Speedier” first to obtain incumbentsolution
■ Anytime algorithms tested with variety of settings: 1.2, 1.5,3.0, 6.0, 10.0 (top two performing are displayed)
■ Show results for: ARA*, RWA*, CS, DAS
■ Deadlines are on a log scale (fractions of second up tominutes)
■ Algorithms compared by solution quality
solution quality = (best solution cost) / (achieved cost)
Results: 15-Puzzle
Introduction
Related Work
DAS
■ Motivation
■ Algorithm (1)
■ Vacillation
■ Exp Delay
■ Calc dmax
■ Algorithm (2)
■ Results
■ Results
■ results
■ Conclusion
Conclusion
DDT
Austin Dionne Heuristic Search Under Deadlines – 28 / 56
Results: Weighted 15-Puzzle
Introduction
Related Work
DAS
■ Motivation
■ Algorithm (1)
■ Vacillation
■ Exp Delay
■ Calc dmax
■ Algorithm (2)
■ Results
■ Results
■ results
■ Conclusion
Conclusion
DDT
Austin Dionne Heuristic Search Under Deadlines – 29 / 56
Results: 4-Way 2000x1200 Unit-Cost Gridworld (p=0.35)
Introduction
Related Work
DAS
■ Motivation
■ Algorithm (1)
■ Vacillation
■ Exp Delay
■ Calc dmax
■ Algorithm (2)
■ Results
■ Results
■ results
■ Conclusion
Conclusion
DDT
Austin Dionne Heuristic Search Under Deadlines – 30 / 56
Results: 4-Way 2000x1200 Life-Cost Gridworld (p=0.35)
Introduction
Related Work
DAS
■ Motivation
■ Algorithm (1)
■ Vacillation
■ Exp Delay
■ Calc dmax
■ Algorithm (2)
■ Results
■ Results
■ results
■ Conclusion
Conclusion
DDT
Austin Dionne Heuristic Search Under Deadlines – 31 / 56
Results: Dynamic Robot Navigation
Introduction
Related Work
DAS
■ Motivation
■ Algorithm (1)
■ Vacillation
■ Exp Delay
■ Calc dmax
■ Algorithm (2)
■ Results
■ Results
■ results
■ Conclusion
Conclusion
DDT
Austin Dionne Heuristic Search Under Deadlines – 32 / 56
Results: Overall
Introduction
Related Work
DAS
■ Motivation
■ Algorithm (1)
■ Vacillation
■ Exp Delay
■ Calc dmax
■ Algorithm (2)
■ Results
■ Results
■ results
■ Conclusion
Conclusion
DDT
Austin Dionne Heuristic Search Under Deadlines – 33 / 56
DAS Conclusion
Introduction
Related Work
DAS
■ Motivation
■ Algorithm (1)
■ Vacillation
■ Exp Delay
■ Calc dmax
■ Algorithm (2)
■ Results
■ Results
■ results
■ Conclusion
Conclusion
DDT
Austin Dionne Heuristic Search Under Deadlines – 34 / 56
■ Parameterless
■ Returns optimal solutions for sufficiently large deadlines
■ Competitive with or outperforms ARA* for variety ofdomains
DAS illustrates that an improved deadline-aware approach canbe constructed!
Conclusion
Introduction
Related Work
DAS
Conclusion
■ Thesis Recap
■ Contributions
DDT
Austin Dionne Heuristic Search Under Deadlines – 35 / 56
Thesis Recap
Introduction
Related Work
DAS
Conclusion
■ Thesis Recap
■ Contributions
DDT
Austin Dionne Heuristic Search Under Deadlines – 36 / 56
■ Search under deadlines is a difficult and important problem
■ Previously proposed approaches don’t work
■ Currently used approaches are unsatisfying
My thesis is that a deadline-cognizant approach which attemptsto expend all available search effort towards a single finalsolution has the potential for outperforming these methodswithout off-line optimization.
Contributions
Introduction
Related Work
DAS
Conclusion
■ Thesis Recap
■ Contributions
DDT
Austin Dionne Heuristic Search Under Deadlines – 37 / 56
In this thesis we have proposed:
■ Corrected single-step error model for d(s) and h(s)
■ Deadline Aware Search (DAS) which can outperformcurrent approaches
■ Extended single-step error model for calculating d∗ and h∗
distributions on-line
■ Deadline Decision Theoretic Search (DDT) which is a moreflexible and theoretically based algorithm that holds somepromise
DAS illustrates that improvement is possible!
Back-up Slides
Introduction
Related Work
DAS
Conclusion
Back-up Slides
■ DAS Pseudo-Code
■ d(s)
DDT
Austin Dionne Heuristic Search Under Deadlines – 19 / 56
DAS Pseudo-Code
Introduction
Related Work
DAS
Conclusion
Back-up Slides
■ DAS Pseudo-Code
■ d(s)
DDT
Austin Dionne Heuristic Search Under Deadlines – 20 / 56
Deadline Aware Search(starting state, deadline)1. open ← {starting state}2. pruned ← {}3. incumbent ← NULL4. while (time) < (deadline) and open is non-empty5. dmax ← calculate d max()6. s← remove state from open with minimal f(s)7. if s is a goal and is better than incumbent8. incumbent ← s
9. else if d(s) < dmax
10. for each child s′ of state s11. add s′ to open12. else13. add s to pruned14. if open is empty16. recover pruned states(open, pruned)17. return incumbent
DAS Pseudo-Code (Continued)
Introduction
Related Work
DAS
Conclusion
Back-up Slides
■ DAS Pseudo-Code
■ d(s)
DDT
Austin Dionne Heuristic Search Under Deadlines – 21 / 56
Recover Pruned States(open, pruned)18. exp ← estimated expansions remaining19. while exp > 0 and pruned is non-empty loop20. s← remove state from pruned with minimal f(s)21. add s to open
23. exp = exp −d(s)
Intention is to replace only a “reachable” set of nodes.
Correcting d(s): Single-Step Error Model
Introduction
Related Work
DAS
Conclusion
Back-up Slides
■ DAS Pseudo-Code
■ d(s)
DDT
Austin Dionne Heuristic Search Under Deadlines – 22 / 56
Single-Step Error Model first introduced in BUGSY (Ruml andDo 2007):
ed = d(soc)− (d(s)− 1)
eh = h(soc)− (h(s)− c(s, soc))
Using average errors ed and eh:
d(s) = d(s) · (1 + ed)
h(s) = h(s) + eh · d(s)
soc is selected as the childstate of s with minimal f
Correcting d(s): Single-Step Error Model (Continued)
Introduction
Related Work
DAS
Conclusion
Back-up Slides
■ DAS Pseudo-Code
■ d(s)
DDT
Austin Dionne Heuristic Search Under Deadlines – 23 / 56
Our new proposed model is more correct:
ed = d(soc)− (d(s)− 1)
eh = h(soc)− (h(s)− c(s, soc))
Using average errors ed and eh:
d(s) =d(s)
1− ed
h(s) = h(s) + eh · d(s)
soc is selected as the childstate of s with minimal fexcluding the parent of s
Time Constrained Search
Introduction
Related Work
DAS
Conclusion
Back-up Slides
■ DAS Pseudo-Code
■ d(s)
DDT
Austin Dionne Heuristic Search Under Deadlines – 24 / 56
Performs dynamically weighted search on f ′(s) = g(s) + h(s) · w
■ Deadline denoted as T
■ Time elapsed denoted as t
■ Define D = h(s0)
■ Define “desired average velocity” as V = D/T
■ Define “effective velocity” as v = (D − hmin)/t
■ If v > V + ǫupper, increase w by ∆w
■ If v < V − ǫlower, decrease w by ∆w
Contract Search
Introduction
Related Work
DAS
Conclusion
Back-up Slides
■ DAS Pseudo-Code
■ d(s)
DDT
Austin Dionne Heuristic Search Under Deadlines – 25 / 56
Performs beam-like search, limiting the number of expansionsdone at each level of the search tree.
■ Off-line computation of k(depth) for each level of searchtree
■ Authors propose models for estimating optimal k(depth)using dynamic programming
■ Once k(depth) expansions are made a particular level, thatlevel is disabled
Problems:
■ Not applicable to domains where solutions may reside at awide range of depths
■ It takes substantial off-line effort to compute k(depth)
Deadline Decision Theoretic Search (DDT)
Introduction
Related Work
DAS
Conclusion
DDT
■ Motivation
■ EC(s)
■ Algorithm
■ Off-line Model
■ On-line Model■ Results: 4-Way2000x1200Unit-Cost Gridworld(p=0.35)
■ Future Work
Austin Dionne Heuristic Search Under Deadlines – 26 / 56
Motivation
Introduction
Related Work
DAS
Conclusion
DDT
■ Motivation
■ EC(s)
■ Algorithm
■ Off-line Model
■ On-line Model■ Results: 4-Way2000x1200Unit-Cost Gridworld(p=0.35)
■ Future Work
Austin Dionne Heuristic Search Under Deadlines – 27 / 56
Searching under a deadline involves a great deal of uncertainty.
Expected Solution Cost EC(s)
Introduction
Related Work
DAS
Conclusion
DDT
■ Motivation
■ EC(s)
■ Algorithm
■ Off-line Model
■ On-line Model■ Results: 4-Way2000x1200Unit-Cost Gridworld(p=0.35)
■ Future Work
Austin Dionne Heuristic Search Under Deadlines – 28 / 56
fdef : cost of default/incumbent solution
fexp : expected value of f∗(s) (if better than incumbent)
Pgoal : probability of finding solution under s before deadline
Pimp : probability that cost of new solution found under simproves on incumbent
Algorithm
Introduction
Related Work
DAS
Conclusion
DDT
■ Motivation
■ EC(s)
■ Algorithm
■ Off-line Model
■ On-line Model■ Results: 4-Way2000x1200Unit-Cost Gridworld(p=0.35)
■ Future Work
Austin Dionne Heuristic Search Under Deadlines – 29 / 56
DDT Search(initial, deadline, default solution)1. open ← {initial}2. incumbent ← default solution3. while (time elapsed) < (deadline) loop5. s← remove state from open with minimum EC(s)6. if s is a goal and is better than incumbent7. incumbent ← s8. recalculate EC(s) for all s in open and resort8. otherwise9. recalculate EC(s)5. s′ ← peek next state from open with minimum EC(s′)10. if EC(s) > EC(s′)11. re-insert s into open12. otherwise13. expand s, adding child states to open14. return incumbent
Off-line Model
Introduction
Related Work
DAS
Conclusion
DDT
■ Motivation
■ EC(s)
■ Algorithm
■ Off-line Model
■ On-line Model■ Results: 4-Way2000x1200Unit-Cost Gridworld(p=0.35)
■ Future Work
Austin Dionne Heuristic Search Under Deadlines – 30 / 56
Pgoal = P (d∗ ≤ dmax) (2)
Pimp = P (f∗ ≤ fdef ) (3)
Pimp · fexp =
∫ fdefault
f=0P (f∗ = f) · f (4)
Off-line Model (Continued)
Introduction
Related Work
DAS
Conclusion
DDT
■ Motivation
■ EC(s)
■ Algorithm
■ Off-line Model
■ On-line Model■ Results: 4-Way2000x1200Unit-Cost Gridworld(p=0.35)
■ Future Work
Austin Dionne Heuristic Search Under Deadlines – 31 / 56
Measurements on 4-Way 2000x1200 Unit-Cost Gridworld
h(s)
Heuristic Error (h-h*)/h*
Unit Grids - Cumulative HED
0
500
1000
1500
2000
2500
3000
-1 -0.8 -0.6 -0.4 -0.2 00
0.2
0.4
0.6
0.8
1
150 200 250 300 350 400
Occ
urre
nces
h*
Unit Grids - HED (h=200)
0
0.5
1
1.5
2
600 800 1000 1200 1400 1600
Occ
urre
nces
h*
Unit Grids - HED (h=750)
0
0.5
1
1.5
2
2.5
1400 1600 1800 2000 2200 2400 2600 2800 3000
Occ
urre
nces
h*
Unit Grids - HED (h=1500)
Currently assume h∗ and d∗ are independant.
On-line Model
Introduction
Related Work
DAS
Conclusion
DDT
■ Motivation
■ EC(s)
■ Algorithm
■ Off-line Model
■ On-line Model■ Results: 4-Way2000x1200Unit-Cost Gridworld(p=0.35)
■ Future Work
Austin Dionne Heuristic Search Under Deadlines – 32 / 56
Extends one-step error model to support calculation of heuristicdistribution functions.
Assume one-step errors areindependant identically dis-tributed random variables. Seefigure for one-step errors in 4-Way Unit-Cost Gridworld.
Then mean one step errors along individual paths are normallydistributed according to the Central Limit Theorem with meanand variance:
µǫd = µǫd (5)
σ2ǫd
=σ2ǫd· (1− µǫd)
d(s)(6)
On-line Model (Continued)
Introduction
Related Work
DAS
Conclusion
DDT
■ Motivation
■ EC(s)
■ Algorithm
■ Off-line Model
■ On-line Model■ Results: 4-Way2000x1200Unit-Cost Gridworld(p=0.35)
■ Future Work
Austin Dionne Heuristic Search Under Deadlines – 33 / 56
Using Equations from slide 17 and the assumption that ǫd and ǫhare normally distributed, we can calculate the CDF for d∗(s):
cdfd∗(x) =1
2·
1 + ERF
(x−d(s)
x− µǫ)
(√2 · σ
2ǫ ·(1−µǫ)d(s) )
(7)
For a given value of d∗ we can assume f∗ is normally distributedwith mean and variance:
µf∗ = g(s) + h(s) + µǫh · d∗(s) (8)
σ2f∗ = σ2
ǫh· (d∗(s)) (9)
Details can be found in thesis document.
On-line Model (Continued)
Introduction
Related Work
DAS
Conclusion
DDT
■ Motivation
■ EC(s)
■ Algorithm
■ Off-line Model
■ On-line Model■ Results: 4-Way2000x1200Unit-Cost Gridworld(p=0.35)
■ Future Work
Austin Dionne Heuristic Search Under Deadlines – 34 / 56
Using CDF for d∗ and Gaussian PDF for calculatingP (f∗ = f |d∗ = d) we can calculate EC(s) as follows:
Pimp = P (f∗ ≤ fdefault|d∗ = d)
EC(s|d∗ = d) =
(∫ fdefault
f=0P (f∗ = f |d∗ = d) · f
)+ (1− Pimp) · fdef
EC(s) =
(∫ dmax
d=0EC(s|d∗ = d)
)+ (1− Pgoal) · fdef
On-line Model Verification
Introduction
Related Work
DAS
Conclusion
DDT
■ Motivation
■ EC(s)
■ Algorithm
■ Off-line Model
■ On-line Model■ Results: 4-Way2000x1200Unit-Cost Gridworld(p=0.35)
■ Future Work
Austin Dionne Heuristic Search Under Deadlines – 35 / 56
Monte Carlo analysis performed on d∗(s) model using heuristicerror from 4-Way Unit-Cost Gridworld.
Model of d∗(s) is accurate unless ǫd
Results: 4-Way 2000x1200 Unit-Cost Gridworld (p=0.35)
Introduction
Related Work
DAS
Conclusion
DDT
■ Motivation
■ EC(s)
■ Algorithm
■ Off-line Model
■ On-line Model■ Results: 4-Way2000x1200Unit-Cost Gridworld(p=0.35)
■ Future Work
Austin Dionne Heuristic Search Under Deadlines – 36 / 56
Even in optimistic case DDT does not outperform DAS!
Future Work
Introduction
Related Work
DAS
Conclusion
DDT
■ Motivation
■ EC(s)
■ Algorithm
■ Off-line Model
■ On-line Model■ Results: 4-Way2000x1200Unit-Cost Gridworld(p=0.35)
■ Future Work
Austin Dionne Heuristic Search Under Deadlines – 37 / 56
■ More empirical evaluation of DAS and DDT
■ Evaluate other methods of calculating d(s) for DAS
■ Evaluate other methods of calculating dmax for DAS/DDT
■ Evaluate accuracy of probabilistic one-step error model
■ Modify Real-Time search to apply to Contract Search