11

Adaptive Problem-Solving for Adaptive Problem-Solving for Large-Scale Scheduling Large-Scale Scheduling Problems: A Case StudyProblems: A Case Study

byby Jonathan Gratch and Steve ChienJonathan Gratch and Steve ChienPublished in JAIR, 1996Published in JAIR, 1996

EARG presentation EARG presentation OctOct 3, 200 3, 20088by Frank Hutterby Frank Hutter

22

OverviewOverview

Problem domainProblem domain Cool: scheduling for the deep space networkCool: scheduling for the deep space network

Scheduling algorithmScheduling algorithm Branch & Bound with a Lagrangian relaxation at each Branch & Bound with a Lagrangian relaxation at each

search nodesearch node

Adaptive Problem SolvingAdaptive Problem Solving Automatic Parameter Tuning by Local SearchAutomatic Parameter Tuning by Local Search Already contains many good ideas, 12 years ago!Already contains many good ideas, 12 years ago!

33

Domain: Scheduling for the Domain: Scheduling for the Deep Space NetworkDeep Space Network

Collection of ground-based radio antennasCollection of ground-based radio antennas Maintain communication with research satellites and Maintain communication with research satellites and

deep space probesdeep space probes

NASA’s Jet Propulsion Laboratory (JPL): NASA’s Jet Propulsion Laboratory (JPL): automate scheduling of 26-meter subnetautomate scheduling of 26-meter subnet Three 26-meter antennasThree 26-meter antennas Goldstone, CA, USAGoldstone, CA, USA Canberra, AustraliaCanberra, Australia Madrid, SpainMadrid, Spain

44

Scheduling problemScheduling problem“When should which antenna talk to which satellite?” “When should which antenna talk to which satellite?”

Project requirementsProject requirements Number of communication events per periodNumber of communication events per period Duration of communication eventsDuration of communication events Allowable gap between communication eventsAllowable gap between communication events

E.g. Nimbus-7 (meteorogical satellite): needs at least E.g. Nimbus-7 (meteorogical satellite): needs at least four 15-minute slots per day, not more than 5 hours apartfour 15-minute slots per day, not more than 5 hours apart

Antenna constraintsAntenna constraints Only one communication at onceOnly one communication at once Antenna can only communicate with satellites in viewAntenna can only communicate with satellites in view Routine maintenance Routine maintenance antenna offline antenna offline

55

Problem formulationProblem formulation

0-1 integer linear programming formulation0-1 integer linear programming formulation Time periods: 0-1 integer variables (in/out)Time periods: 0-1 integer variables (in/out) Typical problem: 700 variables, 1300 constraintsTypical problem: 700 variables, 1300 constraints

Scheduling has to be fastScheduling has to be fast So human user can try “what if” scenariosSo human user can try “what if” scenarios ““For these reasons, the focus of development is upon For these reasons, the focus of development is upon

heuristic techniques that do not necessarily uncover heuristic techniques that do not necessarily uncover the optimal schedule, but rather produce adequate the optimal schedule, but rather produce adequate schedules quickly.”schedules quickly.”

Alas, they still don’t use local search ;-)Alas, they still don’t use local search ;-)

66

Scheduling algorithmScheduling algorithm

Branch and Bound (“Split-and-prune”)Branch and Bound (“Split-and-prune”)

At each node: At each node: arc consistency (check all constraints arc consistency (check all constraints

containing time period just committed)containing time period just committed) Lagrangian relaxation: each antenna by itselfLagrangian relaxation: each antenna by itself

Can be solved in linear time (dynamic Can be solved in linear time (dynamic programming for each antenna to get “non-programming for each antenna to get “non-exclusive sequence of time periods with maximum exclusive sequence of time periods with maximum cumulative weight”)cumulative weight”)

77

Lagrangian relaxationLagrangian relaxation

Relax project constraints, penalize violation by weight uRelax project constraints, penalize violation by weight ujj ; ; weight search for best vector uweight search for best vector u

88

The LR-26 schedulerThe LR-26 scheduler

99

Search Algorithm ParametersSearch Algorithm Parameters

Constraint ordering: Constraint ordering: ““Choose a constraint that maximally constrains the rest of Choose a constraint that maximally constrains the rest of

the search space”the search space” 99 heuristics, same heuristics, same 99 as secondary tie-breakers as secondary tie-breakers

Value ordering: “maximize the number of options Value ordering: “maximize the number of options available for future assignments”available for future assignments” 55 heuristics implemented heuristics implemented

Weight search (for weight vector u)Weight search (for weight vector u) 44 methods implemented methods implemented

Refinement methodsRefinement methods 22 options: Standard B&B vs. (A=x fails, then try B=y options: Standard B&B vs. (A=x fails, then try B=y

instead of A=1-x --- does this have a name??)instead of A=1-x --- does this have a name??)

1010

Problem distributionProblem distribution

Not many problem instances availableNot many problem instances available Syntactic manipulation of set of real problemsSyntactic manipulation of set of real problems Yields 6,600 problem instancesYields 6,600 problem instances

Only use subset of these 6,600 instancesOnly use subset of these 6,600 instances Some generated instances seemed much harder than Some generated instances seemed much harder than

original instancesoriginal instances Discard “intractable” instances (original or generated)Discard “intractable” instances (original or generated)

Intractable: instances taking longer than 5 Intractable: instances taking longer than 5 minutesminutes

1111

Determination of Resource BoundDetermination of Resource BoundOnly 12% of problems unsolved in 5 minutes were solved in an hourOnly 12% of problems unsolved in 5 minutes were solved in an hourReference to statistical analysis for that factorReference to statistical analysis for that factor should read that in EARG (Etzioni & Etzioni, 1994)should read that in EARG (Etzioni & Etzioni, 1994)

1212

Adaptive Problem Solving: ApproachesAdaptive Problem Solving: ApproachesSyntactic approachSyntactic approach

Transform into more efficient form, using only Transform into more efficient form, using only syntactic structuresyntactic structure Recognize structural properties that influence effectiveness of different heuristic Recognize structural properties that influence effectiveness of different heuristic

methodsmethodsBig lookup table, specifying heuristic to useBig lookup table, specifying heuristic to useSomewhat similar to SATzilla Somewhat similar to SATzilla Lin should look into it Lin should look into it

(I think: includes newer research on symmetry breaking, etc)(I think: includes newer research on symmetry breaking, etc)

Generative approachGenerative approach Generate new heuristics based on partial runs of solver Generate new heuristics based on partial runs of solver focus on inefficiencies in focus on inefficiencies in

previous runsprevious runs ““Often learning is within an instance and does not generalize to distributions of Often learning is within an instance and does not generalize to distributions of

problems”problems”

Statistical approachStatistical approach Explicitly reason about performance of different heuristics across distribution of Explicitly reason about performance of different heuristics across distribution of

problemsproblems Often: statistical generate-and-test approachesOften: statistical generate-and-test approaches Widely applicable (domains, utility functions)Widely applicable (domains, utility functions) Computationally expensive; local optima (Computationally expensive; local optima ( ParamILS) ParamILS)

1313

Adaptive Problem Solving: Adaptive Problem Solving: ComposerComposer

Statistical approachStatistical approach Generate-and-test hillclimbingGenerate-and-test hillclimbing When evaluating a move:When evaluating a move:

Perform runs with neighbourPerform runs with neighbourCollect differences in performanceCollect differences in performancePerform test to see if mean(differences) < 0 or >0Perform test to see if mean(differences) < 0 or >0Test assumes Normal distribution of differencesTest assumes Normal distribution of differences

Terminate in first local optimumTerminate in first local optimum Evaluation:Evaluation:

On large set of test instances (1000)On large set of test instances (1000)

1414

Meta-Control Knowledge in Meta-Control Knowledge in Composer: Layered SearchComposer: Layered Search

Order parameters by their importanceOrder parameters by their importance First only allow move in the first level, then allow move in First only allow move in the first level, then allow move in

the second level, etcthe second level, etc Not sure whether they iterateNot sure whether they iterate

LevelsLevels Level 0: weight search methodLevel 0: weight search method Level 1: Refinement methodLevel 1: Refinement method Level 2: Secondary refinement, value orderingLevel 2: Secondary refinement, value ordering Level 3: Primary constraint orderingLevel 3: Primary constraint ordering

(this comes last since they strongly believed their manual one was (this comes last since they strongly believed their manual one was best – it was indeed chosen)best – it was indeed chosen)

1515

Composer pseudo codeComposer pseudo code

1616

Empirical evaluationEmpirical evaluation

Setting of Composer parametersSetting of Composer parameters = 0.05, n= 0.05, n00 = 15 (empirically determined) = 15 (empirically determined)

Training set: 300 problem instancesTraining set: 300 problem instancesTest set: 1000 problem instancesTest set: 1000 problem instances They say “independent”, but I don’t think disjointThey say “independent”, but I don’t think disjoint

Stochasticity from drawing instances at randomStochasticity from drawing instances at random Estimate Estimate expectedexpected performance as average over multiple performance as average over multiple

experimental trialsexperimental trials But don’t tell us how many trials they did But don’t tell us how many trials they did

Measure performance every 20 Measure performance every 20 samplessamples

1717

Experimental results: subsetExperimental results: subset

1818

Experimental result: full setExperimental result: full set

1919

Kernel density estimate of Kernel density estimate of strategies: subsetstrategies: subset

2020

Kernel density estimate of Kernel density estimate of strategies: full setstrategies: full set

2121

My view of their approachMy view of their approach

Some very good ideas, already 12 years agoSome very good ideas, already 12 years ago Proper use of training/test setProper use of training/test set Statistical test for move is interestingStatistical test for move is interesting

Problems I seeProblems I see If a move neither decreases nor increases expected If a move neither decreases nor increases expected

utility, the statistical test can force “an infinite number” utility, the statistical test can force “an infinite number” of evaluationsof evaluations

Even if this just decides between two poor configurationsEven if this just decides between two poor configurations Stuck in local minimaStuck in local minima Never re-using instances? Once they’re out of Never re-using instances? Once they’re out of

instances, they stop (also still a little unclear)instances, they stop (also still a little unclear)