LEARNING THE HEURISTIC DISTRIBUTION BY AN EVOLUTIONARY HYPER-HEURISTIC
Edmund Burke, Nam Pham, Rong Qu
Outline
Motivation Constructive hyper-heuristics Heuristic representation Our evolutionary hyper-heuristic
approach An application to graph colouring An application to examination
timetabling Conclusions and Future Work
2
Nam Pham
Motivation
Hyper-heuristics are at higher generality Applicable for different problems without
many modifications Hyper-heuristics may learn effective and
ineffective heuristics It is observed by human that a heuristic
may be better at giving decisions at different stages during the search
Example: ‘largest degree’ and ‘saturation degree’ for graph colouring
3
Nam Pham
Constructive Hyper-heuristics Constructive heuristics
Consist of step-by-step decisions Build a complete solution from scratch
Different problems have different set of decisions at each step
Constructive hyper-heuristics Are heuristics Intelligently choose different constructive
heuristics for different situations
4
Nam Pham
A Heuristic Representation
Sequences of low-level heuristics Each element represents a set of heuristics That set of heuristics provides a number of
decisions to construct one step towards a complete solution
Elements in a sequence are applied consecutively until a complete solution is obtained
We use this representation for problems in this talk.
5
Nam Pham
Our Evolutionary Hyper-heuristic An evolutionary algorithm on the high level
search Chromosomes: sequence representation Divide a sequence into a number of intervals
Learn the appearance frequency of simple low-level heuristics in different intervals of sequences Maintain a list of heuristic probability distribution Update the list by using fittest sequences
With a good probability distribution, we are likely to generate fitter sequences
6
Nam Pham
Our Evolutionary Hyper-heuristic An evolutionary algorithm on the high level
search Chromosomes: sequence representation Divide a sequence into a number of intervals
Learn the appearance frequency of simple low-level heuristics in different intervals of sequences Maintain a list of heuristic probability distribution Update the list by using fittest sequences
With a good probability distribution, we are likely to generate fitter sequences
7
Nam Pham
A Sequence of Heuristics
H1
H1
H1
H1
H2
H3
H1
H1
H2
H3
H2
8
Nam Pham
Intervals of a sequence
H1
H1
H1
H1
H2
H3
H1
H1
H2
H3
H2
9
Nam Pham
Our Evolutionary Hyper-heuristic An evolutionary algorithm on the high level
search Chromosomes: sequence representation Divide a sequence into a number of intervals
Learn the appearance frequency of simple low-level heuristics in different intervals of sequences Maintain a list of heuristic probability distribution Update the list by using fittest sequences
With a good probability distribution, we are likely to generate fitter sequences
10
Nam Pham
List of heuristic probability distribution
H1
H1
H1
H1
H2
H3
H1
H1
H2
H3
H2
H1
100%
H2
0%
H3
0%
H1
33.3%
H2
33.3%
H3
33.3%
H1
0%
H2
66.6%
H3
33.3%
11
Nam Pham
Update the list
H1
H1
H1
H1
H2
H3
H1
H1
H2
H3
H2
H1
100%
H2
0%
H3
0%
H1
33.3%
H2
33.3%
H3
33.3%
H1
0%
H2
66.6%
H3
33.3%
H1
H1
H1
H1
H2
H3
H1
H1
H2
H3
H2
H1
H1
H1
H1
H2
H3
H1
H1
H2
H3
H2
Average Probabilities
12
Nam Pham
Our Evolutionary Hyper-heuristic An evolutionary algorithm on the high level
search Chromosomes: sequence representation Divide a sequence into a number of intervals
Learn the appearance frequency of simple low-level heuristics in different intervals of sequences Maintain a list of heuristic probability distribution Update the list by using fittest sequences
With a good probability distribution, we are likely to generate fitter sequences
13
Nam Pham
Our Hyper-heuristic Outline
Initialise heuristic probability distribution such that all low-level heuristics have the same chance to appear in every interval
Do Sort the evaluation function of all chromosomes Update the list of heuristic distribution using p =
10% of the best fit chromosomes Generate chromosomes for the next generation.
Until stopping condition is met
14
Nam Pham
The Creation of New Chromosomes Keep chromosomes that always select a
particular low-level heuristic Keep the best 10% from the previous generation 10% of the population will be generated
randomly 10% of the population will be generated based
on the probabilities in the list of heuristic distribution
The remaining chromosomes are generated by using a crossover operator Fitter chromosomes are more likely to be selected
15
Nam Pham
Crossover Operator
H1
H1
H1
H1
H2
H3
H1
H1
H2
H3
H2
H1
100%
H2
0%
H3
0%
H1
33.3%
H2
33.3%
H3
33.3%
H1
0%
H2
66.6%
H3
33.3%
Chromosome 1
16
Nam Pham
Crossover Operator
H1
H2
H1
H1
H2
H1
H1
H1
H2
H3
H2
H1
66.6%
H2
33.3%
H3
0%
H1
66.6%
H2
33.3%
H3
0%
H1
0%
H2
66.6%
H3
33.3%
Chromosome 2
17
Nam Pham
Crossover Operator
H1
100%H2
0%H3
0%
H1
33.3%H
233.3
%H3
33.3%
H1
0%H2
66.6%H
333.3
%
H1
66.6%H
233.3
%H3
0%
H1
66.6%H
233.3
%H3
0%
H1
0%H2
66.6%H
333.3
%
Average Probabilities
H1
83.3%H
216.6
%H3
0%
H1
50%H2
33.3%H
30%
H1
0%H2
66.6%H
333.3
%
18
Nam Pham
Generality of our Hyper-heuristic Not designed with only a problem in mind No problem dependent information
Learn from the frequency of low-level heuristics in sequences
Applicable to different problems We test on the graph colouring problem and
the examination timetabling problem The only modifications are the pool of low-
level heuristics and the fitness evaluation for sequences
19
Nam Pham
An Application to Graph Colouring Problem description Pool of low-level heuristics Fitness evaluation for sequences Computational results
20
Nam Pham
Problem description
NP-hard combinatorial optimisation problem (Papadimitriou and Steiglitz, 1982)
Assigning colours to vertices such that adjacent vertices receive different colours
Find a colouring that requires as few colours as possible
21
Nam Pham
Low-level heuristics
Sequence representation Each element in a sequence consists of a vertex-
selection(VS) heuristic and a colour-selection(CS) heuristic
We currently use the same colour-selection heuristic and concern only vertex-selection heuristics
Strategy: a vertex that is likely to cause trouble if deferred until later should be coloured first
VS1
CS
VS1
CS
VS1
CS
VS1
CS
VS2
CS
VS3
CS
VS1
CS
VS1
CS
VS2
CS
VS3
CS
VS2
CS
22
Nam Pham
Pool of low-level heuristics
SD: Least Saturation Degree: Vertices are ordered increasingly by the number of available colours during the colouring process
LD: Largest Degree: Vertices are ordered decreasingly by the number of neighbours in the graph
LCD: Largest Coloured Degree: Vertices are ordered decreasingly by the number of coloured neighbours
SD2, SD3, LD2, LD3, LCD2, LCD3
23
Nam Pham
Fitness Evaluation for Sequences We currently use the evaluation of a
colouring as the sequence evaluation the number of colours required to obtain a
non-conflict colouring breaking ties by the distance to a colouring
using one colour less than the current colouring
24
Nam Pham
Computational Results
Each interval consists of 10 decisions to select heuristics
For each instance, we evolved 5000 generations
Compare our hyper-heuristic with results obtained from other approaches for the Toronto benchmark (Qu et al., 2008)
The probability distribution of heuristics are reported
25
Nam Pham
Computational Results
Instances
Vertices
Best single
heuristic
RandomSequence
s
Our Hyper-
heuristic
Max Clique
Best Reported
Car91 682 30 29 28 23 28
Car92 543 29 28 27 24 28
Ear83 190 23 22 22 21 22
Hec92 81 19 17 17 17 17
Kfu93 461 19 19 19 19 19
Lse91 381 17 17 17 17 17
Rye92 482 22 21 21 21 21
Sta83 139 13 13 13 13 13
Tre92 261 23 20 20 20 20
Uta92 622 31 30 29 26 30
Ute92 184 10 10 10 10 10
Yor83 181 21 19 18 18 19
26
Nam Pham
Heuristic Probability Distribution
27
Nam Pham
Effective and Ineffective Heuristics
28
Nam Pham
Effective and Ineffective Heuristics
29
Nam Pham
An Application to Exam Timetabling Problem description Pool of low-level heuristics Fitness evaluation for sequences Computational results
30
Nam Pham
Problem description
A generalisation of the graph colouring problem Assigning exam to timeslots such that
students do not have to sit two exams at the same time the timetable with the best spread of exams for
students is preferred Using graph colouring to model exam timetabling
vertices represent exams colours represent timeslots
Find a non-conflict timetable that has the smallest penalty for the spread of exams for students
31
Nam Pham
Low-level heuristics
Sequence representation Each element in a sequence consists of a exam-
selection(ES) heuristic and a timeslot-selection(TS) heuristic
We currently use the same timeslot-selection heuristic and concern only exam-selection heuristics
Use the same strategy to select exams as for the graph colouring problem
ES1
TS
ES1
TS
ES1
TS
ES1
TS
ES2
TS
ES3
TS
ES1
TS
ES1
TS
ES2
TS
ES3
TS
ES2
TS
32
Nam Pham
Pool of low-level heuristics
SD: Least Saturation Degree: Exams are ordered increasingly by the number of available timeslots
LD: Largest Degree: Exams are ordered decreasingly by the number of neighbours in the graph
LCD: Largest Coloured Degree: Exams are ordered decreasingly by the number of already assigned neighbours
LE: Largest Enrolment: Exams are ordered decreasingly by the enrolment
LWD: Largest Weighted Degree: Exams are ordered decreasingly by the total number of students in conflict
SD2, SD3, LD2, LD3, LCD2, LCD3, LE2, LE3, LWD2, LWD3
33
Nam Pham
Fitness Evaluation for Sequences If a feasible timetable is found
The evaluation is the average penalty of students sitting in exams close together
If a feasible timetable cannot be found Record the first position, i, that causes infeasibility We use the evaluation as follows
where l is the length of the sequence and controls the degree of involvement for infeasible solutions
),(.)( ilvaluationbestKnownEsf
34
Nam Pham
Computational Results
Each interval consists of 10 decisions to select heuristics
For each instance, we evolved 5000 generations
Compare our hyper-heuristic with results obtained from other constructive approaches for the Toronto examination timetabling benchmark (Qu et al., 2008)
The probability distribution of heuristics are reported
35
Nam Pham
Computational Results
Instances Vertices
OurHyper-
heuristic
Best ReportedConstructi
ve
PercentageDifference
car91 I 682 5.14 4.97 +3.42%
car92 I 543 4.21 4.32 -2.54%
ear83 I 190 35.87 36.16 -0.8%
hec92 I 81 11.65 10.8 +7.87%
kfu93 I 461 14.59 14.0 +4.21%
lse91 381 11.1 10.5 +5.71%
rye92 482 9.55 7.3 +30.8%
sta83 I 139 157.95 158.19 -0.15%
tre92 I 261 8.44 8.38 +0.71%
uta92 I 622 3.43 3.36 +2.08%
ute92 184 27.04 25.8 +4.8%
yor83 I 181 39.07 39.8 -1.83%
36
Nam Pham
Heuristic Probability Distribution
37
Nam Pham
Conclusions and Future Work This evolutionary hyper-heuristic is re-usable for
different problems and requires little problem specific information We will experiment on other problems in future work
Produce solutions of acceptable quality. Some of them are competitive.
The probability distribution of heuristics shows how such heuristics work at different stages of the search
We can learn whether a heuristic is effective or ineffective in different situations
38
Nam Pham
Reference
PAPADIMITRIOU, C. H. & STEIGLITZ, K. (1982) Combinatorial Optimization: Algorithms and Complexity, Prentice-Hall.
QU, R., BURKE, E. K., MCCOLLUM, B., MERLOT, L. T. G. & LEE, S. Y. (2008) A Survey of Search Methodologies and Automated Approaches for Examination Timetabling. to appear in Journal of Scheduling.
39
Nam Pham
Questions/Comments
Thank you!
40
Nam Pham