Societies of Hill-Climbers

• Before presenting SoHCs let’s first talk about Hill-Climbing in general. For this lecture we will confine ourselves to binary-coded representations; however, it is possible for us to extend the concept of hill-climbing to real-coded representations as well.

Deterministic Hill-Climbing

• In our discussion on Hill-Climbing lets try to solve two function optimization problems (Maximization):– F1(x) = x2, and– F2(x) = Σ xi

• We will apply two types of hill-climbing– Steepest Ascent and– Next Ascent.

Deterministic Hill-Climbing

• Hill-Climbers are local search techniques. Therefore it is important to define the neighbor of the local search.

• For our demonstration, all CSs that are a hamming distance of 1 away from the parent (indivdual being mutated) are considered neighbors.

Deterministic Steepest AscentHill-Climbing

• Consider the representation for CS for F1 and F2 to be composed of 5-bits.

• Deterministic Steepest Ascent Hill-Climbing works as follows:– Step1: Randomly generate and evaluate, b, the current

best individual– Step2: Generate and evaluate ALL of the neighbors of

b. Let Neighbors = the set of λ neighbors.– Step3: b = best-of(b Neighbors) – Step4: if the the fitness of b does not improve either

terminate the search or goto Step1; Else goto Step2.

Deterministic Steepest Ascent Hill-Climbing

• Let’s use DSAHC to solve F1– Let b = (01001)– Neighbors = {(11001), (00001), (01101),

01011), (01000)}– The best of b Neighbors is (11001)

therefore,– Let b = (11001)

Deterministic Steepest Ascent Hill-Climbing

• Continuing (second cycle)– Let b = (11001)– Neighbors = {(01001), (10001), (11101),

11011), (11000)}– The best of b Neighbors is (11101)

therefore,– Let b = (11101)

Deterministic Steepest Ascent Hill-Climbing

• Continuing (third cycle)– Let b = (11101)

– Neighbors = {(01101), (10101), (11001), 11111), (11100)}

– The best of b Neighbors is (11111) therefore,

– Let b = (11111)

• On the fourth cycle, we cannot find a candidate solution that is better than b so we will terminate our search.

Deterministic Steepest Ascent Hill-Climbing

• For some problems the defined neighbor may be too large to exhaustively search.

• When this is the case, a Stochastic Steepest Ascent Hill-Climber may be preferred.

• In a Stochastic Steepest Ascent Hill-Climber, Neighbors is a set of a user-specified number of neighbors are generated from the neighborhood.

• b = best-of(b Neighbors)

Deterministic Next AscentHill-Climbing

• Deterministic Next Ascent Hill-Climbing works as follows:– Step1: Randomly generate and evaluate, b, the current best

individualFor k = 1 to Number of Neighbors Do– Step2: Generate and evaluate the neighbors of b one at a time.– Generate and evaluate Neighbork – Step3: b = best-of(b,Neighbork) End (For)– Step4: if the the fitness of b does not improve either terminate the

search or goto Step1; Else goto Step2.

Deterministic Next Ascent Hill-Climbing

• Let’s use DNAHC to solve F1– Let b = (01001)– Neighbor = (11001)– The best of b and Neighbor is (11001)

therefore,– Let b = (11001)

Deterministic Next Ascent Hill-Climbing

• Continuing (second cycle)– Let b = (11001)

– Neighbor = (01001)

– The best of b and Neighbor is (11001)

– Neighbor = (10001)

– The best of b and Neighbor is (11001)

– Neighbor = (11101)

– The best of b and Neighbor is (11101)

– Let b = (11101), goto third cycle

Deterministic Next Ascent Hill-Climbing

• Continuing (third cycle)– Let b = (11101)– Neighbor = (01101)– The best of b and Neighbor is (11101) – Neighbor = (10101)– The best of b and Neighbor is (11101)– Neighbor = (11001)– The best of b and Neighbor is (11101)– Neighbor = (11111)– The best of b and Neighbor is (11111)– Let b = (11111), goto fourth cycle

Deterministic Next Ascent Hill-Climbing

• Continuing (fourth cycle)– Let b = (11111)– Neighbor = (01111)– The best of b and Neighbor is (11111) – Neighbor = (10111)– The best of b and Neighbor is (11111)– Neighbor = (11101)– The best of b and Neighbor is (11111)– Neighbor = (11110)– The best of b and Neighbor is (11111)– Let b = (11111), Terminate Search

Stochastic Next Ascent Hill-Climbing

• In order to do deterministic hill-climbing one must also specify the order in which the neighbors are to be generated and evaluated. This order can dramatically affect the performance of hill-climbing.

• In a Stochastic Next Ascent Hill-Climber, neighbors are randomly selected from the neighborhood.

Some Hill-Climbing Questions

• Of the two type of hill-climbers which will perform better on F2?

• For what types of problems will Steepest Ascent Hill-Climbing outperform Next Ascent Hill-Climbing?

• For what types of problems will Next Ascent Hill-Climbing outperform Steepest Ascent Hill-Climbing?

Societies of Hill-Climbers

• A Society of Hill-Climbers (SoHCs) as proposed by Sebag & Schoenauer (1997) can be seen as being composed of:– A (μ+λ)-EC, and – A repoussoir (repeller) which is an adaptive

external memory structure that is used to ‘push’ the search away from known sub-optimal solutions.

Societies of Hill-Climbers

• Societies of Hill-Climbers, since they evolve an adaptive external memory structure, can be consider a swarm (Swarm Intelligence Method) similar to:– Particle Swarm Optimization [P-Vectors],– Ant Colony Optimization [Pheromone

Matrix], – Cultural Algorithms [Belief Space], – Tabu Search [Tabu List]

Societies of Hill-Climbers

• Sebag & Schoenaur describe two types of Hill-Climber Societies:– Natural Societies <μ, λ, M>

• μ represents the number of hill-climbers (candidate solutions),

• λ represents the number of offspring created (λ/μ offspring for each parent), and

• M represents the number of bits to be mutated to create an offspring.

Societies of Hill-Climbers

– Historical Societies <μ, λ, M, R, α, T>• R the percentage of the worst fit CSs that will be

used to create the AvgWorst in order to update the repeller,

• α represents the decay rate for the components of R, and

• T is the tournament size for selecting a variable (gene) of the parent to mutate.

• The only sensitive parameter for historical societies is M.

Societies of Hill-ClimbersHistorical Societies

• Repeller is a real-valued vector of n where n is the number of variables.

• Each component of Repeller is initially assigned a value of 0.5.

• The Repeller is updated as follows:• Repeller = (1-α) Repeller + αAvgWorst.

Societies of Hill-Climbers:Historical Societies

• An Example SoHCProcedure SoHC{ t = 0; Initialize Pop(t); Evaluate Pop(t); Intialize Repoussor(t); While (Not Done) {

Offspring(t) = Procreate(Pop(t),Repoussor(t));Evaluate(Offspring(t));Pop(t+1) = Best_μ_of(Pop(t),Offspring(t));AvgWorst = Worst_R%_of(Pop(t),Offspring(t));Repoussoir(t+1) = (1-α)Repoussoir(t) + αAvgWorst; t = t + 1;

}

Societies of Hill-Climbers:Historical Societies

• Let’s try an example by hand for F1!• <μ=2, λ=4, M=1, R=50%, α=0.5, T=2>Pop(0):

Person 1: 01010 Fit: 100Person 2: 00011 Fit: 9

Offspring(0):Child 1: 00010 Fit: 4Child 2: 01110 Fit: 196Child 3: 10011 Fit: 361Child 4: 00001 Fit: 1

Societies of Hill-Climbers:Historical Societies

Person 2: 00011 Fit: 9

Child 1: 00010 Fit: 4

Child 4: 00001 Fit: 1

----------------------------------

AvgWorst: <0.00, 0.00, 0.00, 0.67, 0.67>

Repoussoir(0): <0.50, 0.50, 0.50, 0.50, 0.50>

Repoussoir(1): <0.25, 0.25, 0.25, 0.58, 0.58>

Societies of Hill-Climbers:Historical Societies

Repoussoir(1): <0.25, 0.25, 0.25, 0.58, 0.58>

Pop(1):

Person 1: 01110 Fit: 196

Person 2: 10011 Fit: 361

Offspring(1):

Child 1: 11011 Fit: 729

Child 2: 10111 Fit: 529

Child 3: 11110 Fit: 900

Child 4: 01111 Fit: 225

Societies of Hill-Climbers:Historical Societies

Person 1: 01110 Fit: 196

Person 2: 10011 Fit: 361

Child 4: 01111 Fit: 225

------------------------------------

AvgWorst: <0.33, 0.67, 0.67, 1.00, 0.67>

Repoussoir(1): <0.25, 0.25, 0.25, 0.58, 0.58>

Repoussoir(2): <0.29, 0.46, 0.46, 0.79, 0.62>

Societies of Hill-Climbers:Historical Societies

Repoussoir(2): <0.29, 0.46, 0.46, 0.79, 0.62> Pop(2):

Person 1: 11011 Fit: 729

Person 2: 11110 Fit: 900

Offspring(1):

Child 1: _____ Fit: ___

Child 2: _____ Fit: ___

Child 3: _____ Fit: ___

Child 4: _____ Fit: ___

Societies of Hill-Climbers:Historical Societies

• Will the Repoussoir work all of the time?• How can we develop more efficient SoHCs?• What have we learned from our brief introduction to Swarm Intelligence?

– Some swarms are ‘pullers’• Ant Swarms, • Ant Colony & Particle Swarm Optimizers, • Cultural Algorithms

– Some swarms are ‘pushers’• Societies of Hill-Climbers• Tabu Search

• Should we really rely on pushing or pulling exclusively?• During what parts of the evolutionary process would ‘pushing’ be more

beneficial?• During what parts of the evolutionary process would ‘pulling’ be more

beneficial?