a study of simulated annealing variants ana pereira polytechnic institute of braganca, portugal ...

A study of simulated annealing variants

Ana PereiraPolytechnic Institute of Braganca, Portugal

email: [email protected]

Edite FernandesUniversity of Minho, Braga, Portugal


SEIO’04 Cádiz, October 25-29, 2004

2A study of simulated annealing variants

Outline

• Motivation

• Simulated annealing algorithm

• Simulated annealing variants

• Computational results – Characterization of the presented variants

• Computational results – Comparison of the presented variants

• Simulated annealing

• Conclusions


Motivation

We aim to find the global solution of the nonlinear optimization problem

Numerical methods

Deterministic methods

Stochastic methodsExamples: Multistart, clustering, genetic algorithms, simulated annealing ….


Simulated annealing

In 1953, Metropolis proposed an algorithm to simulate the behavior of physical systems in the presence of a heat bath.

In 1983, based on ideas of Metropolis algorithm, Kirkpatrick, Gelatt and Vecchi, and in 1985 Cerny, proposed the simulated annealing (SA) algorithm to solve combinatorial optimization problems.

In 1986, Bohachevsky, Johnson and Stein applied the SA algorithm to solve continuous optimization problems.

Since then, the SA algorithm has been subject to various modifications and has been applied in many areas such as graph coloring, circuit design, data analysis, image reconstruction, biology, …


Simulated annealing

Advantages:

Easily implemented.

Can be applied to any optimization problem.

Does not use derivative information.

Does not require specific conditions on the objective function.

Asymptotically converges to a global maximum.

Disadvantages:

Requires a great number of function evaluations.

Many authors have been proposing variants of the SA algorithm.


Given an initial approximation , a control parameter and the number of iterations with the same control parameter

while stopping criterion is not reached do

Generate a new candidate point y

Analyze the acceptance criterion

end

Update

Reduce the control parameter

end

Simulated annealing algorithm

kcN

0c0t

1 kcj Nfor to do

kc

1 k k

kcN



Generation of a new candidate point

The new point is found using the current approximation, , and the generating probability density function, .

Acceptance criterion

is the acceptance function and it represents the probability of accepting the point y when is the current point. The acceptance criterion has the following form

The most used acceptance function is

kt k

kt y

f c

kk

t yA c

kt

1 if , 0,1

otherwise

kk

k t y

k

y A ct U

t

min 1,

k

k

k

g t g y

k ct y

A c e



Reduction of the control parameter

The function is called the control parameter and must be a decreasing function that verifies

Stopping criterion

The stopping criterion is based on the idea that the algorithm should terminate when no changes occur.

We propose that the algorithm stops when successive approximations to a global maximum are similar, i.e, the algorithm stops if the following condition is verified for successive iterations

where represents the previous approximation to an optimum value.

kc

lim 0

k

kc

*N* * antf f

*antf


Simulated annealing variants

Standard SA variant (SSA)

Generation of a new point

Reduction of the control

parameter

Length of the chain: constant

, 1,1 kiy t U

1 , 0,1 k kc c

kcN


Simulated annealing variantsCorana SA variant (CSA)



parameter

Length of the chain: constant

, 1,1 ,

is the euclidian vector and

k k k ki i i i

i

y t d e d U

e

1 , 0,1 k kc c

kcN

0.6* 1 * 0.60.4

0.4 0.6

0.40.41 *

0.4

ki i

k ki i

ki

i

rV r

r

rrV

where is constant throughout the processiV


ASA variant

Main difference :

There are two control parameters: one associated with the generation of the new points and another associated with the acceptance function.

And it is possible the redefinition of the control parameters.



ASA variant


Redefinition of the control parameters


parameters

Length of the chain: 1


2 1

.

Consider 0,1 , is given by

1 1sgn 1 1 .2 i

i

k k

ki

u

k ki Gk

G

y t b a

u U

u cc

1

1

0

0

1, for 1 and

1.

i i

nGi

i i

nA

G G

kkG G

A A

kkA A

k ki n

c c e

k k

c c e

kcN


SALO variant

Similar to ASA algorithm except on generation of a new point.



Obtain doing a slight perturbation on t ky

y LocalSearch y


ASALO variant

Based on ASA and SALO algorithms.

Generation of a new point Like ASA algorithm.

If y is infeasible, applies the reflection technique proposed by Romeijn and Smith

If the new point y is accepted


if if

if

i i i i i

i i i i i

i i i i i

a a y y ar y y a y b

b y b y b

y LocalSearch r y


SSA

CSA

Computational results – Characterization of the presented variants

0 and .cCrucial parameters:


ASA

SALO

ASALO

Computational results – Characterization of the presented variants

0 and .c Crucial parameters:


Computational results – Comparison of the presented variants

Test functions n NFE NAP

B 2 1000 311 -0.3978875 -0.3978874

GP 2 1000 306 -3.0000004 -3.0000001

R2 2 26015 20745 -0.0394671 -0.0049240

R4 4 64262 4131 -0.0362470 -0.0230065

Me3 3 1000 122

H3 3 2068 374 3.8627819 3.8627821

Ra4 4 2680 174

ASA

algorithm

ASALO

algorithm

*g*mg

82.8 10

79.4 10

105.3 10

88.3 10

Test functions n NFE NAP

B 2 15531 284 -0.3978874 -0.3978874

GP 2 15944 285 -3 -3

R2 2 16671 294

R4 4 40923 577 -0.0265422 -0.0049288

Me3 3 5717 98

H3 3 15237 260 3.8627815 3.8627821

Ra4 4 10293 143

*g*mg

94.0 10

61.6 10

112.6 10

85.5 10

62.8 10 88.8 10


Conclusions

SSA variant requires a large number of function evaluations.

SSA and CSA variants recognize more than one solution, when the problem has more than one global maximum.

CSA is the variant that has more accepted points.

ASA variant requires a reduced number of function evaluations.


Conclusions

ASA variant did not converge to a global maximum in some runs of some problems.

CSA and ASALO variants provide better approximations to the global maximum.

SALO and ASALO variants have a small number of accepted points.

CSA and ASALO variants did not converge to a global maximum only in one run of a problem.


A study of simulated annealing variants

Ana PereiraPolytechnic Institute of Braganca, Portugal


Edite FernandesUniversity of Minho, Braga, Portugal


a study of simulated annealing variants ana pereira polytechnic institute of braganca, portugal ...

Documents