8
Varying Dimensional Particle Swarm Optimization Yanjun Yan and Lisa Ann Osadciw Department of Electrical Engineering and Computer Science Syracuse University Syracuse, NY 13244 Email: {yayan,laosadci}@syr.edu Abstract —A new alg ori thm, var ying dimens ion al par tic le swarm opt imization (VD-PS O), is pr oposed for ent iti es with varying-dimens ional comp onent s and each component assumes continuous-valued parameters. Such problems are distinct from current benc hmar k prob lems where the dime nsio n of the par - ti cles is fixed. One well-studi ed appl ic at ion of VD-PSO is prob abil ity dens ity esti matio n by Gauss ian Mixt ure Mode ls. A particle in VD-PSO includes a discrete number as the number of components and a set of real-valued component parameters. The numbe r of compo nent s vari es accordin g to a rando m scheme, which dictates how many sets of components remain or expand. The component parameters are matched up supporting a linked update . Thr ee other methods, the last two of whi ch ar e also proposed by us as intuitive attempts to solve the varying dimen- sional problems, are compared with VD-PSO: 1. binary-headered PSO, 2. exhaustive PSO, 3. discrete-headered PSO. Simulations on known dat a wit h spe cifi ed component s sho w that VD- PSO provides a competitive density estimation to the exhaustive PSO, but spends the least wall-clock-time among all algorithms, almost 12% of the exh aus tiv e PSO, whi le the bin ary -he ade re d PSO or disc rete -head ere d PSO do not achi eve simi lar perf orma nce, becau se the binary-he ader ed PSO is adver sely affected by the dummy parameter s, and the disc rete -head ere d PSO make s too many dimension variations before the parameters are well tuned. The new VD-PSO algorithm is a viable and efficient solution to varying dimensional optimization problems. Index T erms —Varying Dimension, Particle Swarm Optimiza- tion, Distribution Estimation, Gaussian Mixture Model I. MOTIVATION AND I NTRODUCTION Par ticle Swar m Opti miza tion (PSO) [1] has successf ully solv ed var ious opti miza tion prob lems [2]. The sear ch space and the dimension of the particles are traditionally determined a priori and fixed throughout the search. However, there are pro ble ms tha t a va lid solut ion take s onl y a sub set of the potential dimensions. The dimension of a solution is defined as the number of components that constitute a complete and fea sib le sol uti on, and eac h component assumes the same structure, with either a (set of) discrete number(s), a (set of) continuous number(s), or a (set of) hybrid structure(s). For instance , Kamath, Ye and Osadc iw [3] impl emen ted a bina ry-he adere d PSO for airc raft inte rrog atio n sche dulin g, where the interrogations can happen between many pairs of airc raft s and thei r near by grou nd trans cei vers , but an opti - mal schedul ing is a subs et of all poten tial interr ogat ions to ensure an accurate detection, mitigate the resource allocation collisions and minimize the transmission costs. In this binary- hea der ed PSO, eac h par tic le is constructed wit h maximal dimens ion . A bin ary hea der ass oci ate d wit h eac h par tic le differentiates the used parameters from the dummy parameters. The binary-headered PSO isolates the solution dimensions, and it is suitable for problems where the parameters at different dime nsion s do not aff ect each other in part icle movements. The interactions between different dimensions appear in fitness evaluations. Anoth er example of the var ying -dimensi onal prob lems is attribute selection for protein classification by a discrete PSO alg ori thm pro pos ed by Cor rea, Fre ita s and Joh nson [4] . In att rib ute sel ect ion , the dis cre te va lues are the ind ice s of attributes (up to 443 in this case) and they are unordered. The number of attributes in each particle is initialized randomly to be different, but the dimension of a particle does not change over iterations. Correa et al. propose to modify the discrete val ues with in a part icle (the indices of selec ted attr ibut es) following the probabilities enhanced by the overlap between the current particle and attractors. This procedure works well for unordered attribute selection. Howe ver , in prob abil ity densi ty estimation probl ems, the “att rib utes ” of poten tial comp onent s are conti nuous, which in val idat es the propo rtio nal like liho od selection method by Cor rea , et al. Ano the r dif fer ence is tha t the valu es of the comp onen t para mete rs are cont inuou s and their magnitud es are order ed. Bas ed on the order ing of the magni tud es, the para mete rs are conn ected cross dime nsion s, and the linkage affects the movements of the particles. Motivated by the need to select a varying number of compo- nents with continuous-valued parameters, varying dimensional part icle swarm optimiz atio n (VD-P SO) is propo sed. In VD- PSO, a par tic le inc lud es a discre te number of compon ent s and eac h component con tai ns a col lec tio n of par ame ter s. The dimens ion s of the par tic les are ran dom, and the y va ry with cert ain prob ability during iter atio ns. The para mete rs in different components are linked based on their values. VD-PSO als o dif fers from Nic hin g [5] or Stretchin g [6] based PSO algorithms, which are utilized to solve multi-modal problems. In multi-modal problems, the searching space is still fixed and common to all particles, although there are multiple optimal solutions and the particles may be clustered or directed to search a partial region. The particles in VD-PSO covers a varying subset of the potential solution space (each part is a component), and the fitness evaluation is on the union of the components altogether . We compa re VD-PS O with three other viable algo rith ms bas ed on PSO to sol ve the dimens ion va ryi ng pro ble ms.
