![Swarm Intelligence Variants of Firefly Algorithm
Variants of Firefly Algorithm
More than 2000 papers about firefly algorithm since 2008. Its literature is
dramatically expanding (Fister et al., 2013)
[http://www.sciencedirect.com/science/article/pii/S2210650213000461 ]
Variants for specific applications:
Continuous optimization, mixed integer programming, ...
Discrete firefly algorithm for scheduling, travelling-salesman problem,
combinatorial optimization ...
Image processing and compression ...
Clustering, classifications and feature selection ...
Chaotic firefly algorithm ...
Hybrid firefly algorithm with other algorithms ...
Multiobjective firefly algorithm.
Xin-She Yang Nature-Inspired Optimization Algorithms Middlesex University London 14 / 44](https://image.slidesharecdn.com/sibgrapi2015yangslidesprintout-160726174112/85/Nature-Inspired-Optimization-Algorithms-14-320.jpg)
![Swarm Intelligence
The normal distribution
p(x; µ, σ2
) =
1
σ
√
2π
exp[−
(x − µ)2
2σ2
], −∞ < x < ∞.
Isotropic Random Walks
SN =
N
i=1
Xi = X1 + ... + XN = SN−1 + XN,
s
Xin-She Yang Nature-Inspired Optimization Algorithms Middlesex University London 18 / 44](https://image.slidesharecdn.com/sibgrapi2015yangslidesprintout-160726174112/85/Nature-Inspired-Optimization-Algorithms-18-320.jpg)
![Swarm Intelligence Bat Algorithm
Bat Algorithm
Acoustics of bat echolocation
λ =
v
f
∼ 2 mm to 14 mm.
Rules used in the bat algorithm (updating equations):
fi = fmin + (fmax − fmin)β, β ∈ [0, 1],
vt+1
i = vt
i + (xt
i − x∗)fi , xt+1
i = xt
i + vt
i .
x∗=best solution found so far.
Variations of Loudness and Pulse Rate
At+1
i ← αAt
i , α ∈ (0, 1], rt+1
i = r0
i [1 − exp(−γt)].
Such variations are linked with the updating equations by a stochastic
switch (by comparing with a uniformly distributed random number),
controlling the balance between exploration and exploitation.
Xin-She Yang Nature-Inspired Optimization Algorithms Middlesex University London 22 / 44](https://image.slidesharecdn.com/sibgrapi2015yangslidesprintout-160726174112/85/Nature-Inspired-Optimization-Algorithms-22-320.jpg)
![Applications Data Mining: Feature Selection (Example 8)
Data Mining: Feature Selection (Example 8)
Bat algorithm and cuckoo search are very competitive (Papa et al, 2013; 2014).
Also effective at dimension reduction (in terms of number of key features).
Dataset # samples #features #classes
Diabetes 768 8 2
DNA 2000 180 3
Mushrooms 8124 112 2
OPF BA CS FA ICS PSO0
10
20
30
40
50
60
70
80
Accuracy[%]
61 61 59
61 61 6160 61 61 62 6263 62 62 62 61
Mushrooms
Baseline
Binary
Sigmoid
Hyperbolic Tangent
Xin-She Yang Nature-Inspired Optimization Algorithms Middlesex University London 36 / 44](https://image.slidesharecdn.com/sibgrapi2015yangslidesprintout-160726174112/85/Nature-Inspired-Optimization-Algorithms-36-320.jpg)
Nature-Inspired Optimization Algorithms
- 1. Nature-Inspired Optimization Algorithms Xin-She Yang Middlesex University London Xin-She Yang Nature-Inspired Optimization Algorithms Middlesex University London 1 / 44
- 2. Live in Nature and Inspired by Nature At the Gates of Nature and Inspired by Nature Xin-She Yang Nature-Inspired Optimization Algorithms Middlesex University London 2 / 44
- 3. The Essence of an Algorithm The Essence of an Algorithm The Essence of an Algorithm Essence of an Optimization Algorithm To move to a new, better point x(t+1) from an existing location x(t). x1 x2 x(t) x(t+1) ? Population-based algorithms use multiple, interacting paths. Different algorithms Different strategies/approaches in generating these moves! Xin-She Yang Nature-Inspired Optimization Algorithms Middlesex University London 3 / 44
- 4. The Essence of an Algorithm The Essence of an Algorithm The Essence of an Algorithm In essence, an algorithm can be written (mathematically) as xt+1 = A(xt , α), For any given xt, the algorithm will generate a new solution xt+1. Functional or a Dynamical System? We can view the above equation as an iterative functional (with fixed points), or a dynamical system (with attractors), or a self-organization system (with organized structures), or a Markov chain (e.g., Markov chain Monte Carlo). The behavoir of the system (algorithm) can be controlled by A and α. For details, please refer to: Xin-She Yang et al., Swarm Intelligence and Bio-Inspired Computation: Theory and Applications, Elsevier, (2013). Xin-She Yang Nature-Inspired Optimization Algorithms Middlesex University London 4 / 44
- 5. The Essence of an Algorithm What’s Wrong with Traditional Algorithms? What’s Wrong with Traditional Algorithms? Some issues with traditional optimization algorithms such as the Newton-Raphson Method, pattern search, quadratic programming, etc. Traditional algorithms are mostly local search, thus cannot guarantee global optimality (except for linear programming and convex optimization). Results often depend on the initial starting points (except linear and convex problems). Methods tend to be problem-specific (e.g., k-opt, branch and bound). Cannot deal with highly nonlinear optimization problems efficiently. Struggle to cope problems with discontinuity. Xin-She Yang Nature-Inspired Optimization Algorithms Middlesex University London 5 / 44
- 6. The Essence of an Algorithm Nature-Inspired Optimization Algorithms Nature-Inspired Optimization Algorithms Almost all nature-inspired metaheuristic optimization algorithms (heuristic or metaheuristic algorithms, as a global optimizer) strive to Increase the probability of finding the global optimality (as a global search tool); Solve a wider class of problems (treating them as a black-box); Try to deal with highly nonlinear problems, problems with discontinuity, and even NP-hard combinatorial problems; Draw design inspiration from Nature. Xin-She Yang Nature-Inspired Optimization Algorithms Middlesex University London 6 / 44
- 7. Nature-Inspired Algorithms Nature-Inspired Optimization Algorithms Nature-Inspired Optimization Algorithms Genetic algorithms (1960s/1970s), evolutionary strategy (Rechenberg & Swefel 1960s), evolutionary programming (Fogel et al. 1960s). Simulated annealing (Kirkpatrick et al. 1983), Tabu search (Glover 1980s), ant colony optimization (Dorigo 1992), genetic programming (Koza 1992), particle swarm optimization (Kennedy & Eberhart 1995), differential evolution (Storn & Price 1996/1997), harmony search (Geem et al. 2001), honeybee algorithm (Nakrani & Tovey 2004), artificial bee colony (Karaboga, 2005). Firefly algorithm (Yang 2008), cuckoo search (Yang & Deb 2009), bat algorithm (Yang, 2010), flower pollination algorithm (2012), ... For details, please refer to: Xin-She Yang, Nature-Inspired Optimization Algorithms, Elsevier, (2014). Xin-She Yang Nature-Inspired Optimization Algorithms Middlesex University London 7 / 44
- 8. Swarm Intelligence Seeking Inspiration in Nature: Swarm Intelligence Seeking Inspiration in Nature: Swarm Intelligence Swarm Intelligence Ants, bees, birds, fish ... Simple rules lead to complex behaviour. Swarming Starlings Xin-She Yang Nature-Inspired Optimization Algorithms Middlesex University London 8 / 44
- 9. Swarm Intelligence PSO PSO Particle Swarm Optimization (Kennedy and Eberhart, 1995) xi g∗ xj vt+1 i = vt i + αǫ1(g∗ − xt i ) + βǫ2(x∗ i − xt i ), xt+1 i = xt i + vt+1 i . α, β = learning parameters, ǫ1, ǫ2=random numbers. Xin-She Yang Nature-Inspired Optimization Algorithms Middlesex University London 9 / 44
- 10. Swarm Intelligence PSO Demo and Disadvantages PSO Demo and Disadvantages The updating equations in PSO form a linear system: xi vi t+1 = 1 1 −(αǫ1 + βǫ2) 1 xi vi t + 0 αǫ1g∗ + βǫ2x∗ i . PSO Demo Premature convergence Xin-She Yang Nature-Inspired Optimization Algorithms Middlesex University London 10 / 44
- 11. Swarm Intelligence Firefly Algorithm Firefly Algorithm Firefly Video Firefly Behaviour and Idealization (Yang, 2008) Fireflies are unisex and brightness varies with distance. Less bright ones will be attracted to bright ones. If no brighter firefly can be seen, a firefly will move randomly. xt+1 i = xt i + β0e−γr2 ij (xj − xi ) + α ǫt i . ⋄ The objective landscape maps to a light-based landscape, and fireflies swarm into the brightest points/regions. ⋄ There is no g∗, therefore, there is no leader. Also, a highly nonlinear iterative system, so subdivision into multiswarms is possible. Xin-She Yang Nature-Inspired Optimization Algorithms Middlesex University London 11 / 44
- 12. Swarm Intelligence FA Demo and Advantages FA Demo and Advantages Paths of fireflies for 3D Rosenbrock function (Husselmann, 2014). Xin-She Yang Nature-Inspired Optimization Algorithms Middlesex University London 12 / 44
- 13. Swarm Intelligence Why is FA so efficient? Why is FA so efficient? FA Demo Multimodal optimality Advantages of Firefly Algorithm Automatically subdivide the whole population into subgroups, and each subgroup swarms around a local mode/optimum. Control modes/ranges by varying γ. Control randomization by tuning parameters such as α. Suitable for multimodal, nonlinear, global optimization problems. Xin-She Yang Nature-Inspired Optimization Algorithms Middlesex University London 13 / 44
- 14. Swarm Intelligence Variants of Firefly Algorithm Variants of Firefly Algorithm More than 2000 papers about firefly algorithm since 2008. Its literature is dramatically expanding (Fister et al., 2013) [http://www.sciencedirect.com/science/article/pii/S2210650213000461 ] Variants for specific applications: Continuous optimization, mixed integer programming, ... Discrete firefly algorithm for scheduling, travelling-salesman problem, combinatorial optimization ... Image processing and compression ... Clustering, classifications and feature selection ... Chaotic firefly algorithm ... Hybrid firefly algorithm with other algorithms ... Multiobjective firefly algorithm. Xin-She Yang Nature-Inspired Optimization Algorithms Middlesex University London 14 / 44
- 15. Swarm Intelligence Cuckoo Breeding Behaviour Cuckoo Breeding Behaviour Evolutionary Advantages Dumps eggs in the nests of host birds and let these host birds raise their chicks, so minimum efforts and maximum success. Cuckoo Video (BBC) Xin-She Yang Nature-Inspired Optimization Algorithms Middlesex University London 15 / 44
- 16. Swarm Intelligence Cuckoo Search (Yang and Deb, 2009) Cuckoo Search (Yang and Deb, 2009) Cuckoo Behaviour and Idealization Each cuckoo lays one egg (solution) at a time, and dumps its egg in a randomly chosen nest. The best nests with high-quality eggs (solutions) will carry out to the next generation. The egg laid by a cuckoo can be discovered by the host bird with a probability pa and a nest will then be built. Xin-She Yang Nature-Inspired Optimization Algorithms Middlesex University London 16 / 44
- 17. Swarm Intelligence Cuckoo Search Cuckoo Search Local random walk: xt+1 i = xt i + s ⊗ H(pa − ǫ) ⊗ (xt j − xt k). [xi , xj , xk are 3 different solutions, H(u) is a Heaviside function, ǫ is a random number drawn from a uniform distribution, and s is the step size. Global random walk via L´evy flights: xt+1 i = xt i + αL(s, λ), L(s, λ) = λΓ(λ) sin(πλ/2) π 1 s1+λ , (s ≫ s0). Generation of new moves by L´evy flights, random walk and elitism. Xin-She Yang Nature-Inspired Optimization Algorithms Middlesex University London 17 / 44
- 18. Swarm Intelligence The normal distribution p(x; µ, σ2 ) = 1 σ √ 2π exp[− (x − µ)2 2σ2 ], −∞ < x < ∞. Isotropic Random Walks SN = N i=1 Xi = X1 + ... + XN = SN−1 + XN, s Xin-She Yang Nature-Inspired Optimization Algorithms Middlesex University London 18 / 44
- 19. Swarm Intelligence L´evy Distribution and L´evy Flights L´evy Distribution and L´evy Flights L(s) = 1 π ∞ 0 cos(τs)e−α τγ dτ, (0 < γ ≤ 2), = γ 2π exp − γ 2(s − µ) 1 (s − µ)3/2 , 0 < µ < s < ∞ (for steps µ > 0). s Self-similar, multi-scale, global random walks. Xin-She Yang Nature-Inspired Optimization Algorithms Middlesex University London 19 / 44
- 20. Swarm Intelligence CS Demo: Highly Efficient! CS Demo: Highly Efficient! In essence, CS is a multiscale global optimizer. More than 1000 papers about cuckoo search since 2009. Its literature is dramatically expanding. CS Demo Efficient search with a focus Xin-She Yang Nature-Inspired Optimization Algorithms Middlesex University London 20 / 44
- 21. Swarm Intelligence Bat Algorithm (Yang, 2010) Bat Algorithm (Yang, 2010) BBC Video Microbats use echolocation for hunting Ultrasonic short pulses as loud as 110dB with a short period of 5 to 20 ms. Frequencies of 25 kHz to 100 kHz. Speed up pulse-emission rate and increase loudness when homing at a prey. Xin-She Yang Nature-Inspired Optimization Algorithms Middlesex University London 21 / 44
- 22. Swarm Intelligence Bat Algorithm Bat Algorithm Acoustics of bat echolocation λ = v f ∼ 2 mm to 14 mm. Rules used in the bat algorithm (updating equations): fi = fmin + (fmax − fmin)β, β ∈ [0, 1], vt+1 i = vt i + (xt i − x∗)fi , xt+1 i = xt i + vt i . x∗=best solution found so far. Variations of Loudness and Pulse Rate At+1 i ← αAt i , α ∈ (0, 1], rt+1 i = r0 i [1 − exp(−γt)]. Such variations are linked with the updating equations by a stochastic switch (by comparing with a uniformly distributed random number), controlling the balance between exploration and exploitation. Xin-She Yang Nature-Inspired Optimization Algorithms Middlesex University London 22 / 44
- 23. Swarm Intelligence Other Algorithms Other Algorithms Genetic algorithms Differential evolution Artificial immune system Harmony search Flower pollination algorithm Memetic algorithm ... Reviews Yang et al., Swarm Intelligence and Bio-Inspired Computation: Theory and Applications, Elsevier, (2013). Yang, X. S., Nature-Inspired Optimization Algorithms, Elsevier, (2014). Xin-She Yang Nature-Inspired Optimization Algorithms Middlesex University London 23 / 44
- 24. Applications Applications Applications Design optimization: structural engineering, product design ... Scheduling, routing and planning: discrete, combinatorial problems ... Image processing and data Mining: image segmentation, feature selection ... Applications in almost all areas (e.g., finance, economics, engineering, industry, ...) 10 Examples in Science and Engineering Xin-She Yang Nature-Inspired Optimization Algorithms Middlesex University London 24 / 44
- 25. Applications Pressure Vessel Design Optimization (Example 1) Pressure Vessel Design Optimization (Example 1) r d1 r L d2 Xin-She Yang Nature-Inspired Optimization Algorithms Middlesex University London 25 / 44
- 26. Applications Pressure Vessel Design Pressure Vessel Design This is a well-known test problem for optimization (e.g., see Cagnina et al. 2008) and it can be written as minimize f (x) = 0.6224d1rL + 1.7781d2r2 + 3.1661d2 1 L + 19.84d2 1 r, subject to g1(x) = −d1 + 0.0193r ≤ 0 g2(x) = −d2 + 0.00954r ≤ 0 g3(x) = −πr2L − 4π 3 r3 + 1296000 ≤ 0 g4(x) = L − 240 ≤ 0. The simple bounds are 0.0625 ≤ d1, d2 ≤ 99 × 0.0625, 10.0 ≤ r, L ≤ 200.0. The best solution (Yang, 2010; Gandomi and Yang, 2011) f∗ = 6059.714, x∗ = (0.8125, 0.4375, 42.0984, 176.6366). Now we have a set of analytical solutions (Yang et al., 2013). Xin-She Yang Nature-Inspired Optimization Algorithms Middlesex University London 26 / 44
- 27. Applications Speed Reducer/Gear Box Design (Example 2) Speed Reducer/Gear Box Design (Example 2) Mixed-Integer Programming: Continuous variables and integers. Xin-She Yang Nature-Inspired Optimization Algorithms Middlesex University London 27 / 44
- 28. Applications f (x1, x2, x3, x4, x5, x6, x7) = 0.7854x1x2 2 (3.3333x2 3 + 14.9334x3 − 43.0934) −1.508x1(x2 6 + x2 7 ) + 7.4777(x3 6 + x3 7 ) + 0.7854(x4x2 6 + x5x2 7 ), subject to g1 = 27 x1x2 2 x3 − 1 ≤ 0, g2 = 397.5 x1x2 2 x2 3 − 1 ≤ 0, g3 = 1.93x3 4 x2x3d4 1 − 1 ≤ 0, g4 = 1.93x3 5 x2x3d4 2 − 1 ≤ 0, g5 = 1 110x3 6 (745x4 hx3 )2 + 16.9 × 106 − 1 ≤ 0, g6 = 1 85x3 7 (745x5 hx3 )2 + 157.5 × 106 − 1 ≤ 0, g7 = x2x3 40 − 1 ≤ 0, g8 = 5x2 x1 − 1 ≤ 0, g9 = x1 12x2 − 1 ≤ 0, g10 = 1.5x6+1.9 x4 − 1 ≤ 0, g11 = 1.1x7+1.9 x5 − 1 ≤ 0. Simple bounds are 2.6 ≤ x1 ≤ 3.6, 0.7 ≤ h ≤ 0.8, 17 ≤ x3 ≤ 28, 7.3 ≤ x4 ≤ 8.3, 7.8 ≤ x5 ≤ 8.3, 2.9 ≤ x6 ≤ 3.9, and 5.0 ≤ x7 ≤ 5.5. z must be integers. Xin-She Yang Nature-Inspired Optimization Algorithms Middlesex University London 28 / 44
- 29. Applications Dome Design (Example 3) Dome Design (Example 3) 120-bar dome: Divided into 7 groups, 120 design elements, about 200 constraints (Gandomi and Yang 2011; Yang et al. 2012). Xin-She Yang Nature-Inspired Optimization Algorithms Middlesex University London 29 / 44
- 30. Applications Tower Design (Example 4) Tower Design (Example 4) 26-storey tower: 942 design elements, 244 nodal links, 59 groups/types, > 4000 nonlinear constraints (Yang et al. 2011; Gandomi & Yang 2012). Xin-She Yang Nature-Inspired Optimization Algorithms Middlesex University London 30 / 44
- 31. Applications Car Door Design (Example 5) Car Door Design (Example 5) Design better, safer and energy-efficient cars Minimize weight, low crash deflection (< 32 mm), lower impact force (< 4 kN), .... Even a side barrier has 11 design variables! Xin-She Yang Nature-Inspired Optimization Algorithms Middlesex University London 31 / 44
- 32. Applications Scheduling(Example 6) Scheduling(Example 6) MK Marichelvam, T Prabaharan, XS Yang, Improved cuckoo search algorithm for hybrid flow shop scheduling problems to minimize makespan, Applied Soft Computing, vol. 19, no. 1,pp. 93-101 (2014). Xin-She Yang Nature-Inspired Optimization Algorithms Middlesex University London 32 / 44
- 33. Applications Soft Testing and Path Generation Soft Testing and Path Generation Srivastava, Mallikarjun, Yang (2013). Optimal test sequence generation using firefly algorithm, Swarm and Evolutionary Computation, 8(2013) 44-53. Xin-She Yang Nature-Inspired Optimization Algorithms Middlesex University London 33 / 44
- 34. Applications Travel Salesman Problem (Example 7) Travel Salesman Problem (Example 7) Xin-She Yang Nature-Inspired Optimization Algorithms Middlesex University London 34 / 44
- 35. Applications Cuckoo Search for TSP Cuckoo Search for TSP Aziz Ouaarab, Belad Ahiod, Xin-She Yang, Discrete cuckoo search algorithm for the travelling salesman problem, Neural Computing and Applications, (2013). http://link.springer.com/article/10.1007/s00521-013-1402-2. Xin-She Yang Nature-Inspired Optimization Algorithms Middlesex University London 35 / 44
- 36. Applications Data Mining: Feature Selection (Example 8) Data Mining: Feature Selection (Example 8) Bat algorithm and cuckoo search are very competitive (Papa et al, 2013; 2014). Also effective at dimension reduction (in terms of number of key features). Dataset # samples #features #classes Diabetes 768 8 2 DNA 2000 180 3 Mushrooms 8124 112 2 OPF BA CS FA ICS PSO0 10 20 30 40 50 60 70 80 Accuracy[%] 61 61 59 61 61 6160 61 61 62 6263 62 62 62 61 Mushrooms Baseline Binary Sigmoid Hyperbolic Tangent Xin-She Yang Nature-Inspired Optimization Algorithms Middlesex University London 36 / 44
- 37. Applications Image Processing (Example 9) Image Processing (Example 9) Satellite images and flooding estimates (Senthilnath & Yang, 2014) Xin-She Yang Nature-Inspired Optimization Algorithms Middlesex University London 37 / 44
- 38. Applications Satellite images Satellite images Three methods are compared, and FA obtained the best results. Algorithms Optimal Worst Mean GA 0.8044 0.7528 0.7824 PSO 0.8057 0.8009 0.8051 FA 0.8126 0.8019 0.8076 Also, Senthilnath et al. compared 14 different algorithms for classifications, and they found that the firefly algorithm seems to be most efficient among 14 different classification methods. Xin-She Yang Nature-Inspired Optimization Algorithms Middlesex University London 38 / 44
- 39. Applications See Fong et al. (2014) for details. Xin-She Yang Nature-Inspired Optimization Algorithms Middlesex University London 39 / 44
- 40. Applications Classifications and Clustering (Example 10) Classifications and Clustering (Example 10) K-means clustering with nature-inspired algorithms can produce far better results than K-means methods (Fong, Deb, Yang, Zhang, 2014). Xin-She Yang Nature-Inspired Optimization Algorithms Middlesex University London 40 / 44
- 41. Applications The combination of K-means methods with the bat algorithm (and other algorithms) can indeed enhance the accuracy of clustering. Xin-She Yang Nature-Inspired Optimization Algorithms Middlesex University London 41 / 44
- 42. Some Open Problems Some Open Problems Some Open Problems Mathematical Analysis: A unified mathematical framework (e.g., Markov chain theory, dynamical systems) is needed. Comparison: What are the best/fair performance measures? Parameter tuning: How to tune the algorithm-dependent parameters so that an algorithm can achieve the best performance? Is there a general framework for this? Scalability: Can the algorithms that work for small-scale problems be directly applied to large-scale/high-dimensional problems (for d > 1000) ? Intelligence: Smart algorithms may be a buzz word, but can truly intelligent algorithms be developed? Xin-She Yang Nature-Inspired Optimization Algorithms Middlesex University London 42 / 44
- 43. Thank you :) Books on Nature-Inspired Algorithms Books on Nature-Inspired Algorithms Xin-She Yang, Nature-Inspired Optimization Algorithms, Elsevier, (2014). Xin-She Yang, Cuckoo Search and Firefly Algorithm, Springer, (2014). Xin-She Yang Nature-Inspired Optimization Algorithms Middlesex University London 43 / 44
- 44. Thank you :) Thank you :) Any questions? Thank you :) Any questions? Xin-She Yang et al., Swarm Intelligence and Bio-Inspired Computation: Theory and Applications, Elsevier, (2013). Xin-She Yang, Artificial Intelligence, Evolutionary Computing and Metaheuristics, (2013). Xin-She Yang Nature-Inspired Optimization Algorithms Middlesex University London 44 / 44