![[email_address] http://www.flickr.com/photos/earlg/275371357/](https://image.slidesharecdn.com/meta08dreoparameters-1226652476241036-9/85/Multi-criteria-meta-parameter-tuning-for-mono-objective-stochastic-metaheuristics-36-320.jpg)
Multi-criteria meta-parameter tuning for mono-objective stochastic metaheuristics
- 1. Multi-objective meta-parameter tuning for mono-objective stochastic metaheuristics Johann Dréo THALES Research & Technology
- 2. Introduction Multi-objective method Parameter tuning Stochastic metaheuristics Performance profiles http://www.flickr.com/photos/k23/2792398403/ Dreo & Siarry, 2004
- 4. Examples of stochastic metaheuristics
- 7. As a mono-objective problem Parameter setting: Improve performance http://www.flickr.com/photos/sigfrid/223626315/
- 8. As a multi-objective problem Parameter setting: What is performance ? -> multi-objective problem http://www.flickr.com/photos/jesusdq/345379863/
- 9. Multi-objective problem Performance ? Precision Speed Robustness Precision Speed Stability (← benchmark) http://www.flickr.com/photos/matthewfch/1688409628/
- 10. Multi-objective problem Performance ? Precision Speed Robustness Precision Speed Stability (← benchmark)
- 11. Meta-parameter tuning Mono-objective problem Stochastic metaheuristic
- 12. Meta-parameter tuning Multi-objective parameter tuning problem Mono-objective problem Stochastic metaheuristic
- 13. Meta-parameter tuning Multi-objective parameter tuning problem Mono-objective problem Stochastic metaheuristic Meta-optimizer
- 14. Complexity Difficult Easier 1 time Multi-objective parameter tuning problem Mono-objective problem Stochastic metaheuristic Meta-optimizer
- 15. Methodology Speed / Precision Median estimation Mono-objective problem Stochastic metaheuristic NSGA-2
- 16. Methodology Speed / Precision Median estimation Mono-objective problem Stochastic metaheuristic NSGA-2
- 17. Results plots Speed Precision Performance profile / front
- 18. Some results
- 19. Example 2 continuous EDA (CEDA, CHEDA) Sampling density parameter Rosenbrock, 2 dimensions Median estimated with 10 runs 10 000 max eval. NSGA-2 20 iter., 50 indiv. 10 runs 3 days computation + Nelder-Mead Search
- 20. Example + simulated annealing stable temperature parameter Rosenbrock, 2 dimensions Median estimated with 10 runs 10 000 max eval. NSGA-2 20 iter., 50 indiv. 10 runs 1 day computation
- 21. Example + genetic algorithm population parameter Rosenbrock, 2 dimensions Median estimated with 10 runs 10 000 max eval. NSGA-2 20 iter., 50 indiv. 10 runs 1 day computation
- 22. SA JGEN CEDA CHEDA Speed Precision
- 23. Behaviour exploration Speed Precision Genetic algorithm Population size
- 24. Performance front Temporal planner, ''Divide & Evolve > CPT'', version ''GOAL'' 2 mutation parameters IPC ''rovers'' problem, instance 06 Median estimated with 10 runs NSGA-2 10 iter., 5 indiv. 30 runs 1 week computation for 1 run
- 25. Performance front in Parameters space Speed Precision M1 M2
- 27. Conclusion
- 28. Drawbacks Computation cost Stochastic M.-O. algo. -> supplementary bias http://www.flickr.com/photos/orvaratli/2690949652/
- 29. Drawbacks Computation cost Stochastic M.-O. algo. -> supplementary bias Valid only for: Algorithm implementation Problem instance Stopping criterion Error Time t steps, improvement < ε http://www.flickr.com/photos/orvaratli/2690949652/
- 30. Drawbacks Computation cost Stochastic M.-O. algo. -> supplementary bias Valid only for: Algorithm implementation Problem instance Stopping criterion Error Time t steps, improvement < ε Fronts often convex -> aggregations ? No benchmarking http://www.flickr.com/photos/orvaratli/2690949652/
- 31. Advantages Performance profiles Objectives space Parameters space Quantification of expert knowledge
- 32. Advantages Performance profiles Objectives space Parameters space Quantification of expert knowledge Automatic parameter tuning One step before use N parameters -> 1 parameter More degrees of freedom
- 33. Advantages Performance profiles Objectives space Parameters space Quantification of expert knowledge Automatic parameter tuning One step before use N parameters -> 1 parameter More degrees of freedom Algorithms comparison Statistical tests more meaningful
- 34. Advantages Performance profiles Objectives space Parameters space Quantification of expert knowledge Automatic parameter tuning One step before use N parameters -> 1 parameter More degrees of freedom Algorithms comparison Statistical tests more meaningful Behaviour understanding
- 35. Perspectives Include robustness Include dispersion estimation Include benchmarking Multi-objective SPO, F-Race Regressions in parameters space Performances / parameters Behaviour models? Links? Fitness Landscape / Performance profiles Run time distribution Taillard's significance plots ... http://www.flickr.com/photos/colourcrazy/2065575762/