![Performance testing of
microservices
IN ACTION
Alexander Kachur
Lead QA Automation Engineer
at Waverley Software
alexander.kachur[at]gmail.com
Julien Dupre](https://image.slidesharecdn.com/seleniumcampperf-180305135309/85/Performance-testing-of-microservices-in-Action-1-320.jpg)
![Flame Graphs with Docker
0. [sudo sysctl kernel.perf_event_paranoid=0]
1. perf record -F 99 -g -p ${PID}
2. docker exec -it ${container_ID} top
3. docker exec ${container_ID} cat /tmp/perf-${internal_PID}.map > /tmp/perf-
${PID}.map
4. perf script -f > out.perf
5. ./FlameGraph/stackcollapse-perf.pl out.perf > out.folded
6. ./FlameGraph/flamegraph.pl --hash out.folded > out.svg](https://image.slidesharecdn.com/seleniumcampperf-180305135309/85/Performance-testing-of-microservices-in-Action-41-320.jpg)
![Flame Graphs with Docker
0. [sudo sysctl kernel.perf_event_paranoid=0]
1. perf record -F 99 -g -p ${PID}
2. docker exec -it ${container_ID} top
3. docker exec ${container_ID} cat /tmp/perf-${internal_PID}.map > /tmp/perf-
${PID}.map
4. perf script -f > out.perf
5. ./FlameGraph/stackcollapse-perf.pl out.perf > out.folded
6. ./FlameGraph/flamegraph.pl --hash out.folded > out.svg](https://image.slidesharecdn.com/seleniumcampperf-180305135309/85/Performance-testing-of-microservices-in-Action-42-320.jpg)
![Flame Graphs with Docker
0. [sudo sysctl kernel.perf_event_paranoid=0]
1. perf record -F 99 -g -p ${PID}
2. docker exec -it ${container_ID} top
3. docker exec ${container_ID} cat /tmp/perf-${internal_PID}.map > /tmp/perf-
${PID}.map
4. perf script -f > out.perf
5. ./FlameGraph/stackcollapse-perf.pl out.perf > out.folded
6. ./FlameGraph/flamegraph.pl --hash out.folded > out.svg](https://image.slidesharecdn.com/seleniumcampperf-180305135309/85/Performance-testing-of-microservices-in-Action-43-320.jpg)
![Flame Graphs with Docker
0. [sudo sysctl kernel.perf_event_paranoid=0]
1. perf record -F 99 -g -p ${PID}
2. docker exec -it ${container_ID} top
3. docker exec ${container_ID} cat /tmp/perf-${internal_PID}.map > /tmp/perf-$
{PID}.map
OR
4. perf script | perl ./FlameGraph/stackcollapse-perf.pl |gzip | curl --data-binary
@- -H "Content-Tye: gzip" -H "Accept-Encoding: gzip" flamegraph-
generator.herokuapp.com/api/generate
https://github.com/olka/flamegraph-generator](https://image.slidesharecdn.com/seleniumcampperf-180305135309/85/Performance-testing-of-microservices-in-Action-44-320.jpg)
Performance testing of microservices in Action
- 1. Performance testing of microservices IN ACTION Alexander Kachur Lead QA Automation Engineer at Waverley Software alexander.kachur[at]gmail.com Julien Dupre
- 2. 1. Consideration points on picking load generator 2. How to use CPU flame graph to identify bottleneck 3. Issues with the Cloud and Docker 4. How to use perf to figure out what happened 5. Links Take Aways
- 6. Containers
- 12. P is for Profiling
- 16. Reporting
- 17. Reporting
- 21. Round 1
- 22. Round 1
- 23. Flame Graphs main() create()find() query() init() print() main() find() main() init() create() main() create() query() print()
- 24. Flame Graphs
- 25. Flame Graphs
- 26. Round 1 DNS
- 27. Round 2
- 28. Round 2
- 29. Flame Graphs
- 30. Flame Graphs
- 31. Flame Graphs
- 33. Round 3
- 34. Round 3
- 35. Flame Graphs
- 36. Flame Graphs
- 37. Flame Graphs
- 39. Final Round
- 40. Final Round
- 41. Flame Graphs with Docker 0. [sudo sysctl kernel.perf_event_paranoid=0] 1. perf record -F 99 -g -p ${PID} 2. docker exec -it ${container_ID} top 3. docker exec ${container_ID} cat /tmp/perf-${internal_PID}.map > /tmp/perf- ${PID}.map 4. perf script -f > out.perf 5. ./FlameGraph/stackcollapse-perf.pl out.perf > out.folded 6. ./FlameGraph/flamegraph.pl --hash out.folded > out.svg
- 42. Flame Graphs with Docker 0. [sudo sysctl kernel.perf_event_paranoid=0] 1. perf record -F 99 -g -p ${PID} 2. docker exec -it ${container_ID} top 3. docker exec ${container_ID} cat /tmp/perf-${internal_PID}.map > /tmp/perf- ${PID}.map 4. perf script -f > out.perf 5. ./FlameGraph/stackcollapse-perf.pl out.perf > out.folded 6. ./FlameGraph/flamegraph.pl --hash out.folded > out.svg
- 43. Flame Graphs with Docker 0. [sudo sysctl kernel.perf_event_paranoid=0] 1. perf record -F 99 -g -p ${PID} 2. docker exec -it ${container_ID} top 3. docker exec ${container_ID} cat /tmp/perf-${internal_PID}.map > /tmp/perf- ${PID}.map 4. perf script -f > out.perf 5. ./FlameGraph/stackcollapse-perf.pl out.perf > out.folded 6. ./FlameGraph/flamegraph.pl --hash out.folded > out.svg
- 44. Flame Graphs with Docker 0. [sudo sysctl kernel.perf_event_paranoid=0] 1. perf record -F 99 -g -p ${PID} 2. docker exec -it ${container_ID} top 3. docker exec ${container_ID} cat /tmp/perf-${internal_PID}.map > /tmp/perf-$ {PID}.map OR 4. perf script | perl ./FlameGraph/stackcollapse-perf.pl |gzip | curl --data-binary @- -H "Content-Tye: gzip" -H "Accept-Encoding: gzip" flamegraph- generator.herokuapp.com/api/generate https://github.com/olka/flamegraph-generator
- 45. RPC problem
- 46. RPC solutions
- 47. Performance in the Cloud
- 48. Performance in the Cloud
- 49. Performance in the Cloud
- 50. Drill Down
- 51. Drill Down
- 52. Drill Down
- 53. Drill Down
- 54. Drill Down
- 55. Conclusion 1. Why? 2. Why? 3. Why? 4. Why? 5. Why?
- 56. Gurus Brendan Gregg - brendangregg.com | FlameGraph Baron Schwartz - xaprb.com Rob Harrop - robharrop.github.io Neil J. Gunther - perfdynamics.com/Manifesto/USLscalability.html Aleksey Shipilëv- shipilev.net Sasha Goldshtein - blog.sashag.net Nitsan Wakart - psy-lob-saw.blogspot.co.il Aleksey Lavrenyuk - events.yandex.ru/lib/people/302066 Vladimir Sitnikov- lektorium.tv/speaker/27401