Modern Linux Performance Tools for Application Troubleshooting
3 likes1,348 views
The document presents a discussion on modern Linux performance tools, particularly focusing on advanced measuring techniques for processes, IO, memory, and CPU. It emphasizes practical applications that can be utilized in production environments, referencing various tools like strace, sysdig, iostat, and perf. The author, Tanel Poder, shares personal expertise while promoting resources and a session on extending Oracle data warehouses with Hadoop.
Modern Linux Performance Tools for Application Troubleshooting
- 2. gluent.com 2 Intro: About me • Tanel Põder • Oracle Database Performance geek (18+ years) • Exadata Performance geek • Linux Performance geek • Hadoop Performance geek • CEO & co-founder: Expert Oracle Exadata book (2nd edition is out now!) Instant promotion
- 4. gluent.com 4 Topics • Not much about basic stuff like vmstat/iostat • Also nothing about SystemTap (or Dtrace) • Practical stuff that you can (relatively) safely run in production • Measuring / tracing processes • Measuring IO • Measuring Memory • Measuring CPU
- 5. gluent.com 5 Measuring Process Activity - 1 • Strace • man strace • strace -p <PID> • https://fritshoogland.files.wordpress.com/2013/03/profiling_of_oracl e_using_function_calls.pdf • SysDig • sysdig fd.name contains tnsnames • sysdig -p "%evt.arg.name" proc.name=cat and evt.type=open • csysdig • http://www.sysdig.org/wiki/sysdig-examples/
- 6. gluent.com 6 Measuring Process Activity - 2 • /proc filesystem • Proc filesystem in Linux is like V$/X$ views in Oracle • man proc • https://www.kernel.org/doc/Documentation/filesystems/proc.txt • http://blog.tanelpoder.com/2013/02/21/peeking-into-linux-kernel- land-using-proc-filesystem-for-quickndirty-troubleshooting/ • psnapper (early prototype!) • ./psnapper <seconds> <pid> • ./psnapper 5 1234 • http://blog.tanelpoder.com/files/scripts/tools/unix/psnapper
- 7. gluent.com 7 Measuring IO • iostat in RHEL 7 • /proc/<PID>/io • iotop (or htop, atop) • blktrace / blkparse (if we have time) • blktrace -d /dev/sda -o - | blkparse -i -
- 8. gluent.com 8 Measuring Memory • pmap -x <PID> • http://tech.e2sn.com/oracle/performance/unix-performance- tools/process-memory-matrix (Solaris example but roughly the same) • cat /proc/PID/smaps • grep MMUPageSize: /proc/`pgrep dbw0`/smaps | sort | uniq –c • https://www.kernel.org/doc/Documentation/filesystems/proc.txt • Smem • sudo smem –u • https://www.selenic.com/smem/
- 9. gluent.com 9 Measuring CPU • perf top • https://fritshoogland.wordpress.com/2012/07/10/the-linux-perf- utility-with-el-6/ • perf record –g && perf report • http://www.brendangregg.com/perf.html • http://externaltable.blogspot.jp/2014/05/flame-graphs-for- oracle.html • perf stat • http://blog.tanelpoder.com/2015/08/09/ram-is-the-new-disk-and- how-to-measure-its-performance-part-1/
- 10. gluent.com 10 Linux performance references • Brendan Gregg: • http://www.brendangregg.com/linuxperf.html • Luca Canali’s articles: • https://db-blog.web.cern.ch/blog/luca-canali/2015-12-extended- stack-profiling-ideas-tools-and-comments • Frits Hoogland: • https://fritshoogland.wordpress.com/