How to inspect a RUNNING perl process
4 likes4,910 views
The document presents methods for inspecting running Perl processes, focusing on troubleshooting issues like memory leaks and infinite loops. It reviews existing tools such as 'strace' and 'gdb', while introducing a new tool called 'inspect-perl-proc' which allows users to attach to a Perl process and execute arbitrary code with minimal dependencies. Additionally, the document explains how to gather stack traces, memory usage, and loaded module information from a Perl process.
![Revisit existing ideas
‘gdbperl’
• https://github.com/ahiguti/gdbperl
• examine state of perl interpreter using gdb
• “debugging Perl script with gdb”, Akira Higuchi, YAPC::Asia Tokyo
2011
http://www.slideshare.net/akirahiguchi/gdbperl
• show stacktrace with perl code☺, package name, line number!!
• depend heavily on perl version/internal structure☹
perl_backtrace:
[8] IO::Socket::accept() <- /.../perl-5.8.8-threads/lib/
site_perl/5.8.8/Starlet/Server.pm:106(Starlet::Server)
[7] (loop) <- /.../perl-5.8.8-threads/lib/site_perl/5.8.8/
Starlet/Server.pm:105(Starlet::Server)
...
[1] Plack::Runner::run() <- /.../perl-5.8.8-threads/bin/
plackup:10(main)](https://image.slidesharecdn.com/inspect-running-perl-130923222212-phpapp02/85/How-to-inspect-a-RUNNING-perl-process-8-320.jpg)
How to inspect a RUNNING perl process
- 1. How to inspect RUNNING perl process @hirose31 ひろせ まさあき YAPC::Asia Tokyo 2013 LT Day 1
- 2. Who am I •PAUSE ID: HIROSE / @hirose31 •Recognize as NON-human on CPAN テキスト https://metacpan.org/module/Acme::CPANAuthors::Nonhuman
- 3. Agenda How to inspect a RUNNING perl process •Motivation •Revisit existing ideas •Introduce a my idea
- 4. Motivation •WHY: shooting trouble of RUNNING perl process •examine where perl process is running •where does process hang up? •where does process loop forever? •which pacage leaks memory? •want to examine from OUTER without editing target process/script
- 5. Revisit existing ideas ‘strace’ • strace -fF -Tttt -s 512 -p PID • current system call • show the time spent in system calls • peep into an ASCII string • no requires☺, just attach process • show just name of system call not perl symbol☹ 1379392067.757575 accept(5, 1379392038.406593 write(5, "220003 select sleep(8) ", 22) = 22 <0.000083> 1379392038.406791 read(5,
- 6. Revisit existing ideas ‘gdb’ • Attach and trace code by gdb • Generate a core file by gcore PID • “Introduction gdb to perl programmers”, @stanaka, Shibuya.pm #9 http://blog.stanaka.org/entry/20080630/1214780557 • show stacktrace with perl symbol #0 0x00007fb086990c20 in __accept_nocancel () from /lib/ libpthread.so.0 #1 0x00000000004f19e8 in Perl_pp_accept (my_perl=0xd69010) at pp_sys.c:2554 ... #5 0x0000000000420eef in main (argc=8, argv=0x7fff34899828, env=0x7fff34899870) at perlmain.c:99
- 7. Revisit existing ideas ‘bulkdbg’ •https://github.com/ahiguti/bulkdbg •bulkdbg PID •show stackstrace with perl symbol •convinient by batch 3183 __accept_nocancel:Perl_pp_accept:Perl_pop_scope:Perl_peep:_dl_r tld_di_serinfo:PerlIO_debug:_dl_rtld_di_serinfo:PerlIO_debug:ca lloc:calloc:__libc_malloc:calloc:_dl_rtld_di_serinfo:S_hv_fetch _common
- 8. Revisit existing ideas ‘gdbperl’ • https://github.com/ahiguti/gdbperl • examine state of perl interpreter using gdb • “debugging Perl script with gdb”, Akira Higuchi, YAPC::Asia Tokyo 2011 http://www.slideshare.net/akirahiguchi/gdbperl • show stacktrace with perl code☺, package name, line number!! • depend heavily on perl version/internal structure☹ perl_backtrace: [8] IO::Socket::accept() <- /.../perl-5.8.8-threads/lib/ site_perl/5.8.8/Starlet/Server.pm:106(Starlet::Server) [7] (loop) <- /.../perl-5.8.8-threads/lib/site_perl/5.8.8/ Starlet/Server.pm:105(Starlet::Server) ... [1] Plack::Runner::run() <- /.../perl-5.8.8-threads/bin/ plackup:10(main)
- 9. Introduce a my idea ‘inspect-perl-proc’ • https://github.com/hirose31/inspect-perl-proc • Attach by gdb and run arbitrary Perl code using Perl_eval_pv • less requires: gdb, perl with debug symbol (- DDEBUGGING=-g) • not depend on perl version / perl internal structure • save output to file, not output STDOUT/ STDERR of target process
- 10. dump-perl-stacktrace • dump-perl-stacktrace = inspect-perl-proc --mode 'dump-stacktrace' • Carp::longmess(‘Dump stacktrace’) Dump stacktrace at /.../perl-5.8.8-threads/lib/5.8.8/x86_64- linux-thread-multi/IO/Socket.pm line 237 IO::Socket::accept('IO::Socket::INET=GLOB()') called at /.../Starlet/Server.pm line 106 Starlet::Server::accept_loop('Plack::Handler:: Starlet=HASH()', 'CODE()', 100) called at /.../Starlet.pm line 67 Plack::Handler::Starlet::run('Plack::Handler:: Starlet=HASH()', 'CODE()') called at /.../Plack/Loader.pm line 84 ... Plack::Runner::run('Plack::Runner=HASH()') called at /.../bin/plackup line 10
- 11. dump-perl-memusage • dump-perl-memusage = inspect-perl-proc --mode 'dump-memusage' • Dump memory usage with B::Size2::Terse::package_size for each packages (Devel::Symdump) • save as as hashref so you can filter or sort easily { 'main' => '95589', ... 'PerlIO::scalar' => '1025', 'Plack' => '3056', 'Plack::App' => '200', 'Plack::App::URLMap' => '26681', 'Plack::Builder' => '31972', ... };
- 12. dump-perl-inc • dump-perl-inc = inspect-perl-proc --mode 'dump-inc' • Dump %INC • list up loaded modules for preloading (CoW) • save as as hashref so you can filter or sort easily { 'parent.pm' => '/.../parent.pm', 'HTTP/Status.pm' => '/.../HTTP/Status.pm', 'POSIX.pm' => '/.../POSIX.pm', 'List/Util.pm' => '/.../List/Util.pm', 'Plack/Loader.pm' => '/.../Plack/Loader.pm', 'Cwd.pm' => '/.../Cwd.pm', 'Fcntl.pm' => '/.../Fcntl.pm', ... };
- 13. Introduce a my idea ‘inspect-perl-proc’ • https://github.com/hirose31/inspect-perl-proc • Attach by gdb and run arbitrary code using Perl_eval_pv • less requires: gdb, perl with debug symbol (- DDEBUGGING=-g) • not depend on perl version / perl internal structure • save output to file, not output STDOUT/ STDERR of target process
- 14. Thanks :D