How to inspectRUNNING perl process

@hirose31ひろせ まさあき

YAPC::Asia Tokyo 2013LT Day 1

Who am I

• PAUSE ID: HIROSE / @hirose31

• Recognize as NON-human on CPAN

テキスト

https://metacpan.org/module/Acme::CPANAuthors::Nonhuman

AgendaHow to inspect a RUNNING perl process

•Motivation

• Revisit existing ideas

• Introduce a my idea

Motivation

•WHY: shooting trouble of RUNNING perl process

• examine where perl process is running

•where does process hang up?

•where does process loop forever?

•which pacage leaks memory?

•want to examine from OUTER without editing target process/script

Revisit existing ideas‘strace’

• strace -fF -Tttt -s 512 -p PID

• current system call

• show the time spent in system calls

• peep into an ASCII string

• no requires☺, just attach process

• show just name of system call not perl symbol☹

1379392067.757575 accept(5,

1379392038.406593 write(5, "\22\0\0\0\3 select sleep(8) ", 22) = 22 <0.000083>1379392038.406791 read(5,

Revisit existing ideas‘gdb’

• Attach and trace code by gdb

• Generate a core file by gcore PID

• “Introduction gdb to perl programmers”, @stanaka, Shibuya.pm #9http://blog.stanaka.org/entry/20080630/1214780557

• show stacktrace with perl symbol

#0 0x00007fb086990c20 in __accept_nocancel () from /lib/libpthread.so.0#1 0x00000000004f19e8 in Perl_pp_accept (my_perl=0xd69010) at pp_sys.c:2554...#5 0x0000000000420eef in main (argc=8, argv=0x7fff34899828, env=0x7fff34899870) at perlmain.c:99

Revisit existing ideas‘bulkdbg’

• https://github.com/ahiguti/bulkdbg

• bulkdbg PID

• show stackstrace with perl symbol

• convinient by batch

3183 __accept_nocancel:Perl_pp_accept:Perl_pop_scope:Perl_peep:_dl_rtld_di_serinfo:PerlIO_debug:_dl_rtld_di_serinfo:PerlIO_debug:calloc:calloc:__libc_malloc:calloc:_dl_rtld_di_serinfo:S_hv_fetch_common

Revisit existing ideas‘gdbperl’

• https://github.com/ahiguti/gdbperl

• examine state of perl interpreter using gdb

• “debugging Perl script with gdb”, Akira Higuchi, YAPC::Asia Tokyo 2011

http://www.slideshare.net/akirahiguchi/gdbperl

• show stacktrace with perl code☺, package name, line number!!

• depend heavily on perl version/internal structure☹

perl_backtrace:[8] IO::Socket::accept() <- /.../perl-5.8.8-threads/lib/site_perl/5.8.8/Starlet/Server.pm:106(Starlet::Server)[7] (loop) <- /.../perl-5.8.8-threads/lib/site_perl/5.8.8/Starlet/Server.pm:105(Starlet::Server)...[1] Plack::Runner::run() <- /.../perl-5.8.8-threads/bin/plackup:10(main)

Introduce a my idea‘inspect-perl-proc’

• https://github.com/hirose31/inspect-perl-proc

• Attach by gdb and run arbitrary Perl code using Perl_eval_pv

• less requires: gdb, perl with debug symbol (-DDEBUGGING=-g)

• not depend on perl version / perl internal structure

• save output to file, not output STDOUT/STDERR of target process

dump-perl-stacktrace• dump-perl-stacktrace

= inspect-perl-proc --mode 'dump-stacktrace'

• Carp::longmess(‘Dump stacktrace’)

Dump stacktrace at /.../perl-5.8.8-threads/lib/5.8.8/x86_64-linux-thread-multi/IO/Socket.pm line 237 IO::Socket::accept('IO::Socket::INET=GLOB()') called at /.../Starlet/Server.pm line 106 Starlet::Server::accept_loop('Plack::Handler:: Starlet=HASH()', 'CODE()', 100) called at /.../Starlet.pm line 67 Plack::Handler::Starlet::run('Plack::Handler:: Starlet=HASH()', 'CODE()') called at /.../Plack/Loader.pm line 84... Plack::Runner::run('Plack::Runner=HASH()') called at /.../bin/plackup line 10

dump-perl-memusage

• dump-perl-memusage

= inspect-perl-proc --mode 'dump-memusage'

• Dump memory usage with B::Size2::Terse::package_size

for each packages (Devel::Symdump)

• save as as hashref so you can filter or sort easily

{ 'main' => '95589',... 'PerlIO::scalar' => '1025', 'Plack' => '3056', 'Plack::App' => '200', 'Plack::App::URLMap' => '26681', 'Plack::Builder' => '31972',...};

dump-perl-inc• dump-perl-inc

= inspect-perl-proc --mode 'dump-inc'

• Dump %INC

• list up loaded modules for preloading (CoW)

• save as as hashref so you can filter or sort easily

{ 'parent.pm' => '/.../parent.pm', 'HTTP/Status.pm' => '/.../HTTP/Status.pm', 'POSIX.pm' => '/.../POSIX.pm', 'List/Util.pm' => '/.../List/Util.pm', 'Plack/Loader.pm' => '/.../Plack/Loader.pm', 'Cwd.pm' => '/.../Cwd.pm', 'Fcntl.pm' => '/.../Fcntl.pm',...};

Introduce a my idea‘inspect-perl-proc’

• https://github.com/hirose31/inspect-perl-proc

• Attach by gdb and run arbitrary code using Perl_eval_pv

• less requires: gdb, perl with debug symbol (-DDEBUGGING=-g)

• not depend on perl version / perl internal structure

• save output to file, not output STDOUT/STDERR of target process

Thanks :D