5.624 Other Diagnostic Options

When PBSDEBUG is set, some client commands will print additional diagnostic information.

$ export PBSDEBUG=yes

$ cmd

To debug different kinds of problems, it can be useful to see where in the code time is being spent. This is called profiling and there is a Linux utility "gprof" that will output a listing of routines and the amount of time spent in these routines. This does require that the code be compiled with special options to instrument the code and to produce a file, gmon.out, that will be written at the end of program execution.

The following listing shows how to build Torque with profiling enabled. Notice that the output file for pbs_mom will end up in the mom_priv directory because its startup code changes the default directory to this location.

# ./configure "CFLAGS=-pg -lgcov -fPIC"

# make -j5

# make install

# pbs_mom ... do some stuff for a while ...

# momctl -s

# cd /var/spool/torque/mom_priv

# gprof -b `which pbs_mom` gmon.out |less

Another way to see areas where a program is spending most of its time is with the valgrind program. The advantage of using valgrind is that the programs do not have to be specially compiled.

# valgrind --tool=callgrind pbs_mom