10.1 Job Logging

New in TORQUE 2.5.3 is the ability to log job information for completed jobs. The information stored in the log file is the same information produced with the command qstat -f. The log file data is stored using an XML format. Data can be extracted from the log using the utility showjobs found in the contrib/ directory of the TORQUE source tree. Custom scripts that can parse the XML data can also be used.

10.1.1 Job Log Location and Name

The job log is kept at $TORQUE_HOME/job_logs. The naming convention for the job log is the same as for the server log or MOM log. The log name is created from the current year/month/day . For example, if today's date is 26 October, 2010 the log file is named 20101026. A new log file is created each new day that data is written to the log.

10.1.2 Enabling Job Logs

There are five new server parameters used to enable job logging. These parameters control what information is stored in the log and manage the log files.

• record_job_info - This must be set to true in order for job logging to be enabled. If not set to true, the remaining server parameters are ignored.
• record_job_script - If set to true, this adds the contents of the script executed by a job to the log.
• job_log_file_max_size - This specifies a soft limit (in kilobytes) for the job log's maximum size. The file size is checked every five minutes and if the current day file size is greater than or equal to this value, it is rolled from <filename> to <filename.1> and a new empty log is opened. If the current day file size exceeds the maximum size a second time, the <filename.1> log file is rolled to <filename.2>, the current log is rolled to <filename.1>, and a new empty log is opened. Each new log causes all other logs to roll to an extension that is one greater than its current number. Any value less than 0 is ignored by pbs_server (meaning the log will not be rolled).
• job_log_file_roll_depth - This sets the maximum number of new log files that are kept in a day if the job_log_file_max_size parameter is set. For example, if the roll depth is set to 3, no file can roll higher than <filename.3>. If a file is already at the specified depth, such as <filename.3>, the file is deleted so it can be replaced by the incoming file roll, <filename.2>.
• job_log_keep_days - This maintains logs for the number of days designated. If set to 4, any log file older than 4 days old is deleted.