Appendix J: Job Submission Filter (aka 'qsub Wrapper')

When a submit filter exists, TORQUE will send the command file (or contents of STDIN if piped to qsub) to that script/executable and allow it to evaluate the submitted request based on specific site policies. The resulting file is then handed back to qsub and processing continues. Submit filters can check user jobs for correctness based on site policies. They can also modify user jobs as they are submitted. Some examples of what a submit filter might evaluate and check for are:

• Memory Request - Verifiy that the job requests memory and rejects if it does not.
• Job event notifications - Check if the job does one of the following and rejects it if it:
  • explicitly requests no notification.
  • requests notifications but does not provide an email address.
• Walltime specified - Verify that the walltime is specified.
• Global Walltime Limit - Verify that the walltime is below the global max walltime.
• Test Walltime Limit - If the job is a test job, this check rejects the job it if it requests a walltime longer than the testing maximum.

The script below reads the original submission request from STDIN and shows how you could insert parameters into a job submit request:

#!/bin/sh

# add default memory constraints to all requests
# that did not specify it in user's script or on command line

echo "#PBS -l mem=16MB"

while read i
  do
    echo $i
  done

Command line arguments passed to qsub are passed as arguments to the submit filter (filter won’t see them in STDIN) in the same order and may be used as needed. It should be noted that as of TORQUE 2.2.0 extended attributes are not passed to the filter. Exit status of -1 will cause qsub to reject the submission with a message stating that it failed due to administrative policies.

The submit filter must be executable, must be available on each of the nodes where users may submit jobs, and by default, must be located at ${libexecdir}/qsub_filter ( for version 2.1 and older: /usr/local/sbin/torque_submitfilter). At run time, if the file does not exist at this new preferred path then qsub will fall back to the old hard-coded path. The submit filter location can be customized by setting the SUBMITFILTER parameter inside the torque.cfg file, as in the following example:

SUBMITFILTER /opt/torque/submit.pl
...

*Initial development courtesy of Oak Ridge National Laboratories