5.8 Setting min_threads and max_threads

There are two threadpools in Torque, one for background tasks and one for incoming requests from the MOMs and through the API (client commands, Moab, and so forth). The min_threads and max_threads parameters control the number of total threads used for both, not for each individually. The incoming requests' threadpool has three-quarters of min_threads for its minimum, and three-quarters of max_threads for its maximum, with the background pool receiving the other one-quarter.

Additionally, pbs_server no longer allows incoming requests to pile up indefinitely. When the threadpool is too busy for incoming requests, it indicates such, returning PBSE_SERVER_BUSY with the accompanying message that "Pbs Server is currently too busy to service this request. Please retry this request." The threshold for this message, if the request is from a manager, is that at least two threads be available in the threadpool. If the request comes from a non-manager, 5% of the threadpool must be available for the request to be serviced. Note that availability is calculated based on the maximum threads and not based on the current number of threads allocated.

If an undesirably large number of requests are given a busy response, one option is to increase the number of maximum threads for the threadpool. If the load on the server is already very high, then this is probably not going to help, but if the CPU load is lower, then it may help. Remember that by default the threadpool shrinks down once the extra threads are no longer needed. This is controlled via the thread_idle_seconds server parameter.

© 2017 Adaptive Computing