4.9
Jobs
This section describes behavior of the Job object in MWS. It contains the URLs, request bodies, and responses delivered to and from MWS.

The supported methods table below requires each resource to be accessed with a URL parameter of api-version=3 in order to behave as documented. For more information, see 3.4 Requesting Specific API Versions.

The 8.4.6 Fields: Jobs reference contains the type and description of all fields in the Job object. It also contains details regarding which fields are valid during PUT and POST actions.
4.9.1 Supported Methods
Resource | GET | PUT | POST | DELETE |
---|---|---|---|---|
/rest/jobs | Get All Jobs | -- | Submit Job | -- |
/rest/jobs/<name> | -- | Cancel Job | ||
/rest/jobs/<name>/<modifyAction> | -- | Perform Actions on Job | -- | -- |
In this section:
The HTTP GET method is used to retrieve Job information. You can also append the command with priority-analysis=true or node-analysis=true to get priority or eligibility information about the job.
Quick Reference
GET https://localhost:8080/mws/rest/jobs/<name>?api-version=3
URLs and Parameters
GET https://localhost:8080/mws/rest/jobs?api-version=3
Parameter | Required | Type | Description | Example |
---|---|---|---|---|
query | No | JSON |
Queries for specific results. It is possible to query by one or more fields based on MongoDB query syntax. |
query={"isActive":true} |
sort | No | JSON | Sort the results. Use 1 for ascending and -1 for descending. | sort={"name":-1} |
See 3.3 Global URL Parameters for available URL parameters.
How to Get All Jobs
GET https://localhost:8080/mws/rest/jobs?api-version=3&fields=name,flags&max=3
------------------------------------
{
"totalCount": 8,
"resultCount": 3,
"results": [
{
"flags": ["GLOBALQUEUE"],
"name": "Moab.1"
},
{
"flags": ["GLOBALQUEUE"],
"name": "Moab.2"
},
{
"flags": ["GLOBALQUEUE"],
"name": "Moab.4"
}
]
}
How to Get a Subset of Jobs
Get active jobs
------------------------------------
https://localhost:8080/mws/rest/jobs?api-version=3&query={"isActive":true}
Get completed jobs
------------------------------------
https://localhost:8080/mws/rest/jobs?api-version=3&query={"isActive":false}
Get jobs owned by a particular user
------------------------------------
https://localhost:8080/mws/rest/jobs?api-version=3&query={"credentials.user":"fred"}
Known Issues
Some jobs are not returned if DisplayFlags UseBlocking is set in the moab.cfg file.
URLs and Parameters
GET https://localhost:8080/mws/rest/jobs/<name>?api-version=3
Parameter | Required | Type | Value | Description |
---|---|---|---|---|
name | Yes | String | -- | The name of the object. |
See 3.3 Global URL Parameters for available URL parameters.
Sample Response
JSON response ------------------------------------ { "arrayIndex": null, "arrayMasterName": null, "attributes": [], "blocks": > { "category": "jobBlock"> "createdDate": "2024-06-22 19:08:30 UTC", "message": null, "partition": null, "type": null } ]> "bypassCount": 0, "cancelCount": 0, "commandFile": "/tmp/test.sh", "commandLineArguments": null, "completionCode": null, "cpuTime": 0, "credentials": { "account": null, "group": "adaptive", "jobClass": null, "qos": "NONE", "qosRequested": null, "user": "adaptive" }, "customName": null, "dates": { "completedDate": null, "createdDate": "2024-10-11 17:58:16 UTC", "deadlineDate": "2037-10-24 12:26:40 UTC", "dispatchedDate": null, "earliestRequestedStartDate": null, "earliestStartDate": "2024-10-11 17:58:18 UTC", "eligibleDate": "2024-10-11 17:59:19 UTC", "lastCanceledDate": null, "lastChargedDate": null, "lastPreemptedDate": null, "lastUpdatedDate": "2024-10-11 17:59:19 UTC", "startDate": null, "submitDate": "2024-10-11 17:58:16 UTC", "terminationDate": "2037-10-24 12:26:40 UTC" }, "deferCount": 0, "dependencies": [], "description": null, "duration": 8639999, "durationActive": 0, "durationMinimum": 0, "durationQueued": 31, "durationRemaining": 0, "durationSuspended": 0, "emailNotifyAddresses": [], "emailNotifyTypes": [], "environmentRequested": false, "environmentVariables": {}, "epilogScript": null, "flags": [ "GLOBALQUEUE" ], "holdDate": null, "holdReason": null, "holds": [], "initialWorkingDirectory": "/tmp", "isActive": true, "jobGroup": null, "masterNode": null, "memorySecondsDedicated": 0, "memorySecondsUtilized": 0, "messages": [], "migrateCount": 0, "minimumPreemptTime": 0, "mwmName": "Moab", "name": "Moab.15", "nodesExcluded": [], "nodesRequested": [], "nodesRequestedPolicy": null, "partitionAccessList": [ "torque" ], "partitionAccessListRequested": [ "mws", "torque", "SHARED" ], "partitionAccessListScheduler": [ "mws", "torque", "SHARED" ], "preemptCount": 0, "priorities": { "run": 0, "start": 1, "system": 0, "user": 0 }, "processorSecondsDedicated": 0, "processorSecondsLimit": 0, "processorSecondsUtilized": 0, "prologScript": null, "queueStatus": "blocked", "rank": 0, "rejectPolicies": [], "requirements": [ { "architecture": null, "attributes": { "matlab": [ { "comparator": "<=", "displayValue": null, "restriction": "must", "value": "7.1" > ], "soffice": [ { "comparator": "%=", "displayValue": null, "restriction": "must", "value": "3.1" } ] }, "dedicateAllProcessors": true, "features": [], "featuresExcluded": [], "featuresExcludedMode": "AND", "featuresRequested": [], "featuresRequestedMode": "AND", "index": 0, "metrics": {}, "nodeAccessPolicy": null, "nodeAllocationPolicy": null, "nodeCount": 0, "nodeSet": null, "nodes": [], "operatingSystem": null, "reservation": null, "resourcesPerTask": { "disk": { "dedicated": 0, "utilized": null }, "memory": { "dedicated": 0, "utilized": 0 }, "processors": { "dedicated": 1, "utilized": 0 }, "swap": { "dedicated": 0, "utilized": null } }, "taskCount": 4, "tasksPerNode": 0, "totalDedicatedProcessors": 1 } ], "reservationRequested": null, "resourceFailPolicy": null, "resourceManagerExtension": null, "resourceManagers": [ { "isDestination": false, "isSource": true, "jobName": "Moab.15", "name": "internal" } ], "shellName": "/bin/bash", "standardErrorFilePath": null, "standardOutputFilePath": null, "startCount": 0, "states": { "state": "Idle", "stateExpected": "Idle", "stateLastUpdatedDate": null, "subState": null }, "submitCommandFile": "/home/ace/jobscript.sh", "submitHost": "0:0:0:0:0:0:0:1", "systemJobType": null, "templates": [ { "name": "DEFAULT" } ], "triggers": [], "variables": {}, "virtualContainers": [] }
Job Arrays
- If a job is the master of a job array, the response will have some additional fields set as shown in the following example. The name field is chosen by the Moab HPC Suite, and the customName field comes from the Fields: Job Arrays name field.
- If a job is a subjob of an array, the response will have other fields set as shown in the following example:
Job array master ------------------------------------ { "name": "Moab.5", "customName": "myarray", "flags": [ "ARRAYMASTER", "GLOBALQUEUE", "CANCELONFIRSTFAILURE", "CANCELONANYSUCCESS" ] }
Array subjob ------------------------------------ { "name": "Moab.5[21]", "customName": "myarray", "arrayIndex": 21, "arrayMasterName": "Moab.5", "flags": [ "ARRAYJOB", "GLOBALQUEUE", "CANCELONFIRSTFAILURE", "CANCELONANYSUCCESS" ] }
4.9.2.C Get Job Priority Information
The priority-analysis parameter is used to get job priority information.
URLs and Parameters
GET https://localhost:8080/mws/rest/jobs/<name>?api-version=3&priority-analysis=true
Parameter | Required | Type | Value | Description |
---|---|---|---|---|
name | Yes | String | -- | The name of the job. |
See 3.3 Global URL Parameters for available URL parameters.
Sample Response
JSON response
------------------------------------
{
priorities: {
start: 36,
system: 0,
components: {
service: {
weight: 2,
queuetime: {
weight: 1,
value: 33
},
xfactor: {
weight: 0,
value: 1.559722
},
deadline: {
weight: 0,
value: 0
},
policyviolation: {
weight: 0,
value: 0
},
userprior: {
weight: 3,
value: -5
},
startcount: {
weight: 0,
value: 0
},
bypass: {
weight: 0,
value: 0
}
},
target: {
weight: 1,
queuetime: {
weight: 0,
value: 0
},
xfactor: {
weight: 0,
value: 0
}
},
credential: {
weight: 1,
user: {
weight: 0,
value: 0
},
group: {
weight: 0,
value: 0
},
account: {
weight: 0,
value: 0
},
qos: {
weight: 0,
value: 0
}
},
attribute: {
weight: 1,
attribute: {
weight: 0,
value: 0
},
gres: {
weight: 0,
value: 0
},
jobid: {
weight: 0,
value: 0
},
jobname: {
weight: 0,
value: 0
},
state: {
weight: 0,
value: 0
}
},
fairshare: {
weight: 1,
user: {
weight: 0,
value: 0
},
group: {
weight: 0,
value: 0
},
account: {
weight: 0,
value: 0
},
qos: {
weight: 0,
value: 0
},
guser: {
weight: 0,
value: 0
},
ggroup: {
weight: 0,
value: 0
},
gaccount: {
weight: 0,
value: 0
},
userwcacc: {
weight: 0,
value: 0
},
jobsperuser: {
weight: 0,
value: 0
},
jobsrunningperuser: {
weight: 0,
value: 0
},
procsperuser: {
weight: 0,
value: 0
},
psperuser: {
weight: 0,
value: 0
}
},
resource: {
weight: 1,
node: {
weight: 0,
value: 0
},
proc: {
weight: 0,
value: 1
},
memory: {
weight: 0,
value: 0
},
swap: {
weight: 0,
value: 0
},
disk: {
weight: 0,
value: 0
},
procsecond: {
weight: 0,
value: 3600
},
procequivalent: {
weight: 0,
value: 1
},
walltime: {
weight: 0,
value: 3600
}
},
usage: {
weight: 1,
consumed: {
weight: 0,
value: 0
},
remaining: {
weight: 0,
value: 0
},
percentconsumed: {
weight: 0,
value: 0
},
executiontime: {
weight: 0,
value: 0
}
}
}
}
}
4.9.2.D Get Job Analysis Information
The job-analysis parameter is used to get an analysis of the job's eligibility to run on the nodes managed by Moab.
URLs and Parameters
GET https://localhost:8080/mws/rest/jobs/<name>?api-version=3&job-analysis=true
Parameter | Required | Type | Value | Description |
---|---|---|---|---|
name | Yes | String | -- | Name of the job. |
See 3.3 Global URL Parameters for available URL parameters.
Sample Response
JSON response
------------------------------------
{
"name": "37",
"warnings": [
"job cannot run (job has hold in place)",
"job cannot run (insufficient available procs: 0 available)"
],
"nodes": [ {
"name": "node01",
"message": "node01 rejected: State (Busy)"
}]
}
The HTTP POST method is used to submit Jobs.
Quick Reference
POST https://localhost:8080/mws/rest/jobs?api-version=3[&proxy-user=<username>]
Restrictions
No more than one virtual container can be specified in the request. The virtual container must already exist.
The credentials.user and credentials.group properties are used to submit a job as the specified user belonging to the specified group.
Job variables have the following restrictions:
- Variable names cannot contain equals (=), semicolon (;), colon (:), plus (+), question mark (?), caret (^), backslash (\), or white space.
- Variable values cannot contain semicolon (;), colon (:), plus (+), or caret (^).
When submitting jobs, the only supported hold type is User.
The proxy-user parameter is ignored unless you set ENABLEPROXY=TRUE in the moab.cfg file, for example:
ADMINCFG[1] USERS=root,ted ENABLEPROXY=TRUE
URLs and Parameters
POST https://localhost:8080/mws/rest/jobs?api-version=3[&proxy-user=<username>]
Parameter | Required | Type | Value | Description |
---|---|---|---|---|
proxy-user | No | String | -- | Perform the action as this user. |
See 3.3 Global URL Parameters for available URL parameters.
Request Body
JSON request body (specified host list) ------------------------------------ { "attributes": [ "attr1", "attr2" ], "commandFile": "/tmp/test.sh", "commandScript": "c2xlZXAgNjAK", "commandLineArguments": "\"a b c\"", "credentials": { "account": "account", "group": "group", "jobClass": "BATCH", "qosRequested": "QOS1", "user": "saadmin" }, "customName": "custom_name_for_job", "dates": { "earliestRequestedStartDate": "2024-11-08 13:18:47 UTC", "deadlineDate": "2024-02-17 14:00:00 UTC" }, "dependencies": [ { "type": "set", "name": "vc1.varA" }, { "type": "set", "name": "vc2.varB" }, { "type": "set", "name": "vc3.varC" } ], "duration": 600, "emailNotifyAddresses": [ "[email protected]", "[email protected]" ], "emailNotifyTypes": [ "JobStart", "JobEnd" ], "environmentRequested": true, "environmentVariables": { "var1": "val1", "var2": "val2" }, "epilogScript": "/tmp/epilog.sh", "flags": [ "RESTARTABLE", "SUSPENDABLE" ], "holds": ["User"], "initialWorkingDirectory": "/tmp", "jobGroup": "job_group", "nodesExcluded": [ {"name": "node07"}, {"name": "node08"} ], "nodesRequested": [ {"name": "node01"}, {"name": "node02"} ], "nodesRequestedPolicy": "SUBSET", "partitionAccessListRequested": [ "p1", "p2" ], "priorities": {"user": -5}, "prologScript": "/tmp/prolog.sh", "requirements": [ { "architecture": "x86_64", "attributes":{ "matlab": [ { "restriction":"must", "comparator": "<=", "value": "7.1" } ], "soffice": [ { "restriction":"must", "comparator": "%=", "value": "3.1" } ] }, "featuresRequested": [ "a", "b", "c" ], "featuresRequestedMode": "OR", "featuresExcluded": [ "d", "e", "f" ], "featuresExcludedMode": "AND", "nodeAccessPolicy": "SINGLEJOB", "nodeAllocationPolicy": "PRIORITY", "nodeCount": 6, "nodeSet":"FIRSTOF:FEATURE:vlan2", "operatingSystem": "linux", "resourcesPerTask": { "disk": {"dedicated": 1024}, "memory": {"dedicated": 512}, "processors": {"dedicated": 2}, "swap": {"dedicated": 4096}, "matlab": {"dedicated": 6}, "intellij": {"dedicated": 2} "gpus": {"dedicated": 2} }, "taskCount": 4, "tasksPerNode": 14 }], "reservationRequested": {"name": "rsv.1"}, "resourceFailPolicy": "RETRY", "resourceManagerExtension": "x=PROC=4", "shellName": "/bin/bash", "standardErrorFilePath": "/tmp/error", "standardOutputFilePath": "/tmp/out", "templates": [ {"name": "template1"}, {"name": "template2"} ], "variables": { "var1": "val1", "var2": "val2" }, "virtualContainers": [{"name": "vc1"}] }
Sample Response
The response of this task is one of three possibilities:
- An object with a single messages property containing a list of error messages on failure:
- An object with a name property containing the name of the newly created job:
- An object with a name property and a virtualContainers list containing the name of the newly created virtual container:
{"messages":["Could not create job - invalid requirements"]}
{"name":"Moab.1"}
{ "name": "Moab.1", "virtualContainers": [{"name": "vc1"}] }

The virtual container will only be reported when a new virtual container has been created by Moab HPC Suite for the job.
This section includes some sample job submission requests.
Example 4-2: Submit job to run on node2 and node3
POST https://localhost:8080/mws/rest/jobs?api-version=3
------------------------------------
{
"commandFile": "/tmp/test.sh",
"credentials": {
"group": "adaptive",
"user": "adaptive"
},
"initialWorkingDirectory": "/tmp",
"nodesRequested": [
{"name": "node2"},
{"name": "node3"}
]
}
Example 4-3: Submit job that requires 20 processors
POST https://localhost:8080/mws/rest/jobs?api-version=3
------------------------------------
{
"commandFile": "/tmp/test.sh",
"credentials": {
"group": "adaptive",
"user": "adaptive"
},
"initialWorkingDirectory": "/tmp",
"requirements": [{"taskCount": 20}]
}
Example 4-4: Submit job to run after a certain time
POST https://localhost:8080/mws/rest/jobs?api-version=3
------------------------------------
{
"commandFile": "/tmp/test.sh",
"credentials": {
"group": "adaptive",
"user": "adaptive"
},
"dates": {"earliestRequestedStartDate": "2024-10-11 18:36:35 UTC"},
"initialWorkingDirectory": "/tmp",
"requirements": [{"taskCount": 20}]
}
Example 4-5: Submit job based on msub
Given this msub command:
msub -l nodes=3:ppn=2,walltime=1:00:00,pmem=100 script2.pbs.cmd
Here is an equivalent MWS request:
POST https://localhost:8080/mws/rest/jobs?api-version=3
------------------------------------
{
"duration": 3600,
"commandFile": "/home/adaptive/script2.pbs.cmd",
"credentials": {
"group": "adaptive",
"user": "adaptive"
},
"initialWorkingDirectory": "/home/adaptive",
"requirements": [ {
"resourcesPerTask": {"memory": {"dedicated": 100}},
"taskCount": 6,
"tasksPerNode": 2
}]
}

To emulate what msub does, make commandFile an absolute path, and add credentials.user, credentials.group, and initialWorkingDirectory. As shown above, nodes=3:ppn=2 is equivalent to setting taskCount to 6 and tasksPerNode to 2.
Example 4-6: Submit a job array (for information on how to submit a job array, see Submitting Job Arrays)
The HTTP PUT method is used to modify Jobs.
Quick Reference
PUT https://localhost:8080/mws/rest/jobs/<name>[/<modifyAction>]?api-version=3[&proxy-user=<username>]
Restrictions
The proxy-user parameter is ignored unless you set ENABLEPROXY=TRUE in the moab.cfg file, for example:
ADMINCFG[1] USERS=root,ted ENABLEPROXY=TRUE
URLs and Parameters
PUT https://localhost:8080/mws/rest/jobs/<name>?api-version=3[&proxy-user=<username>][&change-mode=set]
Parameter | Required | Type | Value | Description |
---|---|---|---|---|
name | Yes | String | -- | The name of the object. |
proxy-user | No | String | -- | Perform the action as this user. |
See 3.3 Global URL Parameters for available URL parameters.
Additional URL Parameters
Parameter | Required | Value | Description |
---|---|---|---|
change-mode | No |
set (default) add remove |
If set, replace all fields with the fields specified. If add, add the specified fields to existing fields. If remove, remove the specified fields from existing fields. |
Request Body
The request body below shows all the fields that are available when modifying a job, along with some sample values:
JSON request body ------------------------------------ { "credentials": { "account": "account", "jobClass": "BATCH", "qosRequested": "QOS1" }, "customName": "custom_name_for_job", "dates": {"earliestRequestedStartDate": "2024-11-08 13:18:47 UTC"}, "duration": 600, "flags": [ "RESTARTABLE", "SUSPENDABLE" ], "holds": ["User"], "messages": [ {"message": "Message one"}, {"message": "Message two"} ], "nodesRequested": [ {"name": "n015"}, {"name": "n016"}, {"name": "n017"}, {"name": "n018"} ], "partitionAccessListRequested": [ "p1", "p2" ], "priorities": { "system": 3, "user": -5 }, "requirements": [ { "features": [ "vlan1", "vlan2" ], "resourcesPerTask": { "matlab": {"dedicated": 1}, "tape": {"dedicated": 2} } }], "reservationRequested": {"name": "rsv.1"}, "variables": { "var1": "val1", "var2": "val2" } }
Sample Response

These messages might not match the messages returned from Moab HPC Suite exactly but are given as an example of the structure of the response.

Not all messages are shown for the above request body.
JSON response ------------------------------------ {"messages": [ "Account modified successfully", "Messages modified successfully", "Variables modified successfully" ]}
Restrictions
Old messages are not removed from jobs; only new messages are added.
Job variables have the restrictions documented in the section Submitting Jobs.
Although the client can modify features and resourcesPerTask, Moab only considers these elements when they appear in the first element of the requirements array. If the requirements array contains two or more elements, all elements but the first are silently ignored.
Jobs can require configurable, site-specific consumable resources called generic resources. For example, some jobs may require a matlab license. Only one job at a time can legally consume this license. Matlab is not a standard resource and may only be available on some sites. Nevertheless Moab HPC Suite allows this to be configured and tracked as is explained in 'Managing Consumable Generic Resources' in the Moab Workload Manager Administrator Guide.
You must specify generic resources in the requirements.resourcesPerTask portion of the JSON document. Any resource in requirements.resourcesPerTask that is not a standard resource is considered a generic resource. Standard resources include disk, memory, processors, and swap. Assume a job has the following in requirements.resourcesPerTask:
{
"resourcesPerTask":{
"processors":{
"dedicated":4,
"utilized":0
},
"memory":{
"dedicated":2048,
"utilized":0
},
"disk":{
"dedicated":4096,
"utilized":0
},
"swap":{
"dedicated":1024,
"utilized":0
},
"tape":{
"dedicated":1,
"utilized":0
},
"matlab":{
"dedicated":2,
"utilized":0
}
}
}
The standard resources the job requires are:
- 4 processors
- 2048 MB of memory
- 4096 MB of disk
- 1024 MB of swap
The generic resources the job requires are:
- 1 tape
- 2 matlab
To modify a job so that it requires 1 matlab license, run the following:
PUT https://localhost:8080/mws/rest/jobs/Moab.2?api-version=3
{
"requirements":[
{
"resourcesPerTask":{
"matlab":{
"dedicated":1
}
}
}
]
}
4.9.4.C Perform Actions on Job
URLs and Parameters
PUT https://localhost:8080/mws/rest/jobs/<name>/<modifyAction>?api-version=3[&proxy-user=<username>]
Parameter | Required | Type | Value | Description |
---|---|---|---|---|
name | Yes | String | -- | The name of the object. |
modifyAction | Yes | String |
cancel checkpoint execute hold requeue rerun resume suspend unhold |
If cancel, attempts to cancel the job (equivalent to deleting a job). If checkpoint, attempts to checkpoint the job. Note that the OS must support checkpointing for this to work. If execute, executes the job (if possible). If hold, attempts to hold the job using the holds set in the request body. If requeue, attempts to requeue the job. If rerun, attempts to rerun the job. If resume, attempts to resume the job. If suspend, attempts to suspend the job. If unhold, attempts to release the holds set in the request body. |
proxy-user | No | String | -- | Perform the action as this user. |
See 3.3 Global URL Parameters for available URL parameters.
Request Body
Request bodies are only required for holding or unholding jobs. All other actions do not require request bodies of any kind.
JSON request body to add holds to a job ------------------------------------ {"holds": ["User"]}
JSON request body to remove holds from a job ------------------------------------ {"holds": ["User"]}

If no holds are specified when unholding a job, all holds will be removed. This is equivalent to specifying holds as a list with a single element of All.
Sample Response

This message might not match the message returned from Moab HPC Suite exactly but is given as an example of the structure of the response.
JSON response ------------------------------------ {"messages": ["Job modified successfully"]}
4.9.5 Deleting (Canceling) Jobs
The HTTP DELETE method is used to cancel Jobs.
Quick Reference
DELETE https://localhost:8080/mws/rest/jobs/<name>?api-version=3[&proxy-user=<username>]
Restrictions
The proxy-user parameter is ignored unless you set ENABLEPROXY=TRUE in the moab.cfg file, for example:
ADMINCFG[1] USERS=root,ted ENABLEPROXY=TRUE
URLs and Parameters
DELETE https://localhost:8080/mws/rest/jobs/<name>?api-version=3[&proxy-user=<username>][&where={"state": "IDLE"}]
Parameter | Required | Type | Value | Description |
---|---|---|---|---|
name | Yes | String | -- | The name of the object. |
proxy-user | No | String | -- | Perform the action as this user. |
where | No | JSON | -- |
Cancel the job only if the where condition is satisfied. For example: To cancel the job only if it is idle, use where={"state": "IDLE"} ![]() The only supported where condition is "state". |
See 3.3 Global URL Parameters for available URL parameters.
Sample Response
JSON response for successful DELETE ------------------------------------ {}

Additional information about the DELETE can be found in the HTTP response header X-MWS-Message.
Related Topics