(Click to open topic with navigation)
This section describes behavior of the Job object in Moab Web Services. It contains the URLs, request bodies, and responses delivered to and from MWS.
The Job API is new with API version 2. The supported methods table below requires each resource to be accessed with a URL parameter of api-version=3 in order to behave as documented.
For more information, see Requesting Specific API Versions.
The Fields: Jobs reference contains the type and description of all fields in the Job object. It also contains details regarding which fields are valid during PUT and POST actions.
5.235.1 Supported Methods
Resource | GET | PUT | POST | DELETE |
---|---|---|---|---|
/rest/jobs | Get All Jobs | -- | Submit Job | -- |
/rest/jobs/<name> | -- | Cancel Job | ||
/rest/jobs/<name>/<modifyAction> | -- | Perform Actions on Job | -- | -- |
Detailed information on each of the supported methods are provided later in this topic, see:
5.235.2 Getting Job Information
The HTTP GET method is used to retrieve Job information. You can also use append the command with priority-analysis=true or node-analysis=true to get priority or eligibility information about the job.
Quick reference
GET http://localhost:8080/mws/rest/jobs/<name>?api-version=3
URLs and parameters
GET http://localhost:8080/mws/rest/jobs?api-version=3
Parameter | Required | Type | Description | Example |
---|---|---|---|---|
query | No | JSON |
Queries for specific results. It is possible to query by one or more fields based on MongoDB query syntax. |
query={"isActive":true} |
sort | No | JSON | Sort the results. Use 1 for ascending and -1 for descending. | sort={"name":-1} |
See Global URL Parameters for available URL parameters.
How to get all jobs
GET http://localhost:8080/mws/rest/jobs?api-version=3&fields=name,flags&max=3
------------------------------------
{
"totalCount": 8,
"resultCount": 3,
"results": [
{
"flags": ["GLOBALQUEUE"],
"name": "Moab.1"
},
{
"flags": ["GLOBALQUEUE"],
"name": "Moab.2"
},
{
"flags": ["GLOBALQUEUE"],
"name": "Moab.4"
}
]
}
How to get a subset of jobs
Get active jobs
------------------------------------
http://localhost:8080/mws/rest/jobs?api-version=3&query={"isActive":true}
Get completed jobs
------------------------------------
http://localhost:8080/mws/rest/jobs?api-version=3&query={"isActive":false}
Get jobs owned by a particular user
------------------------------------
http://localhost:8080/mws/rest/jobs?api-version=3&query={"credentials.user":"fred"}
Known issues
Some jobs are not returned if DisplayFlags UseBlocking is set in the moab.cfg file.
URLs and Parameters
GET http://localhost:8080/mws/rest/jobs/<name>?api-version=3
Parameter | Required | Type | Valid values | Description |
---|---|---|---|---|
name | Yes | String | -- | The name of the object. |
See Global URL Parameters for available URL parameters.
Sample response
JSON response ------------------------------------ { "arrayIndex": null, "arrayMasterName": null, "attributes": [], "blocks": > { "category": "jobBlock"> "createdDate": "2016-06-22 19:08:30 UTC", "message": null, "partition": null, "type": null } ]> "bypassCount": 0, "cancelCount": 0, "commandFile": "/tmp/test.sh", "commandLineArguments": null, "completionCode": null, "cpuTime": 0, "credentials": { "account": null, "group": "adaptive", "jobClass": null, "qos": "NONE", "qosRequested": null, "user": "adaptive" }, "customName": null, "dates": { "completedDate": null, "createdDate": "2012-10-11 17:58:16 UTC", "deadlineDate": "2037-10-24 12:26:40 UTC", "dispatchedDate": null, "earliestRequestedStartDate": null, "earliestStartDate": "2012-10-11 17:58:18 UTC", "eligibleDate": "2012-10-11 17:59:19 UTC", "lastCanceledDate": null, "lastChargedDate": null, "lastPreemptedDate": null, "lastUpdatedDate": "2012-10-11 17:59:19 UTC", "startDate": null, "submitDate": "2012-10-11 17:58:16 UTC", "terminationDate": "2037-10-24 12:26:40 UTC" }, "deferCount": 0, "dependencies": [], "description": null, "duration": 8639999, "durationActive": 0, "durationMinimum": 0, "durationQueued": 31, "durationRemaining": 0, "durationSuspended": 0, "emailNotifyAddresses": [], "emailNotifyTypes": [], "environmentRequested": false, "environmentVariables": {}, "epilogScript": null, "flags": [ "GLOBALQUEUE" ], "holdDate": null, "holdReason": null, "holds": [], "initialWorkingDirectory": "/tmp", "isActive": true, "jobGroup": null, "masterNode": null, "memorySecondsDedicated": 0, "memorySecondsUtilized": 0, "messages": [], "migrateCount": 0, "minimumPreemptTime": 0, "mwmName": "Moab", "name": "Moab.15", "nodesExcluded": [], "nodesRequested": [], "nodesRequestedPolicy": null, "partitionAccessList": [ "torque" ], "partitionAccessListRequested": [ "mws", "torque", "SHARED" ], "partitionAccessListScheduler": [ "mws", "torque", "SHARED" ], "preemptCount": 0, "priorities": { "run": 0, "start": 1, "system": 0, "user": 0 }, "processorSecondsDedicated": 0, "processorSecondsLimit": 0, "processorSecondsUtilized": 0, "prologScript": null, "queueStatus": "blocked", "rank": 0, "rejectPolicies": [], "requirements": [ { "architecture": null, "attributes": { "matlab": [ { "comparator": "<=", "displayValue": null, "restriction": "must", "value": "7.1" > ], "soffice": [ { "comparator": "%=", "displayValue": null, "restriction": "must", "value": "3.1" } ] }, "dedicateAllProcessors": true, "features": [], "featuresExcluded": [], "featuresExcludedMode": "AND", "featuresRequested": [], "featuresRequestedMode": "AND", "image": null, "index": 0, "metrics": {}, "nodeAccessPolicy": null, "nodeAllocationPolicy": null, "nodeCount": 0, "nodeSet": null, "nodes": [], "reservation": null, "resourcesPerTask": { "disk": { "dedicated": 0, "utilized": null }, "memory": { "dedicated": 0, "utilized": 0 }, "processors": { "dedicated": 1, "utilized": 0 }, "swap": { "dedicated": 0, "utilized": null } }, "taskCount": 4, "tasksPerNode": 0, "totalDedicatedProcessors": 1 } ], "reservationRequested": null, "resourceFailPolicy": null, "resourceManagerExtension": null, "resourceManagers": [ { "isDestination": false, "isSource": true, "jobName": "Moab.15", "name": "internal" } ], "shellName": "/bin/bash", "standardErrorFilePath": null, "standardOutputFilePath": null, "startCount": 0, "states": { "state": "Idle", "stateExpected": "Idle", "stateLastUpdatedDate": null, "subState": null }, "submitCommandFile": "/home/ace/jobscript.sh", "submitHost": "0:0:0:0:0:0:0:1", "systemJobAction": null, "systemJobType": null, "targetedJobAction": null, "targetedJobName": null, "templates": [ { "name": "DEFAULT" } ], "triggers": [], "variables": {}, "virtualContainers": [], "virtualMachines": [], "vmUsagePolicy": null }
Job arrays
Job array master ------------------------------------ { "name": "Moab.5", "customName": "myarray", "flags": [ "ARRAYMASTER", "GLOBALQUEUE", "CANCELONFIRSTFAILURE", "CANCELONANYSUCCESS" ] }
Array sub-job ------------------------------------ { "name": "Moab.5[21]", "customName": "myarray", "arrayIndex": 21, "arrayMasterName": "Moab.5", "flags": [ "ARRAYJOB", "GLOBALQUEUE", "CANCELONFIRSTFAILURE", "CANCELONANYSUCCESS" ] }
5.235.2.C Get Job Priority Information
The priority-analysis parameter is used to get job priority information.
URLs and Parameters
GET http://localhost:8080/mws/rest/jobs/<name>?api-version=3&priority-analysis=true
Parameter | Required | Type | Valid values | Description |
---|---|---|---|---|
name | Yes | String | -- | The name of the job. |
See Global URL Parameters for available URL parameters.
Sample Response
JSON response
------------------------------------
{
priorities: {
start: 36,
system: 0,
components: {
service: {
weight: 2,
queuetime: {
weight: 1,
value: 33
},
xfactor: {
weight: 0,
value: 1.559722
},
deadline: {
weight: 0,
value: 0
},
policyviolation: {
weight: 0,
value: 0
},
userprior: {
weight: 3,
value: -5
},
startcount: {
weight: 0,
value: 0
},
bypass: {
weight: 0,
value: 0
}
},
target: {
weight: 1,
queuetime: {
weight: 0,
value: 0
},
xfactor: {
weight: 0,
value: 0
}
},
credential: {
weight: 1,
user: {
weight: 0,
value: 0
},
group: {
weight: 0,
value: 0
},
account: {
weight: 0,
value: 0
},
qos: {
weight: 0,
value: 0
}
},
attribute: {
weight: 1,
attribute: {
weight: 0,
value: 0
},
gres: {
weight: 0,
value: 0
},
jobid: {
weight: 0,
value: 0
},
jobname: {
weight: 0,
value: 0
},
state: {
weight: 0,
value: 0
}
},
fairshare: {
weight: 1,
user: {
weight: 0,
value: 0
},
group: {
weight: 0,
value: 0
},
account: {
weight: 0,
value: 0
},
qos: {
weight: 0,
value: 0
},
guser: {
weight: 0,
value: 0
},
ggroup: {
weight: 0,
value: 0
},
gaccount: {
weight: 0,
value: 0
},
userwcacc: {
weight: 0,
value: 0
},
jobsperuser: {
weight: 0,
value: 0
},
jobsrunningperuser: {
weight: 0,
value: 0
},
procsperuser: {
weight: 0,
value: 0
},
psperuser: {
weight: 0,
value: 0
}
},
resource: {
weight: 1,
node: {
weight: 0,
value: 0
},
proc: {
weight: 0,
value: 1
},
memory: {
weight: 0,
value: 0
},
swap: {
weight: 0,
value: 0
},
disk: {
weight: 0,
value: 0
},
procsecond: {
weight: 0,
value: 3600
},
procequivalent: {
weight: 0,
value: 1
},
walltime: {
weight: 0,
value: 3600
}
},
usage: {
weight: 1,
consumed: {
weight: 0,
value: 0
},
remaining: {
weight: 0,
value: 0
},
percentconsumed: {
weight: 0,
value: 0
},
executiontime: {
weight: 0,
value: 0
}
}
}
}
}
5.235.2.D Get Job Analysis Information
The job-analysis parameter is used to get an analysis of the job's eligibility to run on the nodes managed by Moab.
URLs and parameters
GET http://localhost:8080/mws/rest/jobs/<name>?api-version=3&job-analysis=true
Parameter | Required | Type | Valid values | Description |
---|---|---|---|---|
name | Yes | String | -- | Name of the job. |
See Global URL Parameters for available URL parameters.
Sample Response
JSON response
------------------------------------
{
"name": "37",
"warnings": [
"job cannot run (job has hold in place)",
"job cannot run (insufficient available procs: 0 available)"
],
"nodes": [ {
"name": "node01",
"message": "node01 rejected: State (Busy)"
}]
}
The HTTP POST method is used to submit Jobs.
Quick reference
POST http://localhost:8080/mws/rest/jobs?api-version=3[&proxy-user=<username>]
Restrictions
ADMINCFG[1] USERS=root,ted ENABLEPROXY=TRUE
URLs and parameters
POST http://localhost:8080/mws/rest/jobs?api-version=3[&proxy-user=<username>]
Parameter | Required | Type | Valid values | Description |
---|---|---|---|---|
proxy-user | No | String | -- | Perform the action as this user. |
See Global URL Parameters for available URL parameters.
Request body
JSON request body (specified host list) ------------------------------------ { "attributes": [ "attr1", "attr2" ], "commandFile": "/tmp/test.sh", "commandScript": "c2xlZXAgNjAK", "commandLineArguments": "\"a b c\"", "credentials": { "account": "account", "group": "group", "jobClass": "BATCH", "qosRequested": "QOS1", "user": "saadmin" }, "customName": "custom_name_for_job", "dates": { "earliestRequestedStartDate": "2012-11-08 13:18:47 UTC", "deadlineDate": "2014-02-17 14:00:00 UTC" }, "dependencies": [ { "type": "set", "name": "vc1.varA" }, { "type": "set", "name": "vc2.varB" }, { "type": "set", "name": "vc3.varC" } ], "duration": 600, "emailNotifyAddresses": [ "[email protected]", "[email protected]" ], "emailNotifyTypes": [ "JobStart", "JobEnd" ], "environmentRequested": true, "environmentVariables": { "var1": "val1", "var2": "val2" }, "epilogScript": "/tmp/epilog.sh", "flags": [ "RESTARTABLE", "SUSPENDABLE" ], "holds": ["User"], "initialWorkingDirectory": "/tmp", "jobGroup": "job_group", "nodesExcluded": [ {"name": "node07"}, {"name": "node08"} ], "nodesRequested": [ {"name": "node01"}, {"name": "node02"} ], "nodesRequestedPolicy": "SUBSET", "partitionAccessListRequested": [ "p1", "p2" ], "priorities": {"user": 5}, "prologScript": "/tmp/prolog.sh", "requirements": [ { "architecture": "x86_64", "attributes":{ "matlab": [ { "restriction":"must", "comparator": "<=", "value": "7.1" } ], "soffice": [ { "restriction":"must", "comparator": "%=", "value": "3.1" } ] }, "featuresRequested": [ "a", "b", "c" ], "featuresRequestedMode": "OR", "featuresExcluded": [ "d", "e", "f" ], "featuresExcludedMode": "AND", "nodeAccessPolicy": "SINGLEJOB", "nodeAllocationPolicy": "PRIORITY", "nodeCount": 6, "nodeSet":"FIRSTOF:FEATURE:vlan2", "image": "linux", "resourcesPerTask": { "disk": {"dedicated": 1024}, "memory": {"dedicated": 512}, "processors": {"dedicated": 2}, "swap": {"dedicated": 4096}, "matlab": {"dedicated": 6}, "intellij": {"dedicated": 2} }, "taskCount": 4, "tasksPerNode": 14 }], "reservationRequested": {"name": "rsv.1"}, "resourceFailPolicy": "RETRY", "resourceManagerExtension": "x=PROC=4", "shellName": "/bin/bash", "standardErrorFilePath": "/tmp/error", "standardOutputFilePath": "/tmp/out", "templates": [ {"name": "template1"}, {"name": "template2"} ], "variables": { "var1": "val1", "var2": "val2" }, "virtualContainers": [{"name": "vc1"}], "vmUsagePolicy": "CREATEVM" }
Sample response
The response of this task is one of three possibilities:
{"messages":["Could not create job - invalid requirements"]}
{"name":"Moab.1"}
{ "name": "Moab.1", "virtualContainers": [{"name": "vc1"}] }
The virtual container will only be reported when a new virtual container has been created by Moab for the job.
This section includes some sample job submission requests.
Example 5-150: Submit job to run on node2 and node3
POST http://localhost:8080/mws/rest/jobs?api-version=3
------------------------------------
{
"commandFile": "/tmp/test.sh",
"credentials": {
"group": "adaptive",
"user": "adaptive"
},
"initialWorkingDirectory": "/tmp",
"nodesRequested": [
{"name": "node2"},
{"name": "node3"}
]
}
Example 5-151: Submit job that requires 20 processors
POST http://localhost:8080/mws/rest/jobs?api-version=3
------------------------------------
{
"commandFile": "/tmp/test.sh",
"credentials": {
"group": "adaptive",
"user": "adaptive"
},
"initialWorkingDirectory": "/tmp",
"requirements": [{"taskCount": 20}]
}
Example 5-152: Submit job to run after a certain time
POST http://localhost:8080/mws/rest/jobs?api-version=3
------------------------------------
{
"commandFile": "/tmp/test.sh",
"credentials": {
"group": "adaptive",
"user": "adaptive"
},
"dates": {"earliestRequestedStartDate": "2012-10-11 18:36:35 UTC"},
"initialWorkingDirectory": "/tmp",
"requirements": [{"taskCount": 20}]
}
Example 5-153: Submit job based on msub example
Given this msub command:
msub -l nodes=3:ppn=2,walltime=1:00:00,pmem=100 script2.pbs.cmd
Here is an equivalent MWS request:
POST http://localhost:8080/mws/rest/jobs?api-version=3
------------------------------------
{
"duration": 3600,
"commandFile": "/home/adaptive/script2.pbs.cmd",
"credentials": {
"group": "adaptive",
"user": "adaptive"
},
"initialWorkingDirectory": "/home/adaptive",
"requirements": [ {
"resourcesPerTask": {"memory": {"dedicated": 100}},
"taskCount": 6,
"tasksPerNode": 2
}]
}
To emulate what msub does, make commandFile an absolute path, and add credentials.user, credentials.group, and initialWorkingDirectory.
As shown above, nodes=3:ppn=2 is equivalent to setting taskCount to 6 and tasksPerNode to 2.
Example 5-154: Submit a job array
For information on how to submit a job array, see Submitting Job Arrays.
The HTTP PUT method is used to modify Jobs.
Quick reference
PUT http://localhost:8080/mws/rest/jobs/<name>[/<modifyAction>]?api-version=3[&proxy-user=<username>]
Restrictions
The proxy-user parameter is ignored unless you set ENABLEPROXY=TRUE in the moab.cfg file. For example:
ADMINCFG[1] USERS=root,ted ENABLEPROXY=TRUE
5.235.4.A Modify Job Attributes
URLs and parameters
PUT http://localhost:8080/mws/rest/jobs/<name>?api-version=3[&proxy-user=<username>][&change-mode=set]
Parameter | Required | Type | Valid values | Description |
---|---|---|---|---|
name | Yes | String | -- | The name of the object. |
proxy-user | No | String | -- | Perform the action as this user. |
See Global URL Parameters for available URL parameters.
Additional URL parameters
Parameter | Required | Valid values | Description |
---|---|---|---|
change-mode | No |
set (default) add remove |
If set, replace all fields with the fields specified. If add, add the specified fields to existing fields. If remove, remove the specified fields from existing fields. |
Request body
The request body below shows all the fields that are available when modifying a job, along with some sample values.
JSON request body ------------------------------------ { "credentials": { "account": "account", "jobClass": "BATCH", "qosRequested": "QOS1" }, "customName": "custom_name_for_job", "dates": {"earliestRequestedStartDate": "2012-11-08 13:18:47 UTC"}, "duration": 600, "flags": [ "RESTARTABLE", "SUSPENDABLE" ], "holds": ["User"], "messages": [ {"message": "Message one"}, {"message": "Message two"} ], "nodesRequested": [ {"name": "n015"}, {"name": "n016"}, {"name": "n017"}, {"name": "n018"} ], "partitionAccessListRequested": [ "p1", "p2" ], "priorities": { "system": 3, "user": 5 }, "requirements": [ { "features": [ "vlan1", "vlan2" ], "resourcesPerTask": { "matlab": {"dedicated": 1}, "tape": {"dedicated": 2} } }], "reservationRequested": {"name": "rsv.1"}, "variables": { "var1": "val1", "var2": "val2" } }
Sample response
These messages may not match the messages returned from Moab exactly, but are given as an example of the structure of the response.
Not all messages are shown for the above request body.
JSON response ------------------------------------ {"messages": [ "Account modified successfully", "Messages modified successfully", "Variables modified successfully" ]}
Restrictions
Although the client can modify features and resourcesPerTask, Moab only considers these elements when they appear in the first element of the requirements array. If the requirements array contains two or more elements, all elements but the first are silently ignored.
Jobs can require configurable, site-specific consumable resources called generic resources. For example, some jobs may require a matlab license. Only one job at a time may legally consume this license. Matlab is not a standard resource and may only be available on some sites. Nevertheless Moab allows this to be configured and tracked as is explained in Managing Consumable Generic Resources.
You must specify generic resources in the requirements.resourcesPerTask portion of the JSON document. Any resource in requirement.resourcesPerTask that is not a standard resource is considered a generic resource. Standard resources include disk, memory, processors, and swap. Assume a job has the following in requirement.resourcesPerTask:
{
"resourcesPerTask":{
"processors":{
"dedicated":4,
"utilized":0
},
"memory":{
"dedicated":2048,
"utilized":0
},
"disk":{
"dedicated":4096,
"utilized":0
},
"swap":{
"dedicated":1024,
"utilized":0
},
"tape":{
"dedicated":1,
"utilized":0
},
"matlab":{
"dedicated":2,
"utilized":0
}
}
}
The standard resources the job requires are:
The generic resources the job requires are
To modify a job so that it requires 1 matlab license, run the following:
PUT http://localhost:8080/mws/rest/jobs/Moab.2?api-version=3
{
"requirements":[
{
"resourcesPerTask":{
"matlab":{
"dedicated":1
}
}
}
]
}
5.235.4.C Perform Actions on Job
URLs and parameters
PUT http://localhost:8080/mws/rest/jobs/<name>/<modifyAction>?api-version=3[&proxy-user=<username>]
Parameter | Required | Type | Valid values | Description |
---|---|---|---|---|
name | Yes | String | -- | The name of the object. |
modifyAction | Yes | String |
cancel checkpoint execute hold requeue rerun resume suspend unhold |
If cancel, attempts to cancel the job (equivalent to deleting a job). If checkpoint, attempts to checkpoint the job. Note that the OS must support checkpointing for this to work. If execute, executes the job (if possible). If hold, attempts to hold the job using the holds set in the request body. If requeue, attempts to requeue the job. If rerun, attempts to rerun the job. If resume, attempts to resume the job. If suspend, attempts to suspend the job. If unhold, attempts to release the holds set in the request body. |
proxy-user | No | String | -- | Perform the action as this user. |
See Global URL Parameters for available URL parameters.
Request body
Request bodies are only required for holding or unholding jobs. All other actions do not require request bodies of any kind.
JSON request body to add holds to a job ------------------------------------ {"holds": ["User"]}
JSON request body to remove holds from a job ------------------------------------ {"holds": ["User"]}
If no holds are specified when unholding a job, all holds will be removed. This is equivalent to specifying holds as a list with a single element of All.
Sample response
This message may not match the message returned from Moab exactly, but is given as an example of the structure of the response.
JSON response ------------------------------------ {"messages": ["Job modified successfully"]}
5.235.5 Deleting (Canceling) Jobs
The HTTP DELETE method is used to cancel Jobs.
Quick reference
DELETE http://localhost:8080/mws/rest/jobs/<name>?api-version=3[&proxy-user=<username>]
Restrictions
The proxy-user parameter is ignored unless you set ENABLEPROXY=TRUE in the moab.cfg file. For example:
ADMINCFG[1] USERS=root,ted ENABLEPROXY=TRUE
URLs and parameters
DELETE http://localhost:8080/mws/rest/jobs/<name>?api-version=3[&proxy-user=<username>][&where={"state": "IDLE"}]
Parameter | Required | Type | Valid values | Description |
---|---|---|---|---|
name | Yes | String | -- | The name of the object. |
proxy-user | No | String | -- | Perform the action as this user. |
where | No | JSON | -- |
Cancel the job only if the where condition is satisfied. For example: To cancel the job only if it is idle, use where={"state": "IDLE"}The only supported where condition is "state". |
See Global URL Parameters for available URL parameters.
Sample response
JSON response for successful DELETE ------------------------------------ {}
Additional information about the DELETE can be found in the HTTP response header X-MWS-Message.
Related Topics