20.1 Configuring VPC Profiles |
|
20.1 Configuring VPC Profiles |
|
Use the VCPROFILE parameter to configure a VPC profile. At its simplest, a VPC profile is a single line that gives the profile a name. The profile can then be requested by name and the user can fill in any other details required. The DESCRIPTION parameter is helpful, but not required. Users will be able to see the description in the listing of VPCs.
Example: A simple VPC with a description
VCPROFILE[example1] DESCRIPTION="A Dedicated Compute Node"
The VPC can then be requested on the command line or through Moab Access Portal. Moab Access Portal provides a customizable portal where users can create, modify and manage VPCs. For more information, see Creating VPCs.
| ACL | |||
| Format: | List of zero or more comma-delimited
<OTYPE>==<OID>[:<OID>] pairs where <OTYPE> is one of the
following: USER, GROUP, ACCT, CLASS, or QOS |
||
| Default: | --- | ||
| Details: | Specifies which credentials can view this profile and can create VPCs using this profile. | ||
| Example: |
VCPROFILE[basic] ACL=USER==steve:john,GROUP==staff Users steve and john and any member of the group staff can use VPC resources.
|
||
| DESCRIPTION | |||
| Format: | <STRING> | ||
| Default: | --- | ||
| Details: | Arbitrary human readable string describing VPC profile usage.
|
||
| Example: |
VCPROFILE[dataset3] DESCRIPTION="custom 1 TB data mining environment with level 3 security" End-users will see this description when specifying which VPC package to select. |
||
| NODEHOURCHARGERATE | |||
| Format: | <DOUBLE> | ||
| Default: | --- | ||
| Details: | Per node chargerate multiplier associated with using resources and services in the specified VPC. See the Allocation Management overview for more information. | ||
| Example: |
VCPROFILE[dataset1] NODEHOURCHARGERATE=1.5 VCPROFILE[dataset2] NODEHOURCHARGERATE=4.0 VCPROFILE[dataset3] NODEHOURCHARGERATE=6.0 Accounts associated with the creation of each data VPC will be charged for VPC utilization with the associated charge multiplier. |
||
| NODESETLIST | |||
| Format: | <FEATURE>[,<FEATURE>]... | ||
| Default: | --- | ||
| Details: | Nodeset constraints to be applied to explicitly specified reqs in format.
|
||
| Example: |
VCPROFILE[dataset3] NODESETLIST=fastio,myrinet Resources will either be allocated entirely from the fastio or the myrinet node set. |
||
| NODESETUPCOST | |||
| Format: | <DOUBLE> | ||
| Default: | --- | ||
| Details: | Per node cost in credits associated with provisioning resources and services to enable the associated VPC. See the Allocation Management overview for more information. | ||
| Example: |
VCPROFILE[dataset1] NODESETUPCOST=20.5 VCPROFILE[dataset2] NODESETUPCOST=20.5 VCPROFILE[dataset3] NODESETUPCOST=65.0 Accounts associated with the creation of a new VPC will be assessed a per-node charge for each node in a newly created VPC associated with the overhead of setting up the needed environment. |
||
| OPRSVPROFILE | |||
| Format: | <RSVPROFILEID> | ||
| Default: | --- | ||
| Details: | The default reservation profile for resource requirements that are explicitly specified by the user. | ||
| Example: |
VCPROFILE[dataset3] OPRSVPROFILE=fastp Resource requirements that are explicitly specified by the requestor and that do not have an explicit reservation profile will use the fastpreservation profile. |
||
| QUERYDATA | |||
| Format: | <RESOURCE_QUERY_STRING> (See mshow command usage.) | ||
| Default: | --- | ||
| Details: | List of requirements (reqs) to be co-allocated along with explicitly specified requirements. | ||
| Example: |
VCPROFILE[pkgA] QUERYDATA=minprocs=1,nodefeature=ionodes,duration=00:30:00 VCPROFILE[pkgA] QUERYDATA=mintasks=1,gres=vlan:1,duration=01:00:00 VCPROFILE[pkgA] QUERYDATA=minprocs=1,nodefeature=ionodes, duration=00:30:00 VCPROFILE[pkgA] QUERYDATA=mintasks=1,gres=vlan:1,duration=01:00:00 One ionode will be allocated for the first 30 minutes of the VPC and one vlan generic resource will be allocated for the first hour.
|
||
| REQENDPAD | |||
| Format: | [[[DD:]HH:]MM:]SS | ||
| Default: | --- | ||
| Details: | Amount of additional time required after each explicitly specified requirement. | ||
| Example: |
VCPROFILE[premium] REQENDPAD=00:05:00 Five minutes of additional walltime will be appended to the end of each explicitly specified requirement. |
||
| REQDEFATTR | |||
| Format: | <RESOURCE_LIST> | ||
| Default: | --- | ||
| Details: | A list of comma-delimited resource requirements that are merged with the resource request requirements in the corresponding mshow -a command. These default resources can be overridden when a VPC is requested. | ||
| Example: |
VCPROFILE[database] REQDEFATTR=duration=1:00:00:00 Indicates a database profile with a default duration of one day. User can specify a shorter or longer duration when the VPC is requested. |
||
| REQSETATTR | |||
| Format: | <RESOURCE_LIST> | ||
| Default: | --- | ||
| Details: | A list of comma-delimited resource requirements that are merged with the resource request requirements in the corresponding mshow -a command. These resources cannot be overridden when the VPC is requested. | ||
| Example: |
VCPROFILE[Large] REQSETATTR=mintasks=4,gres=tape:2 A request for four tasks and two generic resources named tape are appended to the user-specified resource request. |
||
| REQSTARTPAD | |||
| Format: | [[[DD:]HH:]MM:]SS | ||
| Default: | --- | ||
| Details: | Amount of additional time required before each explicitly specified req. | ||
| Example: |
VCPROFILE[premium] REQSTARTPAD=00:15:00 Fifteen minutes of additional walltime will be prepended to the start of each explicitly specified requirement. |
||
The reservation profiles of the VPC set with the QUERYDATA or OPRSVPROFILE attributes can be used to manage provisioning of VPC resources. This provisioning is generally enabled via the use of triggers although it may also be enabled through a resource manager interface.
Any number of resources or other attributes can be associated with a VPC profile. Examples include the number of nodes required, the duration of the VPC, or required node features. Default attributes that can be overridden when the VPC is requested are specified with REQDEFATTR. Required attributes that cannot be overridden are specified with REQSETATTR.
For resources that need to be allocated once per request (coallocation), see Coallocation of Additional Resources.
Example: Database server that will only run on specific nodes
VCPROFILE[database] DESCRIPTION="Database Server with Fast IO" VCPROFILE[database] REQSETATTR=nodefeature=fastio VCPROFILE[database] REQDEFATTR=duration=1:00:00:00
The preceding example indicates a database profile with a default duration of one day. The user can specify a shorter or longer duration when the VPC is requested. The database profile will only run on nodes with the fastio feature, which cannot be overridden by the user.
When a VPC is created, a reservation is created on the cluster. A reservation profile can be specified for the reservation. Setup and teardown actions, as well as other attributes, can be configured as part of the reservation profile. Reservation profiles are specified using the OPRSVPROFILE parameter.
Setup and teardown actions are specified using triggers. Triggers allow actions ranging from simple notification to complex workflows. If the setup time for the VPC is significant, the REQSTARTPAD and REQENDPAD parameters can be used to compensate the user for lost time. For actions that take place when the VPC is cancelled, a special variable (ACTIVE) can be passed to the trigger script. ACTIVE is set to the string TRUE if the VPC is currently active, or UNDEFINED if it is not. For more information see trigger variables.
Reservation profiles also allow a variety of flags to be specified. For more information see Using Reservation Profiles.
Example: A VPC that provisions an operating system and logs information
VCPROFILE[vpctest] DESCRIPTION="Test Environment" VCPROFILE[vpctest] OPRSVPROFILE=test RSVPROFILE[rsvtest] FLAGS=SYSTEMJOB RSVPROFILE[rsvtest] TRIGGER=ATYPE=exec,Action="$HOME/logdate.pl TEST START $VPCHOSTLIST $OID $HOSTLIST $ACTIVE",EType=start RSVPROFILE[rsvtest] TRIGGER=ATYPE=exec,Action="$HOME/installos.pl $HOSTLIST rhel",EType=start RSVPROFILE[rsvtest] TRIGGER=ATYPE=exec,Action="$HOME/logdate.pl TEST END $VPCHOSTLIST $OID $HOSTLIST $ACTIVE",EType=end RSVPROFILE[rsvtest] TRIGGER=ATYPE=exec,Action="$HOME/logdate.pl TEST CANCEL $VPCHOSTLIST $OID $HOSTLIST $ACTIVE",EType=cancel
In the preceding example, a logging script (logdate.pl) is called when the VPC starts, ends, or is cancelled. An operating system is installed on all nodes in the VPC when it starts.
Additional units of other resources can also be packaged with VPC on a per instance basis. For example, if a user requests a four-node cluster, a head node (in addition to the four requested nodes) can be allocated automatically. This is done using the QUERYDATA parameter.
Example: Database profile with a single server and multiple client nodes
VCPROFILE[database] DESCRIPTION="Database client/server package" VCPROFILE[database] QUERYDATA=minnodes=1,label=server,rsvprofile=dbserver VCPROFILE[database] OPRSVPROFILE=dbclient
In the preceding example, a database profile is configured that will give the user the number of database clients that they request (the dbclient profile), plus a single database server.
Example: LAMP stack profile with multiple parts
VCPROFILE[lampstack] DESCRIPTION="LAMP stack" VCPROFILE[lampstack] OPRSVPROFILE=lampstack VCPROFILE[lampstack] NODEHOURCHARGERATE=6.6 NODESETUPCOST=66.6 VCPROFILE[lampstack] FLAGS=AUTOACCESS VCPROFILE[lampstack] QUERYDATA=minnodes=1,label=apache_lb,rsvprofile=apache_lb VCPROFILE[lampstack] QUERYDATA=minnodes=3,label=apache_pool,rsvprofile=apache_pool VCPROFILE[lampstack] QUERYDATA=minnodes=1,label=db_lb,rsvprofile=db_lb VCPROFILE[lampstack] QUERYDATA=minnodes=2,label=db_pool,rsvprofile=db_pool VCPROFILE[lampstack] QUERYDATA=minnodes=1,label=db_backup,rsvprofile=db_backup VCPROFILE[lampstack] QUERYDATA=minnodes=2,label=script_compute,rsvprofile=script_compute RSVPROFILE[rsvlampstack] FLAGS=EXCLUDEMYGROUP,SYSTEMJOB RSVPROFILE[rsvlampstack] TRIGGER=ATYPE=exec,Action="/opt/moab/tools/lamp/lampstack.py --etype=$ETYPE --vpcid=$VPCID --hostlist=$HOSTLIST",EType=start,sets=^lampstack,requires=apache_lb.db_lb RSVPROFILE[rsvlampstack] TRIGGER=ATYPE=exec,Action="/opt/moab/tools/lamp/lampstack_shepherd.py --vpcid=$VPCID",EType=start,MultiFire=TRUE,RearmTime=05:00,sets=!badlamp,requires=lampstack RSVPROFILE[rsvlampstack] TRIGGER=ATYPE=mail,Action="There was a problem with this vpc: $VPCID",EType=start,requires=badlamp,unsets=badlamp RSVPROFILE[rsvlampstack] TRIGGER=ATYPE=exec,Action="mschedctl -d vpc:$VPCID",EType=start,requires=selfdestruct RSVPROFILE[apache_lb] TRIGGER=ATYPE=exec,Action="/opt/moab/tools/lamp/apache_lb_ctl.py --etype=$ETYPE --vpcid=$VPCID --hostlist=$HOSTLIST --pool=$apache_pool",EType=start,sets=^apache_lb,requires=apache_pool.script_compute RSVPROFILE[apache_pool] TRIGGER=ATYPE=exec,Action="/opt/moab/tools/lamp/apache_pool_ctl.py --etype=$ETYPE --vpcid=$VPCID --hostlist=$HOSTLIST",EType=start,sets=^apache_pool RSVPROFILE[script_compute] TRIGGER=ATYPE=exec,Action="/opt/moab/tools/lamp/script_compute.py --etype=$ETYPE --db_lb=$db_lb --vpcid=$VPCID --hostlist=$HOSTLIST",EType=start,requires=db_lb,sets=^script_compute RSVPROFILE[db_lb] TRIGGER=ATYPE=exec,Action="/opt/moab/tools/lamp/db_lb_ctl.py --etype=$ETYPE --vpcid=$VPCID --hostlist=$HOSTLIST --pool=$db_pool --backup=$db_backup --active=$ACTIVE",EType=start,sets=^db_lb,requires=db_pool.db_backup RSVPROFILE[db_backup] TRIGGER=ATYPE=exec,Action="/opt/moab/tools/lamp/db_backup_ctl.py --etype=$ETYPE --vpcid=$VPCID --hostlist=$HOSTLIST",EType=start,sets=^db_backup RSVPROFILE[db_pool] TRIGGER=ATYPE=exec,Action="/opt/moab/tools/lamp/db_pool_zero.py --etype=$ETYPE --vpcid=$VPCID",EType=start,sets=db_pool_go RSVPROFILE[db_pool] TRIGGER=ATYPE=exec,Action="/opt/moab/tools/lamp/db_pool_ctl.py --etype=$ETYPE --vpcid=$VPCID --hostlist=$HOSTLIST",EType=start,MultiFire=TRUE,sets=^db_pool.!db_pool_retry,unsets=db_pool_go.!db_pool_go,requires=db_pool_go RSVPROFILE[db_pool] TRIGGER=ATYPE=exec,Action="/opt/moab/tools/lamp/db_pool_retry.py --etype=$ETYPE --vpcid=$VPCID",EType=start,MultiFire=TRUE,sets=^db_pool_go.!^selfdestruct,unsets=db_pool_retry.!db_pool_retry,requires=db_pool_retry
VCPROFILE[lampstack] DESCRIPTION="LAMP stack" VCPROFILE[lampstack] OPRSVPROFILE=lampstack VCPROFILE[lampstack] NODEHOURCHARGERATE=6.6 NODESETUPCOST=66.6 VCPROFILE[lampstack] FLAGS=AUTOACCESS VCPROFILE[lampstack] QUERYDATA=minnodes=1, label=apache_lb,rsvprofile=apache_lb VCPROFILE[lampstack] QUERYDATA=minnodes=3, label=apache_pool,rsvprofile=apache_pool VCPROFILE[lampstack] QUERYDATA=minnodes=1, label=db_lb,rsvprofile=db_lb VCPROFILE[lampstack] QUERYDATA=minnodes=2, label=db_pool,rsvprofile=db_pool VCPROFILE[lampstack] QUERYDATA=minnodes=1, label=db_backup,rsvprofile=db_backup VCPROFILE[lampstack] QUERYDATA=minnodes=2, label=script_compute,rsvprofile=script_compute RSVPROFILE[rsvlampstack] FLAGS=EXCLUDEMYGROUP, SYSTEMJOB RSVPROFILE[rsvlampstack] TRIGGER=ATYPE=exec, Action="/opt/moab/tools/lamp/lampstack.py --etype=$ETYPE --vpcid=$VPCID --hostlist=$HOSTLIST",EType=start,sets= ^lampstack,requires=apache_lb.db_lb RSVPROFILE[rsvlampstack] TRIGGER=ATYPE=exec, Action="/opt/moab/tools/lamp/ lampstack_shepherd.py --vpcid=$VPCID", EType=start,MultiFire=TRUE,RearmTime=05:00, sets=!badlamp,requires=lampstack RSVPROFILE[rsvlampstack] TRIGGER=ATYPE=mail, Action="There was a problem with this vpc: $VPCID",EType=start,requires=badlamp,unsets=badlamp RSVPROFILE[rsvlampstack] TRIGGER=ATYPE=exec,Action= "mschedctl -d vpc:$VPCID",EType=start,requires= selfdestruct RSVPROFILE[apache_lb] TRIGGER=ATYPE=exec,Action= "/opt/moab/tools/lamp/apache_lb_ctl.py --etype=$ETYPE --vpcid=$VPCID --hostlist=$HOSTLIST --pool=$apache_pool",EType=start,sets=^apache_lb, requires=apache_pool.script_compute RSVPROFILE[apache_pool] TRIGGER=ATYPE=exec,Action= "/opt/moab/tools/lamp/apache_pool_ctl.py --etype=$ETYPE --vpcid=$VPCID --hostlist= $HOSTLIST",EType=start,sets=^apache_pool RSVPROFILE[script_compute] TRIGGER=ATYPE=exec, Action="/opt/moab/tools/lamp/script_compute.py --etype=$ETYPE --db_lb=$db_lb --vpcid=$VPCID --hostlist=$HOSTLIST",EType=start,requires=db_lb, sets=^script_compute RSVPROFILE[db_lb] TRIGGER=ATYPE=exec,Action= "/opt/moab/tools/lamp/db_lb_ctl.py --etype=$ETYPE --vpcid=$VPCID --hostlist=$HOSTLIST --pool=$db_pool --backup=$db_backup --active= $ACTIVE",EType=start,sets=^db_lb,requires= db_pool.db_backup RSVPROFILE[db_backup] TRIGGER=ATYPE=exec,Action= "/opt/moab/tools/lamp/db_backup_ctl.py --etype=$ETYPE --vpcid=$VPCID --hostlist=$HOSTLIST", EType=start,sets=^db_backup RSVPROFILE[db_pool] TRIGGER=ATYPE=exec,Action= "/opt/moab/tools/lamp/db_pool_zero.py --etype=$ETYPE --vpcid=$VPCID",EType=start, sets=db_pool_go RSVPROFILE[db_pool] TRIGGER=ATYPE=exec,Action= "/opt/moab/tools/lamp/db_pool_ctl.py --etype=$ETYPE --vpcid=$VPCID --hostlist= $HOSTLIST",EType=start,MultiFire=TRUE,sets= ^db_pool.!db_pool_retry,unsets= db_pool_go.!db_pool_go,requires=db_pool_go RSVPROFILE[db_pool] TRIGGER=ATYPE=exec,Action= "/opt/moab/tools/lamp/db_pool_retry.py - -etype=$ETYPE --vpcid=$VPCID",EType=start, MultiFire=TRUE,sets=^db_pool_go.!^selfdestruct, unsets=db_pool_retry.!db_pool_retry,requires= db_pool_retry
In the preceding example, a profile is configured that makes a request for a LAMP stack that is comprised of an Apache load balancer, a pool of three Apache worker nodes, two nodes used to offload script-based computing, a database load balancer, a pool of two database nodes, and a database backup node.
The example shows how to handle two failure conditions. First, a shepherd trigger - a trigger that check the condition of a running job - is assigned to watch over the health of the lampstack in 5-minute intervals. If this trigger fails (returns -1), Moab sends an email to the administrator indicating the VPCID of the culprit VPC.
The second form of failure is illustrated using the db_pool example. The db_pool_zero script runs at VPC startup, resets the counter associated with the given VPCID (in this case stored in an SQLite3 database), and sets a variable allowing the second db_pool trigger to fire. If this trigger is successful, normal workflow conditions exist, and the rest of the VPC LAMP stack is created. If it fails, it sets a flag that the db_pool_retry script responds to. The db_pool_retry script increments the counter associated with the current VPCID, and resets the trigger variables to a state that allows the db_pool_ctl trigger to refire. If the db_pool_retry script gets called more than a prescribed number of times, this trigger fails, setting a variable that destroys the VPC.
Administrators can specify the credentials that are allowed to create each VPC profile using the ACL parameter. The ACL parameter uses the Moab ACL syntax.
Example: A VPC that can only be created by members of the test group
VCPROFILE[test] DESCRIPTION="Testing Environment" VCPROFILE[test] OPRSVPROFILE=test VCPROFILE[test] ACL=GROUP:test
The preceding example specifies that only members of the test group may request the test profile.
When a VPC is requested, an ACL can be specified for the reservation associated with the VPC. This allows users to give others access to their VPCs. This also allows users to submit batch jobs into the VPC that they have created. This capability can be used to allow trusted users to create limited reservations on the system and submit batch jobs into them. It also allows automatic charging for these reservations. If the AUTOACCESS flag is specified on the VPC, the owner of that VPC is automatically added to the ACL of its reservation.
Example: A profile that allows the owner and administrators to submit batch jobs into it
VCPROFILE[private] DESCRIPTION="Private cluster access" VCPROFILE[private] REQSETATTR=nodefeature=dedicated,acl=group=administrators VCPROFILE[private] FLAGS=AUTOACCESS
In the preceding example, the specified profile creates a reservation on the cluster that only administrators and the owner of the VPC may access. Only nodes with the dedicated feature may be reserved in this way.
Example: A user creates a VPC and submits jobs into it
To submit a job, the user specifies the VPC's reservation on the command line using -l advres.
$ mshow -a -i -x -o --flags=tid,summary,future -p private -w minnodes=1,duration=10000 Partition Tasks Nodes Duration StartOffset StartDate --------- ----- ----- ------------ ------------ -------------- ALL 2 1 2:46:40 00:00:00 16:19:14_03/10 TID=13 ALL 2 1 2:46:40 2:46:40 19:05:54_03/10 TID=14 ALL 2 1 2:46:40 5:33:20 21:52:34_03/10 TID=15 $ mschedctl -c vpc -a resources=13 vpc.5 $ mschedctl -l vpc:vpc.5 VPC vpc.5 (active) --------- User: user1 Owner: user:user1 Size: 2 Task Resources: PROCS: [ALL] Available Time: 16:19:14_03/10 (-00:01:56) Available Duration: 2:46:40 Provision Start Time: 16:19:22_03/10 (-00:01:48) Cleanup End Time: 19:05:54_03/10 (2:44:44) PurgeTime: 19:05:54_03/11 Variables: VPCHOSTLIST=node009;VPCID=vpc.5; Rsv Group: PRIVATE.5 (PRIVATE.5) VPC Profile: PRIVATE Total Charge: 0.00 $ echo sleep 30 | msub -l advres=PRIVATE.5,walltime=30,nodes=1 moab.2
VPC charging is configured with the NODECHARGERATE and NODESETUPCOST parameters.
Example: Charging
VCPROFILE[dbclient] NODEHOURCHARGERATE=25.0 NODESETUPCOST=50.0
Copyright © 2012 Adaptive Computing Enterprises, Inc.®
