1.15 Charging a Workflow

There are two new files in the directory containing the NAMI scripts. The config.pl is used to abstract away all Gold-specific charge rates and variables from the scripts and centralize them in one file for easy modification. The NamiUtils.pm creates a shared object in the NAMI scripts to distribute the data in the config.pl as well as abstract away duplicated code in multiple NAMI scripts. You can also use initial_setup.pl to create a Moab home directory with the NAMI Scripts in a workload-driven environment.

1.15.1 Modifying Moab.cfg

In charging for workflows, Moab.cfg doesn’t use Create or Modify NAMI URL. Instead, it uses ReserveURL to reserve funds and charge for setup costs, ChargeURL to reserve funds and charge for reoccurring costs, and uses DeleteURL to unreserve the funds. The ReserveFailureAction should be set in case the customer doesn't have enough funds to proceed. A sample moab.cfg might look like this in the NAMI section:

AMCFG[nami] TYPE=NATIVE
AMCFG[nami] QuoteURL=exec:///$HOME/tools/billing/bank.quote.db.pl
AMCFG[nami] ReserveURL=exec:///$HOME/tools/billing/bank.reserve.db.pl
AMCFG[nami] ChargeURL=exec:///$HOME/tools/billing/bank.charge.db.pl
AMCFG[nami] DeleteURL=exec:///$HOME/tools/billing/bank.delete.db.pl
AMCFG[nami] FLUSHINTERVAL=hour
AMCFG[nami] ReserveFailureAction=HOLD

1.15.2 Defining the config.pl to match the moab.cfg

First, it is important to make sure the workflow set in moab.cfg matches the exact names in the $templatesToCharge hash.

1.15.2.1 moab.cfg

JOBCFG[newvm]            FLAGS=VMTRACKING    SELECT=true
JOBCFG[newvm]            TEMPLATEDEPEND=AFTEROK:newvmprovision
JOBCFG[newvm]            DESTROYTEMPLATE=destroyvm
JOBCFG[newvmprovision]   GENERICSYSJOB=TRUE
JOBCFG[newvmprovision]   INHERITRES=TRUE
JOBCFG[newvmprovision]   TRIGGER=AType=exec,Action="$TOOLSDIR/newvmprovision.pl",EType=start,Timeout=2:00,Flags=objectxmlstdin,Sets=^VMID	
JOBCFG[newstorage]     SELECT=true
JOBCFG[newstorage]     TEMPLATEDEPEND=AFTEROK:newstorage2
JOBCFG[newstorage]     FLAGS=GRESONLY,NORMSTART
JOBCFG[newstorage]     DESTROYTEMPLATE=destroystorage
JOBCFG[newstorage2]    GENERICSYSJOB=TRUE
JOBCFG[newstorage2]    INHERITRES=TRUE
JOBCFG[newstorage2]    FLAGS=GRESONLY,NORMSTART
JOBCFG[newstorage2]    TEMPLATEDEPEND=AFTEROK:newstorage1
JOBCFG[newstorage2]    TRIGGER=AType=exec,Action="$TOOLSDIR/newstorageprovision.pl $USER",EType=start,Flags=objectxmlstdin,Timeout=2:00,Sets=^SM.^SH
JOBCFG[newstorage1]    GENERICSYSJOB=TRUE
JOBCFG[newstorage1]    INHERITRES=TRUE
JOBCFG[newstorage1]    FLAGS=GRESONLY,NORMSTART
JOBCFG[newstorage1]    TRIGGER=AType=exec,Action="$TOOLSDIR/twentyseconds.pl",EType=start,Flags=objectxmlstdin,Timeout=2:00
1.15.2.1.1 matching config.pl
	our $templatesToCharge = {
          "newvm"=>[{"newvmprovision" =>30}],
          "newstorage"=>[{"newstorage2" =>50}, {"newstorage1" =>40}],
        };

1.15.3 Defining Gold Charge Rates to match config

Second, make sure that a VBF ChargeRate of 1 in Gold exists to match the exact names in the $templatesToCharge hash for the provisioning jobs, and that the $resourcesToCharge hash maps NAMI ids to the correct charge rates in Gold. There must be at least one active machine that matches the $goldMachineName.

1.15.3.1 config.pl

	our $templatesToCharge = {
          "newvm"=>[{"newvmprovision" =>30}],
          "newstorage"=>[{"newstorage2" =>50}, {"newstorage1" =>40}],
        };
	our $resourceToChargeRate = {
  "processors" => "cpuPerSecond",
  "memory" => "megabytePerSecond"
};

1.15.3.2 Matching Gold output from "goldsh ChargeRate Query"

Type Name              Instance Rate     Description
---- ----------------- -------- -------- -----------
VBR  megabytePerSecond          .0000001
VBR  cpuPerSecond               1
VBF  newstorage1                1
VBF  newstorage2                1
VBF  newvmprovision             1

1.15.3.3 config.pl

our $goldMachineName = "cloud";

1.15.3.4 Matching Gold output from "glsmachine"

Name  Active Architecture OperatingSystem Organization Description
----- ------ ------------ --------------- ------------ -------------
cloud True                                             Linux Cluster

1.15.4 Running the Workflow

Before you run the workflow, make sure that Moab accounts are enabled. When you type the msub command, make sure the name of your account matches exactly the name of your Gold account.

1.15.4.1 Example Moab Command

msub -A integration -l nodes=1:ppn=2,mem=2048,walltime=INFINITY,template=newvm,os=windowsXP /dev/null

1.15.4.2 Matching Gold output from "glsaccount"

Id Name        Amount        Projects    Users Machines Description
-- ----------- ------------- ----------- ----- -------- --------------
1  integration 3599999989560 integration ANY   ANY      Auto-generated

1.15.5 How to Simulate Moab to adapt your scripts

You will notice that the NAMI scripts generate both Gold and Moab logs. The Moab log shows exactly what Moab is sending the NAMI scripts. The Moab log or other test-case logs can be used to simulate Moab behavior in a Perl script. This is done by using logToTestCase.pl (the only caveat is the script needs to be run in the testScripts folder or be correctly linked to the test perl objects in the testScripts folder).

This will give you a Perl script that will simulate Moab commands to the NAMI scripts so that you can build tests and more easily modify the scripts with confidence that it will work when Moab is run.