(Click to open topic with navigation)
If you want to debit funds in Moab Accounting Manager but need higher throughput than is achieved when using liens and quotes, you can use the more relaxed fast-allocation accounting mode. When using this mode without any auxilliary setup, there is no enforcement of allocation limits. Quotes and liens are disabled in this mode. It is possible to implement a less-synchronized method of allocation enforcement by using custom NAMI scripts for the AMCFG[]CREATEURL, STARTURL, and ENDURL parameters in Moab to check for account disablement when submitting and starting jobs. The customization needed to do this would be the responsibility of the customer and is beyond the scope of this document.
Set the accounting.mode parameter to fast-allocation in both the goldd.conf and gold.conf files.
Example 22-12: Setting the accounting mode to fast-allocation
The accounting.mode parameter must be set in the server and client configuration files (goldd.conf and gold.conf). After editing the goldd.conf file, you will need to restart goldd.
$ vi /opt/mam/etc/goldd.conf
accounting.mode = fast-allocation
$ vi /opt/mam/etc/gold.conf
accounting.mode = fast-allocation
$ goldd -r
Usage record customization (optional)
It may be desirable to customize the usage record to display usage properties unique to your site. See the section on Customizing the usage record object for examples.
Decide on a currency and set the currency precision
Since we will be calculating charges, we will need to decide on a currency unit and set the currency precision. For this example we will define the currency to be in dollars (and cents). Deposits into funds will be made in this currency. Resource charges will be calculated from charge rates based on this currency. Since dollars and cents are represented as a floating point number with two decimal places we must specify a currency precision of two.
Example 22-13: Setting the currency precision to two
The currency precision value must be set in the server and client configuration files (goldd.conf and gold.conf). It must also be set in the GUI configuration file (goldg.conf) if you will be using the web GUI.
$ vi /var/gold/etc/goldd.conf
currency.precision = 2
$ vi /var/gold/etc/gold.conf
currency.precision = 2
Since we are charging, we must establish the charge rates for the usage. In our example, we will establish a charging scheme that charges 1 dollar for each processor hour utilized by the job.
Since we want to charge 1 dollar per hour of usage per processor and because time-based charge rates are multiplied by the duration in seconds, we need create a charge rate for processors that charges 1/3600th of a dollar per second. See the Managing charge rates chapter for more detailed information on setting up charge rates.
Example 22-14: Define a charge rate for processors
$ gmkrate -n Processors -z 1/h -d "1 dollar per processor-hour"
Successfully created 1 charge rate
$ glsrate
Name Value Amount Description
---------- ----- ------ ---------------------------
Processors 1/h 1 dollar per processor-hour
Next we must decide to which entities we want to entitle our allocations. In this example, we will distribute the funds among different accounts. Each account will be assigned a set of user members that can charge to that account. Moab Accounting Manager can be customized to associate funds with any number of arbitrary entities such as Users, Groups, Accounts, Organizations, Classes, Machines, and the like. We will start by defining some accounts and the associated user members of the accounts. We will also associate each account with an organization so that usage reports can be generated for the organization level as well as the account and user level.
We will create accounts for biology, chemistry and film and assign them some users. The biology and chemistry account will be associated with the sciences organization while the film account will be associated with the arts organization. See Managing accounts for more information on setting up accounts.
Example 22-15: Define the biology, chemistry, and film accounts
$ gmkaccount -a biology -o sciences -u amy,bob -d "Biology Department"
Successfully created 1 account
$ gmkaccount -a chemistry -o sciences -u amy,dave -d "Chemistry Department"
Successfully created 1 account
$ gmkaccount -a film -o arts -u bob,dave -d "Film Department"
Successfully created 1 account
$ glsaccount
Name Active Users Organization Description
--------- ------ -------- ------------ --------------------
biology True amy,bob sciences Biology Department
chemistry True amy,dave sciences Chemistry Department
film True bob,dave arts Film Department
The next task will be to create the funds which will hold the allocated credits. A fund is much like a numbered bank account, where credits can be deposited and are defined by constraints that distinguish who or what can use the contained credits and for what purposes. In this example, we will create a fund for each of the three accounts. Had we enabled fund auto-generation or used the --create-fund=True option with the gmkaccount command, a fund would have been created automatically with the creation of each account. See Managing funds for more detailed information on setting up funds.
Note that in most cases, referenced objects will be auto-generated, such as the users and organizations that were auto-generated as they were added to the accounts. Undefined accounts, would have likewise been auto-generated as they were newly associated with funds, but had we taken that ordering, we would still have needed to go back and associate the appropriate users, organization, and the like, to these accounts. We could have also created the users explicitly and provided additional detail about each user, before adding them to the accounts. Of course, even though they were auto-generated, we can still go back and add detailed information to each user as desired. See Managing users for more detailed information on setting up users.
If you want to establish a periodic allocation cycle with predesignated allocations being deposited on a regular schedule, you can associate a default deposit amount with the fund. See Managing allocations for more information on periodic allocations.
Example 22-16: Create a fund for each of the three accounts
$ gmkfund -a biology -n "biology" --default-deposit 5000
Successfully created 1 fund with id 1 and 1 constraint
$ gmkfund -a chemistry -n "chemistry" --default-deposit 3000
Successfully created 1 fund with id 2 and 1 constraint
$ gmkfund -a film -n "film"
Successfully created 1 fund with id 3 and 1 constraint
$ glsfund
Id Name Constraints Amount Allocated DefaultDeposit Description
-- --------- ----------------- ------ --------- -------------- -----------
1 biology Account=biology 0.00 0.00 5000.00
2 chemistry Account=chemistry 0.00 0.00 3000.00
3 film Account=film 0.00 0.00
Now we need to allocate credits to these funds by making deposits to them. An allocation has a start and end time associated with it declaring the time frame in which it can be used (defaulting to a start time of the present and an end time of infinity). It can also have a credit limit which defines the extent to which the allocation is allowed to go negative. Allocations can be reset on a periodic basis or future allocations with different time frames can be pre-created within a fund to establish an allocation cycle and set expectations for credit expenditure. See Managing allocations and Making deposits for additional information.
In this example, we will allocate 5000 and 3000 dollars to the biology and chemistry accounts respectively. The film account will be given a credit limit of 2000 dollars which allows them to charge up to 2000 dollars before settling their fund. When making a deposit we must specify the fund we are depositing into unless the fund can be unambiguously determined by its constraint references (i.e. there is only a single fund associated with the account biology). In the next example, we will utilize the fund's default deposit amount in the first deposit, specify the amount explicitly in the second deposit and establish a credit allocation in the third deposit.
Example 22-17: Making deposits
$ gdeposit -a biology
Successfully deposited 5000.00 credits into fund 1
Successfully created 1 allocation
$ gdeposit -z 3000 -a chemistry
Successfully deposited 3000.00 credits into fund 2
Successfully created 1 allocation
$ gdeposit -L 2000 -a film
Successfully deposited 0.00 credits into fund 3
Successfully created 1 allocation
Let's examine the allocations we just created.
$ glsalloc
Id Fund StartTime EndTime InitialDeposit Allocated CreditLimit Remaining PercentUsed
-- ---- ------------------- -------- -------------- --------- ----------- --------- -----------
1 1 2014-05-06 10:06:56 Infinity 5000.00 5000.00 0.00 5000.00 0.00
2 2 2014-05-06 10:06:56 Infinity 3000.00 3000.00 0.00 3000.00 0.00
3 3 2014-05-06 10:06:57 Infinity 0.00 0.00 2000.00 0.00 0.00
0.00 0.00
$ glsfund
Id Name Constraints Amount Allocated DefaultDeposit Description
-- --------- ----------------- ------- --------- -------------- -----------
1 biology Account=biology 5000.00 5000.00 5000.00
2 chemistry Account=chemistry 3000.00 3000.00 3000.00
3 film Account=film 0.00 0.00
We can verify the resulting balance (see Querying the balance).
Example 22-18: Amy's balance
$ gbalance -u amy
Id Name Available Allocated PercentUsed
-- --------- --------- --------- -----------
1 biology 5000.00 5000.00 0.00
2 chemistry 3000.00 3000.00 0.00
Integrate Moab Accounting Manager with Moab Workload Manager
If you have not done so already, you will need to configure the Moab suite to interact with Moab Accounting Manager (see Integrate Moab Accounting Manager with Moab Workload Manager).
When a job is submitted, it is useful to check that the user's fund has enough credits for the requested usage. This will be verified when the job starts, but by that point the job may have waited some time in the queue only to find out it never could have run in the first place. The usage quote step (see Obtaining usage quotes) can fill this function. Additionally, the quote can be used to determine the cheapest place to run, and to guarantee the current rates will be used when the usage is charged.
Example 22-19: Let's see how much it will cost to use the resources
$ gquote -u amy -a chemistry -g faculty -c batch -m colony -N 1 -P 16 -t 3600
Successfully quoted 16.00 credits
Now let's submit a job and examine the effects on the accounting system.
Example 22-20: Submit a job
$ echo sleep 300 | msub -A chemistry -l procs=12,walltime=600
After a job completes, a charge is issued against the appropriate allocations based on the resources and wallclock time actually used by the job. An allocation is debited and the usage record is modified with the charge and usage information.
Example 22-21: Examine the effect of a completed job on the accounting system
Your balance will have gone down by the amount of the charge.
$ gbalance -u amy -a chemistry --show=Id,Name,Balance,CreditLimit,Available
Id Name Balance CreditLimit Available
-- --------- ------- ----------- ---------
2 chemistry 2999.00 0.00 2999.00
The usage record for the job was updated as a side-effect of the charge (see Querying usage records).
$ glsusage -u amy -a chemistry
Id Type Instance Charge Stage User Group Account Organization Class QualityOfService Machine Nodes Processors CPUTime Memory Disk Duration StartTime EndTime Description
-- ---- -------- ------ ------ ---- ------- --------- ------------ ----- ---------------- ------- ----- ---------- ------- ------ ---- -------- ------------------- ------------------- -----------
1 Job Moab.1 1.00 Charge amy faculty chemistry sciences batch colony 1 12 300 2014-05-06 10:11:49 2014-05-06 10:16:49
Now, since this was a test job, we will refund the user's fund (see Issuing usage refunds).
Example 22-22: Let's issue a refund for our job
$ grefund -J Moab.1
Successfully refunded 1.00 credits to usage record 1 for instance Moab.1
Our balance is back as it was before the job ran.
$ gbalance -u amy -a chemistry --show=Id,Name,Amount,Reserved,Balance,CreditLimit,Available
Id Name Amount Reserved Balance CreditLimit Available
-- --------- ------- -------- ------- ----------- ---------
2 chemistry 3000.00 0.00 3000.00 0.00 3000.00
The allocation, of course, is likewise restored.
$ glsalloc -u amy -a chemistry
Id Fund StartTime EndTime InitialDeposit Allocated CreditLimit Remaining PercentUsed
-- ---- ------------------- -------- -------------- --------- ----------- --------- -----------
2 2 2014-05-06 10:06:56 Infinity 3000.00 3000.00 0.00 3000.00 0.00
Notice that the usage charge is now zero because the job has been fully refunded.
$ glsusage
Id Type Instance Charge Stage User Group Account Organization Class QualityOfService Machine Nodes Processors CPUTime Memory Disk Duration StartTime EndTime Description
-- ---- -------- ------ ------ ---- ------- --------- ------------ ----- ---------------- ------- ----- ---------- ------- ------ ---- -------- ------------------- ------------------- -----------
1 Job Moab.1 0.00 Charge amy faculty chemistry sciences batch colony 1 12 300 2014-05-06 10:11:49 2014-05-06 10:16:49
Examine fund statement
Finally, you can examine the fund statement for our activities (see Obtaining a fund statement).
Example 22-23: We can request an itemized fund statement over all time for user amy and the chemistry account (fund 2)
$ gstatement -u amy -a chemistry
################################################################################
#
# Includes fund 2 (chemistry for amy)
# Generated on Tue May 5 10:18:22 2014.
# Reporting fund activity from -Infinity to Now.
#################################################################################
Beginning Balance: 0.00
------------------ --------------------
Total Credits: 3001.00
Total Debits: -1.00
------------------ --------------------
Ending Balance: 3000.00
############################### Credit Detail ##################################
Object Action Instance Amount Time
----------- ------- -------- ------- -------------------
Fund Deposit 3000.00 2014-05-06 10:06:56
UsageRecord Refund Moab.1 1.00 2014-05-06 10:17:20
############################### Debit Detail ###################################
Object Action Instance Account User Machine Amount Time
----------- ------ -------- --------- ---- ------- ------ -------------------
UsageRecord Charge moab.1 chemistry amy colony -1.00 2014-05-06 10:16:49
############################### End of Report ##################################