5.206 Configuring Moab Web Services

This section describes the location of the MWS configuration files. It also shows some examples of how to configure logging.

To see a full reference to all configuration and logging parameters available in MWS, see Configuration.

This topic contains these sections:

• 5.206.1 Home Directory
• 5.206.2 Configuration Files
• 5.206.3 Logging Configuration Using /opt/mws/etc/mws-config.groovy
• 5.206.4 LDAP Configuration Using /opt/mws/etc/mws-config.groovy
• 5.206.5 PAM (Pluggable Authentication Module) Configuration Using /opt/mws/etc/mws-config.groovy
• 5.206.6 OAuth Configuration Using /opt/mws/etc/mws-config.groovy

MWS does not support LDAP and PAM authentication at the same time.

5.206.1 Home Directory

The MWS home directory contains configuration files, log files, and files that serve features of MWS such as hooks and plugins. You should set the location of the MWS home directory using the MWS_HOME property. If you do not set MWS_HOME as a Java property or as an environment variable, then MWS will use /opt/mws as the default MWS_HOME.

For documentation clarity, the default "/opt/mws/" is used in the file names for the MWS_HOME property.

5.206.2 Configuration Files

The primary configuration file is /opt/mws/etc/mws-config.groovy. If this file is missing or contains errors, MWS will not start.

Configuration files can also be placed in the /opt/mws/etc/mws.d directory. Any configuration files here get merged with /opt/mws/etc/mws-config.groovy. In case of conflict, the configuration in /opt/mws/etc/mws.d takes precedence.

If /opt/mws/etc/log4j.properties exists, MWS will load it as well.

5.206.3 Logging Configuration Using /opt/mws/etc/mws-config.groovy

Shown below is an example that logs all error messages and fatal messages to /opt/mws/log/mws.log (For information about the format of the MWS logs, see "Standard Log Format" in the Moab Workload Manager Administrator Guide.). It also logs all stack traces to /opt/mws/log/stacktrace.log. Note that this example is not configured to log events; for details on logging events, see 5.206.3.A Configuring an Event Log.

Minimal logging configuration
-----------------------------

log4j = {
   appenders {
      rollingFile name: 'stacktrace',
         file: '/opt/mws/log/stacktrace.log',
         maxFileSize: '1GB'
      rollingFile name: 'rootLog',
         file: '/opt/mws/log/mws.log',
         threshold: org.apache.log4j.Level.ERROR,
         maxFileSize: '1GB'
   }
   root {
      debug 'rootLog'
   }
}

Alternatively, you may configure a console appender instead of a rolling file, as shown below.

Console logging configuration
-----------------------------

log4j = {
   appenders {
      rollingFile name: 'stacktrace',
         file: '/opt/mws/log/stacktrace.log',
         maxFileSize: '1GB'
      console name: 'consoleLog',
         threshold: org.apache.log4j.Level.ERROR
   }
   root {
      debug 'consoleLog'
   }
}

You may configure logging by using either /opt/mws/etc/mws-config.groovy or /opt/mws/etc/log4j.properties.

If you do not define any log4j configuration, MWS will write its log files to java.io.tmpdir. For Tomcat, java.io.tmpdir is generally set to $CATALINA_BASE/temp or CATALINA_TMPDIR.

5.206.3.A Configuring an Event Log

Logging events to a flat file requires that you make a few changes to the configuration in the log4j section of the /opt/mws/etc/mws-config.groovy file so that events will be logged to the events.log file, and all other MWS logging information will be sent to the mws.log file.

Causing events.log to roll based on a time window

You can specify how often the events.log file rolls. The following example illustrates the configuration changes you will need make to /opt/mws/etc/mws-config.groovy to cause the events.log file to roll based on a time window. Note the following three examples:

• In this example, /opt/mws/etc/mws-config.groovy is configured so that events.log rolls daily at midnight.

Daily rolling events.log configuration in mws-config.groovy
------------------------------------

log4j = {
  def eventAppender = new org.apache.log4j.rolling.RollingFileAppender(name: 'events', layout: pattern(conversionPattern: "%m%n"))
  def rollingPolicy = new org.apache.log4j.rolling.TimeBasedRollingPolicy(fileNamePattern: '/tmp/events.%d{yyyy-MM-dd}', activeFileName: '/tmp/events.log')
  rollingPolicy.activateOptions()
  eventAppender.setRollingPolicy(rollingPolicy)

  appenders {
    appender eventAppender

    rollingFile name: 'rootLog',
      file: '/tmp/mws.log',
      maxFileSize: '1GB'
  }

  root {
    warn 'rootLog'
  }

  trace additivity:false, events:'com.ace.mws.events.EventFlatFileWriter'
}

Note the RollingFileAppender and the TimeBasedRollingPolicy lines. These lines configure MWS to write the event log to the events.log file. Rolled log files will have a date appended to their name in this format: "yyyy-MM-dd" (for example, events.log.2012-02-28).

• If you want the event log file to roll at the beginning of each month, change the fileNamePattern TimeBasedRollingPolicy date format to yyyy-MM. For example:

Monthly event logs
------------------------------------

def rollingPolicy = new org.apache.log4j.rolling.TimeBasedRollingPolicy(fileNamePattern: '/tmp/events.%d{yyyy-MM}', activeFileName: '/tmp/events.log')

• If you want the event log file to roll at the beginning of each hour, change the date format to yyyy-MM-dd_HH:00. For example:

Hourly event logs
------------------------------------

def rollingPolicy = new org.apache.log4j.rolling.TimeBasedRollingPolicy(fileNamePattern: '/tmp/events.%d{yyyy-MM-dd_HH:00}', activeFileName: '/tmp/events.log')

Configuring events.log to roll based on a file size threshold

You can also configure the events.log file to roll when the log size exceeds a specified threshold. The following example illustrates the configuration changes you will need to make to /opt/mws/etc/mws-config.groovy to cause the events.log file to roll on a size threshold. (In this example, /opt/mws/etc/mws-config.groovy is configured so that events.log rolls when its size exceeds 50 MB.)

mws-config.groovy configuration that rolls events.log based on file size
------------------------------------

log4j = {
  appenders {
    rollingFile name: 'events',
      file: '/tmp/events.log',
      maxFileSize: '50MB',
      maxBackupIndex:10

    rollingFile name: 'rootLog',
      file: '/tmp/mws.log',
      maxFileSize: '1GB'
  }

  root {
    warn 'rootLog'
  }

  trace additivity:false, events:'com.ace.mws.events.EventFlatFileWriter'
}

Note that maxFileSize is set to "50MB." This means that when the events.log file exceeds 50 MB, it will roll.

The name for the rolled log will be "events.log.1". When the new events.log file exceeds 50 MB, it will roll and be named "events.log.1", while the old "events.log.1" file will be renamed "events.log.2". This process will continue until the optional maxBackupIndex value is met. In the example above, maxBackIndex is set to 10. This means that MWS will delete all but the ten most recent events.log files. Using this feature helps prevent hard drives from filling up.

Additivity

The additivity attribute of the EventFlatFileWriter logger can be either true or false. If you specify true, events will be logged to the events.log file and the mws.log file. If you specify false, events will be logged to the events.log file only. (All other MWS logging information will be logged to the mws.log file, as configured by the rootLog appender.)

To log events to the mws.log file in addition to the events.log file, make the additivity:true configuration. For example:

Logging events to both events.log and mws.log
------------------------------------

trace additivity:true, events:'com.ace.mws.events.EventFlatFileWriter'

For more configuration options, see Apache Extras Companion for log4j.

Deleting old events

If your MongoDB server is version 2.2 or later, MongoDB will automatically delete events older than 30 days (by default). For more information, including how to change this default, see mws.events.expireAfterSeconds in Configuration.

If your MongoDB server is older than version 2.2, MongoDB will store event data indefinitely. However, if disk space is limited, you may want to regularly delete old, unneeded events from MongoDB. This section contains some examples of how you can do this.

Let's say that you want to delete events that are older than 90 days. (There are 86,400,000 milliseconds in a day, so in this example, 90*86400000 corresponds to 90 days in milliseconds.):

• You could run this script:

Delete events older than 90 days
------------------------------------

$ mongo
MongoDB shell version: 2.4.8
connecting to: test
> use mws
> db.event.remove({eventTime:{$lt:new Date(new Date().getTime()-90*86400000)}})
> exit

• To create a script to perform this task:

deleteOldEvents.sh
------------------------------------

#!/bin/bash
printf 'use mws_dev\ndb.event.remove({eventTime:{$lt:new Date(new Date().getTime()-90*86400000)}})\nexit' | mongo

• Now say that you want to set up a cron job so that old events are automatically deleted on a certain day of the week (for example, every Sunday at 2:00 a.m.). You would add an entry like this:

cron table entry to delete old events
------------------------------------

00 02 * * 0 /root/deleteOldEvents.sh

5.206.3.B Configuring an Audit Trail Log

Audit logging enables you to track changes to Permissions, Roles, and Principals.

mws-config.groovy configuration that enables audit logging
------------------------------------
    // Configure an appender for the audit log.
    def auditAppender = new org.apache.log4j.rolling.RollingFileAppender(
            name: 'audit',
            layout: new com.ace.mws.logging.ACPatternLayout("%j\t\t\t%c{1}\t\t\t%m%n"))
    def auditRollingPolicy = new org.apache.log4j.rolling.TimeBasedRollingPolicy(
            fileNamePattern: '/opt/mws/log/audit.%d{yyyy-MM-dd}',
            activeFileName: '/opt/mws/log/audit.log')
    auditRollingPolicy.activateOptions()
    auditAppender.setRollingPolicy(auditRollingPolicy)

    appenders {
        rollingFile name: 'stacktrace',
                file: '/opt/mws/log/stacktrace.log',
                maxFileSize: '100MB'
        rollingFile name: 'rootLog',
                file: '/opt/mws/log/mws.log',
                maxFileSize: '100MB', //The maximum file size for a single log file
                maxBackupIndex: 10, //Retain only the 10 most recent log files, delete older logs to save space
                layout: new com.ace.mws.logging.ACPatternLayout(), //Configures the output format of each log entry
                threshold: org.apache.log4j.Level.ERROR //Ignore any logging entries less verbose than this threshold

        appender eventAppender
        appender auditAppender
    }

You can customize audit logging in ways you can customize event logging. For example, you can specify how often the audit.log file rolls. You can also configure the audit.log file to roll when the log size exceeds a specified threshold.

Follow the same steps indicated in the previous section on Configuring an Event Log for instruction on customizing audit logging; customization processes are the same for audit logging as for events logging.

audit.log file format

The default location to which the audit trail log is written is /opt/mws/log/audit.log. The log format is yyyy-MM-dd HH:mm:ss resource username action data. The following table offers a description for attributes included in the log format:

Sample audit.log format:

Audit trail log format
------------------------------------

2013-10-30 14:39:32,120 TENANT 'admin' updated resource named 'Engineering2' with values: 
     "name": "Engineering3",
     "attachedPrincipals": [{"name": "Engineering"}]

5.206.4 LDAP Configuration Using /opt/mws/etc/mws-config.groovy

The LDAP configuration provided below is for MWS to authenticate against a single LDAP server. If you wish to use LDAP to authenticate multiple servers, you must create and use a custom PAM module.

5.206.4.A Using a Supported LDAP Directory Type

To configure an MWS connection to an LDAP server, add the following parameters to /opt/mws/etc/mws-config.groovy:

Throughout the following examples in this topic, you will see dc=acme,dc=com. "acme" is only used as an example to illustrate what you would use as your own domain controller if your domain name was "acme.com." You should replace any references to "acme" with your own organization's domain name.

Here is a sample configuration for OpenLDAP.

If you followed the Adaptive Computing tutorial [link]"Setting up OpenLDAP on CentOS 6" your ldap.directory.type should be set to "OpenLDAP Using InetOrgPerson Schema".

Sample OpenLDAP configuration
-----------------------------

ldap.server  = "192.168.0.5"
ldap.port = 389
ldap.baseDNs = ["dc=acme,dc=com"]
ldap.bindUser = "cn=Manager,dc=acme,dc=com"
ldap.password = "*****"
ldap.directory.type = "OpenLDAP Using InetOrgPerson Schema"

Here is a sample configuration for Microsoft Active Directory.

Sample Active Directory configuration
-------------------------------------

ldap.server  = "192.168.0.5"
ldap.port = 389
ldap.baseDNs = ["CN=Users,DC=acme,DC=com","OU=Europe,DC=acme,DC=com"]
ldap.bindUser = "cn=Administrator,cn=Users,DC=acme,DC=com"
ldap.password = "*****"
ldap.directory.type = "Microsoft Active Directory"

To see how to configure a secure connection to the LDAP server, see Securing the LDAP Connection.

5.206.4.B Using an Unsupported LDAP Directory Type

If you are not using one of the supported directory types, you can explicitly configure MWS to work with your LDAP schema by using the following parameters:

For example:

Advanced Active Directory configuration
-----------------------------

ldap.server  = "myldaphostname"
ldap.port = 389
ldap.baseDNs = ["CN=Users,DC=acme,DC=com","OU=Europe,DC=acme,DC=com"]
ldap.bindUser = "cn=Administrator,cn=Users,DC=acme,DC=com"
ldap.password = "*****"
ldap.user.objectClass = "person"
ldap.group.objectClass = "group"
ldap.ou.objectClass = "organizationalUnit"
ldap.user.membership.attribute = "memberof"
ldap.group.membership.attribute = "member"
ldap.user.name.attribute = "sAMAccountName"

Here is a similar example for OpenLDAP. Note there is no user membership attribute in the OpenLDAP InetOrgPerson schema and thus ldap.user.membership.attribute is set to null. This is allowable because the ldap.group.membership.attribute is set.

Advanced OpenLDAP configuration
-----------------------------

ldap.server  = "myldaphostname"
ldap.port = 389
ldap.baseDNs = ["dc=acme,dc=com"]
ldap.bindUser = "cn=Manager,dc=acme,dc=com"
ldap.password = "*****"
ldap.user.objectClass = "inetOrgPerson"
ldap.group.objectClass = "groupOfNames"
ldap.ou.objectClass = "organizationalUnit"
ldap.user.membership.attribute = null
ldap.group.membership.attribute = "memberUid"
ldap.user.name.attribute = "uid"

5.206.4.C Overriding Attributes in a Supported LDAP Directory Type

You can also override attributes in supported directory types. For example, say you are using OpenLDAP with an NIS Schema. The group objectClass for NIS defaults to "groupOfNames," but you want to use "groupOfUniqueNames" instead while retaining all other defaults for NIS. You can do this by setting ldap.directory.type to "OpenLDAP Using NIS Schema" and overriding the ldap.group.objectClass attribute as follows:

Advanced OpenLDAP configuration
-----------------------------

ldap.directory.type = "OpenLDAP Using NIS Schema"
ldap.group.objectClass = "groupOfUniqueNames"

The user class in your LDAP schema must have an attribute that uniquely identifies a user (for example: "uid" or "sAMAccountName").

5.206.5 PAM (Pluggable Authentication Module) Configuration Using /opt/mws/etc/mws-config.groovy

“PAM provides a way to develop programs that are independent of authentication scheme. These programs need ‘authentication modules’ to be attached to them at run-time in order to work. Which authentication module is to be attached is dependent upon the local system setup and is at the discretion of the local system administrator.”, The Linux Kernel Archives, accessed October 26, 2016, https://www.kernel.org/pub/linux/libs/pam/whatispam.html.

5.206.5.A Requirements for PAM

To use PAM with MWS, the following are required:

• The PAM package must be installed. For example:

yum install pam

• The /etc/pam.d directory must contain at least one PAM configuration file. For example, here is the login configuration file from SLES 12:

#%PAM-1.0
auth	 requisite	pam_nologin.so
auth	 [user_unknown=ignore success=ok ignore=ignore auth_err=die default=bad]	pam_securetty.so
auth	 include	common-auth
account  include 	common-account
password include	common-password
session  required	pam_loginuid.so
session	 include	common-session
#session  optional       pam_lastlog.so nowtmp showfailed
session  optional       pam_mail.so standard

5.206.5.B Configuring MWS to Use PAM

To configure an MWS connection to PAM, add the following parameter to /opt/mws/etc/mws-config.groovy:

For example:

pam.configuration.service = "login"

You can configure only one authentication method in /opt/mws/etc/mws-config.groovy: LDAP or PAM, but not both. If you have configured both LDAP and PAM, MWS defaults to using LDAP.

If you need multiple authentication methods, you must add them to your local PAM configuration. See your distribution documentation for details.

If you configure MWS to authenticate via PAM using local files or NIS, you need to run Tomcat as root. This configuration is highly discouraged and is not supported by Adaptive Computing. The recommended approach is to configure PAM and NSS to authenticate against LDAP.

For more information about PAM, please see the following SUSE and RedHat documentation.

5.206.6 OAuth Configuration Using /opt/mws/etc/mws-config.groovy

OAuth is a security framework designed to simplify authentication in web technologies. In the case of MWS, OAuth allows trusted client applications to securely delegate authentication to MWS. Once MWS has authenticated a user by verifying the username and password in LDAP, PAM, or NIS, MWS returns an access token to the client. The client then presents this access token to MWS to access resources. OAuth is very flexible and allows MWS to work in many different scenarios by use of grant types. For more information on OAuth and grant types, please see the following OAuth documentation.

5.206.6.A Example Using 'password' Grant Type

Terminology

Resource Owner: The person accessing and manipulating data. For MWS, this would be the person who logs into the client (the user).

Service Provider: The site or service where protected resources live. This can be (but is not necessarily) also the identify provider, where usernames and passwords are stored. This is the MWS service itself.

Client: The application that wants to access a resource. For MWS this is the user interface, potentially including APIs and command-line tools.

Protected Resource: The data for which protection is desired. For MWS this would be Moab itself, and interaction with Moab.

Access Token: Instead of user credentials, OAuth uses tokens to issue requests, and the tokens get signed to indicate authorization.

5.206.6.B Register a Client in MWS

Oauth requires client registration. Its client credentials are used to validate that the client is allowed to authenticate on behalf of a resource owner. It involves giving the client its own credentials (username and password). MWS will first authenticate the client using a client id (username) and client secret (password), then will authenticate the resource owner.

Add the following lines to /opt/mws/etc/mws-config.groovy:

grails.plugin.springsecurity.oauthProvider.clients = [
     [
     clientId:"THE_CLIENT_ID",
     clientSecret:"THE_CLIENT_SECRET",
     authorizedGrantTypes:["password"]
     ]
]

Replace THE_CLIENT_ID with client id (username). For example: clientId:"iris". Also, replace THE_CLIENT_SECRET with client secret (password). For example: clientSecret:"irisclientpassword",. Note that the values for clientId and clientSecret are case sensitive.

You can register more than one client. For example:

grails.plugin.springsecurity.oauthProvider.clients = [
     [
     clientId:"client_id_1",
     clientSecret:"client_secret_1",
     authorizedGrantTypes:["password"]
     ],
     [
     clientId:"client_id_2",
     clientSecret:"client_secret_1",
     authorizedGrantTypes:["password"]
     ]
]

5.206.6.C Obtaining an Access Token from MWS for a Resource Owner (Logging In)

Before the client can access private data in MWS, the client must obtain an access token that grants access to the API. The token endpoint url is only used to gain an access token and log in a user.

Getting an access token:

POST http://localhost:8080/mws/rest/oauth/token?api-version=3
Adding header: 
     "Content-Type: application/x-www-form-urlencoded"
Request body (String):

grant_type=password&client_id=THE_CLIENT_ID&client_secret=THE_CLIENT_SECRET&username=RESOURCE_OWNER_USERNAME&password=RESOURCE_OWNER_PASSWORD

Example using curl:

curl -X POST -H "Content-Type: application/x-www-form-urlencoded" -v -d 'grant_type=password&client_id=iris&client_secret=irisclientpassword&username=moab-admin&password=secret' 'http://localhost:8080/mws/oauth/token'

Produces the following response:

* About to connect() to localhost port 8080 (#0)
*   Trying 127.0.0.1... connected
* Connected to localhost (127.0.0.1) port 8080 (#0)
> POST /mws/oauth/token HTTP/1.1
> User-Agent: curl/7.19.7 (x86_64-redhat-linux-gnu) libcurl/7.19.7 NSS/3.14.0.0 zlib/1.2.3 libidn/1.18 libssh2/1.4.2
> Host: localhost:8080
> Accept: */*
> Content-Type: application/x-www-form-urlencoded
> Content-Length: 126
> 
< HTTP/1.1 200 OK
< Server: Apache-Coyote/1.1
< Cache-Control: no-store
< Pragma: no-cache
< Set-Cookie: JSESSIONID=6CE8F9E7C454575FABCF3D156B153CFD; Path=/mws
< Content-Type: application/json;charset=UTF-8
< Transfer-Encoding: chunked
< Date: Fri, 18 May 2014 18:16:42 GMT
< 
* Connection #0 to host localhost left intact
* Closing connection #0
{"access_token":"b693eec0-6c93-4540-8b2f-1e170be08046","token_type":"bearer","expires_in":43096}

5.206.6.D Sending the Access Token to MWS When Requesting Protected Resource

After the client obtains an access token, it will send the access token to MWS in an HTTP authorization header for each rest call.

The client is responsible for handling user sessions with each access token, meaning the client has to request a new access token when a new user logs in.

Requesting an MWS resource (getting list of all nodes for example):

GET http://localhost:8080/mws/rest/nodes?api-version=3&fields=name
Adding authorization header: 
     "Authorization: Bearer ACCESS_TOKEN"

Example using curl:

curl -X GET -H "Authorization: Bearer b693eec0-6c93-4540-8b2f-1e170be08046" -v 'http://localhost:8080/mws/rest/nodes?api-version=3&fields=name'

Produces the following response:

* About to connect() to localhost port 8080 (#0)
*   Trying 127.0.0.1... connected
* Connected to localhost (127.0.0.1) port 8080 (#0)
> GET /mws/rest/nodes?api-version=3&fields=name HTTP/1.1
> User-Agent: curl/7.19.7 (x86_64-redhat-linux-gnu) libcurl/7.19.7 NSS/3.14.0.0 zlib/1.2.3 libidn/1.18 libssh2/1.4.2
> Host: localhost:8080
> Accept: */*
> Authorization: Bearer b693eec0-6c93-4540-8b2f-1e170be08046
> 
< HTTP/1.1 200 OK
< Server: Apache-Coyote/1.1
< Content-Type: application/json;charset=UTF-8
< Pragma: no-cache
< Set-Cookie: JSESSIONID=6CE8F9E7C454575FABCF3D156B153CFD; Path=/mws
< Content-Type: application/json;charset=UTF-8
< Content-Language: en-US
< Transfer-Encoding: chunked
< Date: Fri, 18 May 2014 18:39:07 GMT
< 
{"totalCount":3,"resultCount":3,"results":[{"name":"node1"},{"name":"node2"},{"name":"node3"}]}

Related Topics 

• Setting up MWS Security
• Version and Build Information

© 2016 Adaptive Computing