Configuring Moab Web Services

This section describes the location of the MWS configuration files. It also shows some examples of how to configure logging.

To see a full reference to all configuration and logging parameters available in MWS, see Configuration.

This topic contains these sections:

• Home Directory
• Configuration Files
• Logging Configuration Using mws-config.groovy
• LDAP Configuration Using mws-config.groovy
• Insight Database Configuration Using mws-config.groovy
• PAM (Pluggable Authentication Module) Configuration Using mws-config.groovy
• OAuth Configuration Using mws-config.groovy

Home Directory

The MWS home directory contains configuration files, log files, and files that serve features of MWS such as hooks and plugins. You should set the location of the MWS home directory using the MWS_HOME property. If you do not set MWS_HOME as a Java property or as an environment variable, then MWS will use /opt/mws as the default MWS_HOME.

Configuration Files

The primary configuration file is MWS_HOME/etc/mws-config.groovy. If this file is missing or contains errors, MWS will not start.

Configuration files can also be placed in the MWS_HOME/etc/mws.d directory. Any configuration files here get merged with MWS_HOME/etc/mws-config.groovy. In case of conflict, the configuration in MWS_HOME/etc/mws.d takes precedence.

If MWS_HOME/etc/log4j.properties exists, MWS will load it as well.

Logging Configuration Using mws-config.groovy

Shown below is an example that logs all error messages and fatal messages to /opt/mws/log/mws.log (For information about the format of the MWS logs, see "Standard Log Format" in the Moab Workload Manager Administrator Guide.). It also logs all stack traces to /opt/mws/log/stacktrace.log. Note that this example is not configured to log events; for details on logging events, see Configuring an Event Log.

Minimal logging configuration
-----------------------------

log4j = {
   appenders {
      rollingFile name: 'stacktrace',
         file: '/opt/mws/log/stacktrace.log',
         maxFileSize: '1GB'
      rollingFile name: 'rootLog',
         file: '/opt/mws/log/mws.log',
         threshold: org.apache.log4j.Level.ERROR,
         maxFileSize: '1GB'
   }
   root {
      debug 'rootLog'
   }
}

Alternatively, you may configure a console appender instead of a rolling file, as shown below.

Console logging configuration
-----------------------------
					
log4j = {
   appenders {
      rollingFile name: 'stacktrace',
         file: '/opt/mws/log/stacktrace.log',
         maxFileSize: '1GB'
      console name: 'consoleLog',
         threshold: org.apache.log4j.Level.ERROR
   }
   root {
      debug 'consoleLog'
   }
}

You may configure logging by using either MWS_HOME/etc/mws-config.groovy or MWS_HOME/etc/log4j.properties.

If you do not define any log4j configuration, MWS will write its log files to java.io.tmpdir. For Tomcat, java.io.tmpdir is generally set to $CATALINA_BASE/temp or CATALINA_TMPDIR.

Configuring an Event Log

Logging events to a flat file requires that you make a few changes to the configuration in the log4j section of the mws-config.groovy file so that events will be logged to the events.log file, and all other MWS logging information will be sent to the mws.log file.

Causing events.log to roll based on a time window

You can specify how often the events.log file rolls. The following example illustrates the configuration changes you will need make to mws-config.groovy to cause the events.log file to roll based on a time window. Note the following three examples:

• In this example, mws-config.groovy is configured so that events.log rolls daily at midnight.

Daily rolling events.log configuration in mws-config.groovy
------------------------------------

log4j = {
  def eventAppender = new org.apache.log4j.rolling.RollingFileAppender(name: 'events', layout: pattern(conversionPattern: "%m%n"))
  def rollingPolicy = new org.apache.log4j.rolling.TimeBasedRollingPolicy(fileNamePattern: '/tmp/events.%d{yyyy-MM-dd}', activeFileName: '/tmp/events.log')
  rollingPolicy.activateOptions()
  eventAppender.setRollingPolicy(rollingPolicy)

  appenders {
    appender eventAppender

    rollingFile name: 'rootLog',
      file: '/tmp/mws.log',
      maxFileSize: '1GB'
  }

  root {
    warn 'rootLog'
  }

  trace additivity:false, events:'com.ace.mws.events.EventFlatFileWriter'
}

Note the RollingFileAppender and the TimeBasedRollingPolicy lines. These lines configure MWS to write the event log to the events.log file. Rolled log files will have a date appended to their name in this format: "yyyy-MM-dd" (for example, events.log.2012-02-28).

• If you want the event log file to roll at the beginning of each month, change the fileNamePattern TimeBasedRollingPolicy date format to yyyy-MM. For example:

Monthly event logs
------------------------------------

def rollingPolicy = new org.apache.log4j.rolling.TimeBasedRollingPolicy(fileNamePattern: '/tmp/events.%d{yyyy-MM}', activeFileName: '/tmp/events.log')

• If you want the event log file to roll at the beginning of each hour, change the date format to yyyy-MM-dd_HH:00. For example:

Hourly event logs
------------------------------------

def rollingPolicy = new org.apache.log4j.rolling.TimeBasedRollingPolicy(fileNamePattern: '/tmp/events.%d{yyyy-MM-dd_HH:00}', activeFileName: '/tmp/events.log')

Configuring events.log to roll based on a file size threshold

You can also configure the events.log file to roll when the log size exceeds a specified threshold. The following example illustrates the configuration changes you will need to make to mws-config.groovy to cause the events.log file to roll on a size threshold. (In this example, mws-config.groovy is configured so that events.log rolls when its size exceeds 50 MB.)

mws-config.groovy configuration that rolls events.log based on file size
------------------------------------

log4j = {
  appenders {
    rollingFile name: 'events',
      file: '/tmp/events.log',
      maxFileSize: '50MB',
      maxBackupIndex:10

    rollingFile name: 'rootLog',
      file: '/tmp/mws.log',
      maxFileSize: '1GB'
  }

  root {
    warn 'rootLog'
  }

  trace additivity:false, events:'com.ace.mws.events.EventFlatFileWriter'
}

Note that maxFileSize is set to "50MB." This means that when the events.log file exceeds 50 MB, it will roll.

The name for the rolled log will be "events.log.1". When the new events.log file exceeds 50 MB, it will roll and be named "events.log.1", while the old "events.log.1" file will be renamed "events.log.2". This process will continue until the optional maxBackupIndex value is met. In the example above, maxBackIndex is set to 10. This means that MWS will delete all but the ten most recent events.log files. Using this feature helps prevent hard drives from filling up.

Additivity

The additivity attribute of the EventFlatFileWriter logger can be either true or false. If you specify true, events will be logged to the events.log file and the mws.log file. If you specify false, events will be logged to the events.log file only. (All other MWS logging information will be logged to the mws.log file, as configured by the rootLog appender.)

To log events to the mws.log file in addition to the events.log file, make the additivity:true configuration. For example:

Logging events to both events.log and mws.log
------------------------------------

trace additivity:true, events:'com.ace.mws.events.EventFlatFileWriter'

For more configuration options, see Apache Extras Companion for log4j.

Deleting old events

If your MongoDB server is version 2.2 or later, MongoDB will automatically delete events older than 30 days (by default). For more information, including how to change this default, see mws.events.expireAfterSeconds in Configuration.

If your MongoDB server is older than version 2.2, MongoDB will store event data indefinitely. However, if disk space is limited, you may want to regularly delete old, unneeded events from MongoDB. This section contains some examples of how you can do this.

Let's say that you want to delete events that are older than 90 days. (There are 86,400,000 milliseconds in a day, so in this example, 90*86400000 corresponds to 90 days in milliseconds.):

• You could run this script:

Delete events older than 90 days
------------------------------------

$ mongo
MongoDB shell version: 2.4.8
connecting to: test
> use mws
> db.event.remove({eventTime:{$lt:new Date(new Date().getTime()-90*86400000)}})
> exit

• To create a script to perform this task:

deleteOldEvents.sh
------------------------------------

#!/bin/bash
printf 'use mws_dev\ndb.event.remove({eventTime:{$lt:new Date(new Date().getTime()-90*86400000)}})\nexit' | mongo

• Now say that you want to set up a cron job ($crontab -e) so that old events are automatically deleted on a certain day of the week (for example, every Sunday at 2:00 a.m.), you would add an entry like this:

cron table entry to delete old events
------------------------------------

00 02 * * 0 /root/deleteOldEvents.sh

Configuring an Audit Trail Log

Audit logging enables you to track changes to Permissions, Roles, Tenants, and Principals.

mws-config.groovy configuration that enables audit logging
------------------------------------

	def auditAppender = new org.apache.log4j.rolling.RollingFileAppender(
			name: 'audit',
			layout: new com.ace.mws.logging.ACPatternLayout("%j\t\t\t%c{1}\t\t\t%m%n"))
	def auditRollingPolicy = new org.apache.log4j.rolling.TimeBasedRollingPolicy(
			fileNamePattern: '/opt/mws/log/audit.%d{yyyy-MM-dd}',
			activeFileName: '/opt/mws/log/audit.log')
	auditRollingPolicy.activateOptions()
	auditAppender.setRollingPolicy(auditRollingPolicy)

	appenders {
		rollingFile name: 'stacktrace',
				file: '/opt/mws/log/stacktrace.log',
				maxFileSize: '100MB'
		rollingFile name: 'rootLog',
				file: '/opt/mws/log/mws.log',
				maxFileSize: '100MB', //The maximum file size for a single log file
				maxBackupIndex: 10, //Retain only the 10 most recent log files, delete older logs to save space
				layout: new com.ace.mws.logging.ACPatternLayout(), //Configures the output format of each log entry
				threshold: org.apache.log4j.Level.ERROR //Ignore any logging entries less verbose than this threshold

		appender eventAppender
		appender auditAppender
	}

You can customize audit logging in ways you can customize event logging. For example, you can specify how often the audit.log file rolls. You can also configure the audit.log file to roll when the log size exceeds a specified threshold.

Follow the same steps indicated in the previous section on Configuring an Event Log for instruction on customizing audit logging; customization processes are the same for audit logging as for events logging.

audit.log file format

The default location to which the audit trail log is written is /opt/mws/log/audit.log. The log format is yyyy-MM-dd HH:mm:ss resource username action data. The following table offers a description for attributes included in the log format:

Sample audit.log format:

Audit trail log format
------------------------------------

2013-10-30 14:39:32,120 TENANT 'admin' updated resource named 'Engineering2' with values: 
     "name": "Engineering3",
     "attachedPrincipals": [{"name": "Engineering"}]

LDAP Configuration Using mws-config.groovy

Using a Supported LDAP Directory Type

To configure an MWS connection to an LDAP server, add the following parameters to mws-config.groovy:

Throughout the following examples in this topic, you will see dc=acme,dc=com. "acme" is only used as an example to illustrate what you would use as your own domain controller if your domain name was "acme.com." You should replace any references to "acme" with your own organization's domain name.

Here is a sample configuration for OpenLDAP.

If you followed the Adaptive Computing tutorial [link]"Setting up OpenLDAP on CentOS 6" your ldap.directory.type should be set to "OpenLDAP Using InetOrgPerson Schema".

Sample OpenLDAP configuration
-----------------------------

ldap.server  = "192.168.0.5"
ldap.port = 389
ldap.baseDNs = ["dc=acme,dc=com"]
ldap.bindUser = "cn=Manager,dc=acme,dc=com"
ldap.password = "*****"
ldap.directory.type = "OpenLDAP Using InetOrgPerson Schema"

Here is a sample configuration for Microsoft Active Directory.

Sample Active Directory configuration
-------------------------------------
					
ldap.server  = "192.168.0.5"
ldap.port = 389
ldap.baseDNs = ["CN=Users,DC=acme,DC=com","OU=Europe,DC=acme,DC=com"]
ldap.bindUser = "cn=Administrator,cn=Users,DC=acme,DC=com"
ldap.password = "*****"
ldap.directory.type = "Microsoft Active Directory"

To see how to configure a secure connection to the LDAP server, see Securing the LDAP Connection.

Using an Unsupported LDAP Directory Type

If you are not using one of the supported directory types, you can explicitly configure MWS to work with your LDAP schema by using the following parameters:

For example:

Advanced Active Directory configuration
-----------------------------

ldap.server  = "myldaphostname"
ldap.port = 389
ldap.baseDNs = ["CN=Users,DC=acme,DC=com","OU=Europe,DC=acme,DC=com"]
ldap.bindUser = "cn=Administrator,cn=Users,DC=acme,DC=com"
ldap.password = "*****"
ldap.user.objectClass = "person"
ldap.group.objectClass = "group"
ldap.ou.objectClass = "organizationalUnit"
ldap.user.membership.attribute = "memberof"
ldap.group.membership.attribute = "member"
ldap.user.name.attribute = "sAMAccountName"

Here is a similar example for OpenLDAP. Note there is no user membership attribute in the OpenLDAP InetOrgPerson schema and thus ldap.user.membership.attribute is set to null. This is allowable because the ldap.group.membership.attribute is set.

Advanced OpenLDAP configuration
-----------------------------

ldap.server  = "myldaphostname"
ldap.port = 389
ldap.baseDNs = ["dc=acme,dc=com"]
ldap.bindUser = "cn=Manager,dc=acme,dc=com"
ldap.password = "*****"
ldap.user.objectClass = "inetOrgPerson"
ldap.group.objectClass = "groupOfNames"
ldap.ou.objectClass = "organizationalUnit"
ldap.user.membership.attribute = null
ldap.group.membership.attribute = "memberUid"
ldap.user.name.attribute = "uid"

Overriding Attributes in a Supported LDAP Directory Type

You can also override attributes in supported directory types. For example, say you are using OpenLDAP with an NIS Schema. The group objectClass for NIS defaults to "groupOfNames," but you want to use "groupOfUniqueNames" instead while retaining all other defaults for NIS. You can do this by setting ldap.directory.type to "OpenLDAP Using NIS Schema" and overriding the ldap.group.objectClass attribute as follows:

Advanced OpenLDAP configuration
-----------------------------

ldap.directory.type = "OpenLDAP Using NIS Schema"
ldap.group.objectClass = "groupOfUniqueNames"

LDAP is not currently used to authenticate users to MWS. LDAP is only used to map principals to roles, as explained in Principals.

The user class in your LDAP schema must have an attribute that uniquely identifies a user (for example: "uid" or "sAMAccountName").

Insight Database Configuration Using mws-config.groovy

You will need to create and configure a read-only user that MWS will use to connect to the Insight database. It is recommended, but not required, that you configure an SSL connection to the Insight database.

To create a read-only PostgresSQL user

• On the machine that contains the PostgreSQL service, run the following commands:

  [root]# su - postgres
  [postgres]$ psql -c "CREATE USER mws WITH PASSWORD 'changeme!'"
  [postgres]$ psql -d moab_insight -U moab_insight -h 127.0.0.1 -c "GRANT SELECT ON ALL TABLES IN SCHEMA public TO mws;"

  [postgres]$ psql -d moab_insight -U moab_insight -h 127.0.0.1 -c "ALTER DEFAULT PRIVILEGES IN SCHEMA public GRANT SELECT ON TABLES TO mws;"

To allow MWS to query the Moab Insight database

• Add the following parameters to mws-config.groovy:

  Parameter	Description
  dataSource_insight.url	The JDBC URL for the Insight database.
  dataSource_insight.username	The username used to log in to the Insight database.
  dataSource_insight.password	The password for the username.

  The following is an example configuration:

  dataSource_insight.url = "jdbc:postgresql://localhost/moab_insight"
  dataSource_insight.username = "mws"
  dataSource_insight.password = "changeme!"

To configure connections to the Insight database over SSL

1. Ensure that your Java Runtime Environment (JRE) trusts the Insight database server’s X.509 certificate. If the certificate was signed by a commercial certificate authority (CA), such as Verisign, then MWS should trust the certificate automatically. Otherwise, you must configure the JRE that MWS is using to trust this certificate explicitly. To do so, follow the steps described in the PostgreSQL documentation that describe how to use keytool to import the certificate.
2. Configure MWS to connect to the database over SSL by appending a URL parameter of ssl=true to your dataSource_insight.url.

   dataSource_insight.url = "jdbc:postgresql://localhost/moab_insight?ssl=true"

PAM (Pluggable Authentication Module) Configuration Using mws-config.groovy

PAM functions as bridge to the underlying Unix authentication system. PAM treats the user as if it is local to the Unix machine doing the authenticating and uses whatever the Unix user is authenticating with, whether it be LDAP or NIS. PAM uses configuration files that specify the how, when, or what for authentication, session management, and account management. Each configuration file can be different. For example, sudo configuration file for the "sudo" command will handle authentication differently than the login configuration file. These configuration files are dynamically read for /etc/pam.d.

Requirements for PAM

In order to use PAM with MWS, the following is required:

• The PAM application package must be installed. For example:

yum install pam

• You must have a PAM configuration file in the /etc/pam.d directory. The following is an example of what a PAM configuration file might look like:

#%PAM-1.0
auth        required      pam_env.so
auth        sufficient    pam_unix.so nullok try_first_pass
auth        requisite     pam_succeed_if.so uid >= 1000 quiet_success
auth        required      pam_deny.so
 
 
account     required      pam_unix.so
account     sufficient    pam_localuser.so
account     sufficient    pam_succeed_if.so uid < 1000 quiet
account     required      pam_permit.so
 
 
password    requisite     pam_pwquality.so try_first_pass retry=3 authtok_type=
password    sufficient    pam_unix.so sha512 shadow nullok try_first_pass use_authtok
password    required      pam_deny.so
 
 
session     optional      pam_keyinit.so revoke
session     required      pam_limits.so
-session     optional      pam_systemd.so
session     [success=1 default=ignore] pam_succeed_if.so service in crond quiet use_uid
session     required      pam_unix.so

• (Optional) You must have PAM modules installed for your specific needs.

The PAM application comes with default modules—for example, pam_unix.xo—that will check username and password credentials with Unix. You may have to install others for your distribution.

Configuring MWS to Use PAM

To configure an MWS connection to PAM, add the following parameter to mws-config.groovy:

For example:

pam.configuration.service = "system-auth"

You can configure only one authentication method in mws-config.groovy—LDAP or PAM, but not both. If you have configured both LDAP and PAM, MWS defaults to using LDAP.

If you need multiple authentication methods, you must add them to your local PAM configuration. See your distribution documentation for details.

There is a security risk when authenticating local users through your PAM configuration. This behavior is highly discouraged and not supported by Adaptive Computing.

For more information about PAM, please see the following SLES and RedHat documentation.

OAuth Configuration Using mws-config.groovy

OAuth is a security framework designed to simplify authentication in web technologies. In the case of MWS, OAuth allows trusted client applications to securely delegate authentication to MWS. Once MWS has authenticated a user by verifying the username and password in LDAP, PAM, or NIS, MWS returns an access token to the client. The client then presents this access token to MWS to access resources. OAuth is very flexible and allows MWS to work in many different scenarios by use of grant types. For more information on OAuth and grant types, please see the following OAuth documentation.

Example Using 'password' Grant Type

Terminology

Resource Owner: The person accessing and manipulating data. For MWS, this would be the person who logs into the client (the user).

Service Provider: The site or service where protected resources live. This can be (but is not necessarily) also the identify provider, where usernames and passwords are stored. This is the MWS service itself.

Client: The application that wants to access a resource. For MWS this is the user interface, potentially including APIs and command-line tools.

Protected Resource: The data for which protection is desired. For MWS this would be Moab itself, and interaction with Moab.

Access Token: Instead of user credentials, OAuth uses tokens to issue requests, and the tokens get signed to indicate authorization.

Register a Client in MWS

Oauth requires client registration. Its client credentials are used to validate that the client is allowed to authenticate on behalf of a resource owner. It involves giving the client its own credentials (username and password). MWS will first authenticate the client using a client id (username) and client secret (password), then will authenticate the resource owner.

Add the following line to /opt/mws/etc/mws-config.groovy:

grails.plugin.springsecurity.oauthProvider.clients = [
     [
     clientId:"THE_CLIENT_ID",
     clientSecret:"THE_CLIENT_SECRET",
     authorizedGrantTypes:["password"]
     ]
]

Replace THE_CLIENT_ID with client id (username). For example: clientId:"iris". Also, replace THE_CLIENT_SECRET with client secret (password). For example: clientSecret:"irisclientpassword",. Note that the values for clientId and clientSecret are case sensitive.

You can register more than one client. For example:

grails.plugin.springsecurity.oauthProvider.clients = [
     [
     clientId:"client_id_1",
     clientSecret:"client_secret_1",
     authorizedGrantTypes:["password"]
     ],
     [
     clientId:"client_id_2",
     clientSecret:"client_secret_1",
     authorizedGrantTypes:["password"]
     ]
]

Obtaining an Access Token from MWS for a Resource Owner (Logging In)

Before the client can access private data in MWS, the client must obtain an access token that grants access to the API. The token endpoint url is only used to gain an access token and log in a user.

Getting an access token:

POST http://localhost:8080/mws/rest/oauth/token?api-version=3
Adding header: 
     "Content-Type: application/x-www-form-urlencoded"
Request body (String):

grant_type=password&client_id=THE_CLIENT_ID&client_secret=THE_CLIENT_SECRET&username=RESOURCE_OWNER_USERNAME&password=RESOURCE_OWNER_PASSWORD

Example using curl:

curl -X POST -H "Content-Type: application/x-www-form-urlencoded" -v -d 'grant_type=password&client_id=iris&client_secret=irisclientpassword&username=moab-admin&password=secret' 'http://localhost:8080/mws/oauth/token'

Produces the following response:

* About to connect() to localhost port 8080 (#0)
*   Trying 127.0.0.1... connected
* Connected to localhost (127.0.0.1) port 8080 (#0)
> POST /mws/oauth/token HTTP/1.1
> User-Agent: curl/7.19.7 (x86_64-redhat-linux-gnu) libcurl/7.19.7 NSS/3.14.0.0 zlib/1.2.3 libidn/1.18 libssh2/1.4.2
> Host: localhost:8080
> Accept: */*
> Content-Type: application/x-www-form-urlencoded
> Content-Length: 126
> 
< HTTP/1.1 200 OK
< Server: Apache-Coyote/1.1
< Cache-Control: no-store
< Pragma: no-cache
< Set-Cookie: JSESSIONID=6CE8F9E7C454575FABCF3D156B153CFD; Path=/mws
< Content-Type: application/json;charset=UTF-8
< Transfer-Encoding: chunked
< Date: Fri, 18 May 2014 18:16:42 GMT
< 
* Connection #0 to host localhost left intact
* Closing connection #0
{"access_token":"b693eec0-6c93-4540-8b2f-1e170be08046","token_type":"bearer","expires_in":43096}

Sending the Access Token to MWS When Requesting Protected Resource

After the client obtains an access token, it will send the access token to MWS in an HTTP authorization header for each rest call.

The client is responsible for handling user sessions with each access token, meaning the client has to request a new access token when a new user logs in.

Requesting an MWS resource (getting list of all nodes for example):

GET http://localhost:8080/mws/rest/nodes?api-version=3&fields=name
Adding authorization header: 
     "Authorization: Bearer ACCESS_TOKEN"

Example using curl:

curl -X GET -H "Authorization: Bearer b693eec0-6c93-4540-8b2f-1e170be08046" -v 'http://localhost:8080/mws/rest/nodes?api-version=3&fields=name'

Produces the following response:

* About to connect() to localhost port 8080 (#0)
*   Trying 127.0.0.1... connected
* Connected to localhost (127.0.0.1) port 8080 (#0)
> GET /mws/rest/nodes?api-version=3&fields=name HTTP/1.1
> User-Agent: curl/7.19.7 (x86_64-redhat-linux-gnu) libcurl/7.19.7 NSS/3.14.0.0 zlib/1.2.3 libidn/1.18 libssh2/1.4.2
> Host: localhost:8080
> Accept: */*
> Authorization: Bearer b693eec0-6c93-4540-8b2f-1e170be08046
> 
< HTTP/1.1 200 OK
< Server: Apache-Coyote/1.1
< Content-Type: application/json;charset=UTF-8
< Pragma: no-cache
< Set-Cookie: JSESSIONID=6CE8F9E7C454575FABCF3D156B153CFD; Path=/mws
< Content-Type: application/json;charset=UTF-8
< Content-Language: en-US
< Transfer-Encoding: chunked					
< Date: Fri, 18 May 2014 18:39:07 GMT
< 
{"totalCount":3,"resultCount":3,"results":[{"name":"node1"},{"name":"node2"},{"name":"node3"}]}

Related Topics 

• Setting up MWS Security
• Version and Build Information

© 2015 Adaptive Computing