Table of Contents

6 MWS Plugins (Beta)

MWS Plugins are currently in beta. Interfaces may change significantly in future releases.

6.1 Plugin Overview

• An introduction to the concept of MWS plugins
• How to configure Moab Workload Manager to interact with MWS plugins
• A description of the plugin lifecycle
• How plugins are driven by events
• How to expose web services from a plugin
• How plugin utility services may be used
• How data report collisions between plugins are consolidated
• How calls from MWM are routed to MWS plugins

6.1.1 Introduction

Features

Plugins can

• be created, modified, and deleted without restarting Moab Workload Manager or MWS.
• be defined in Groovy and uploaded to MWS without restarting.
• have individual data storage space and configuration.
• access MWS configuration and RESTful web services.
• log to a standard location configured in MWS.
• be polled at a regular interval (configured on a per-plugin basis).
• report current state data to Moab Workload Manager through the Resource Manager interface.
• be informed of important system events.
• be individually stopped, started, paused, and resumed.
• expose secured and unsecured custom web services for external use.
• be manipulated via a full RESTful API (see Resources for more information).
• be manipulated via a full user interface in a web browser.

Terminology

There are two distinct terms in the plugin layer: plugin types and plugins (also called plugin instances).

Plugin Types

Plugin Types can be considered plugin templates with built-in logic. In object-oriented programming languages, this relates to the concept of a class. They possess certain abilities, or methods, that can be called by Moab Web Services to query or update information about certain resources. They also can define methods which will be exposed to external clients as web services. They do not contain any configuration or current data, but they are often tied to a type of component, such as components that communicate with Moab's WIKI Protocol or those that are built on a certain product.

They can define several types of methods:

1. Instance methods that return information about the current plugin, such as getState. While these are defined in the plugin type, the plugin type itself does not have a state.
2. The poll event method that is called at a configured interval.
3. Lifecycle event methods of plugins created from the plugin type, such as beforeStart and afterStart.
4. RM event methods that are called by Moab when certain events occur.
5. Web service methods that expose custom functionality as public web services.

Some examples of plugin types include the Native and MSM plugin types.

Plugins (Instances)

Plugins (also called plugin instances) are created from plugin types. They contain current data or configuration and use the plugin type methods to interact with resources.

Interactions with MWM as a Resource Manager

The plugin layer in MWS is integrated with Moab Workload Manager via the Native Resource Manager (RM) interface. When utilizing plugins, MWS is configured as a RM in Moab as explained in the next section. Events from Moab are pushed through the RM interface to MWS which is then pushed to each plugin in turn. The relationship between Moab Web Services, Moab, and plugins is shown in the following image:

See Consolidating Data and Reporting State Data for more information.

6.1.2 Configuring Moab

RMCFG[mws]              TYPE=NATIVE
RMCFG[mws]              FLAGS=UserSpaceIsSeparate
RMCFG[mws]              CLUSTERQUERYURL=exec://$TOOLSDIR/mws/cluster.query.mws.pl
RMCFG[mws]              WORKLOADQUERYURL=exec://$TOOLSDIR/mws/workload.query.mws.pl
RMCFG[mws]              JOBCANCELURL=exec://$TOOLSDIR/mws/job.cancel.mws.pl
RMCFG[mws]              JOBMIGRATEURL=exec://$TOOLSDIR/mws/vm.migrate.mws.pl
RMCFG[mws]              JOBMODIFYURL=exec://$TOOLSDIR/mws/job.modify.mws.pl
RMCFG[mws]              JOBREQUEUEURL=exec://$TOOLSDIR/mws/job.requeue.mws.pl
RMCFG[mws]              JOBRESUMEURL=exec://$TOOLSDIR/mws/job.resume.mws.pl
RMCFG[mws]              JOBSTARTURL=exec://$TOOLSDIR/mws/job.start.mws.pl
RMCFG[mws]              JOBSUBMITURL=exec://$TOOLSDIR/mws/job.submit.mws.pl
RMCFG[mws]              JOBSUSPENDURL=exec://$TOOLSDIR/mws/job.suspend.mws.pl
RMCFG[mws]              NODEMODIFYURL=exec://$TOOLSDIR/mws/node.modify.mws.pl
RMCFG[mws]              NODEPOWERURL=exec://$TOOLSDIR/mws/node.power.mws.pl
RMCFG[mws]              RESOURCECREATEURL=exec://$TOOLSDIR/mws/resource.create.mws.pl
RMCFG[mws]              SYSTEMMODIFYURL=exec://$TOOLSDIR/mws/system.modify.mws.pl
RMCFG[mws]              SYSTEMQUERYURL=exec://$TOOLSDIR/mws/system.query.mws.pl

The next step is to edit the MWS values in /opt/moab/etc/cloud.cfg. Here are the default values:

CONFIG[default]                 MWS_URL=http://localhost:8080/mws
CONFIG[default]                 MWS_USERNAME=admin
CONFIG[default]                 MWS_PASSWORD=adminpw

MWS_USERNAME and MWS_PASSWORD must match the values of auth.defaultUser.username and auth.defaultUser.password, respectively, found in /opt/mws/etc/mws-config.groovy.

The *.mws.pl scripts should be located in the tools/mws of the Moab home directory. The Moab/WebServices.pm module must also be available to the scripts. All of these files may be found in the tools/mws and lib/perl5 directories of the Moab tar file. They are automatically installed if Moab is configured with the --with-mws flag or they can be copied directly from there to the tools folder in your Moab home directory.

To enable such actions as submitting jobs as different users, the ENABLEPROXY=TRUE option must be present in the ADMINCFG configuration line and the OSCREDLOOKUP option must be set to NEVER as follows:

ADMINCFG[1]           USERS=root      ENABLEPROXY=TRUE
OSCREDLOOKUP          NEVER

6.1.3 Lifecycle States

To see descriptions of each state, see the PluginState API.

Events that occur during lifecycle state changes may be found in the Events section.

6.1.4 Events

6.1.5 Custom Web Services

For example, suppose a resource needs to notify a plugin that provisioning of a virtual machine has been completed. Instead of having the plugin poll the resource to verify that the provisioning was finished, the plugin could expose a custom web service to handle notification from the resource itself.

def vmProvisionFinished(Map params) {
	// Handle event
	return [messages:["Event successfully processed"]]
}

Additionally, plugin types may define web services which are unsecured, meaning that a user or application account is not required to access it. A full explanation of the syntax and creation of custom secured and unsecured web services may be seen on the Exposing Web Services page.

For information how resources can access plugin web services, see Accessing Plugin Web Services.

6.1.6 Utility Services

• Accessing the individual datastore.
• Reporting state data to Moab through the Resource Manager interface.
• Manipulating other plugins and controlling their lifecycle.
• Accessing REST resources from Moab Web Services.

All bundled services are covered on their respective pages in the Quick Reference guide under "Plugin Services".

It may also be necessary or desired to create additional utility services when creating new plugin types. The easiest way to do this is to create a utility service which is called by convention a Translator, due to the fact that it typically can "translate" from a specific resource or API to data which can be used by the plugin type.

Finally, custom components may be used to fulfill use cases not covered by bundled services or custom translators.

6.1.7 Data Consolidation

For example, suppose two plugins exist, pluginA and pluginB. These plugins both report data for a node with an ID of node1. However, each reports a different node power state. Plugin A reports the power as ON, while plugin B reports the power as OFF. The data collision that occurs due to these two contradictory reports is resolved by the timing of when they save their node reports. Assume that both plugins report node data when polling only. If plugin A is polled first and plugin B second, the node will be reported as OFF until plugin A is polled again and vice versa.

The simple workaround for this issue is to ensure that no two plugins report the same resource or that they report different properties of the same resource. For example, if plugin A only modified the power state and plugin B only modified the available disk resource, these two plugins would work in harmony to provide a consistent view of the node resource.

6.1.8 Routing

Currently all plugins receive all commands from Moab. This means that each plugin will receive the command to start a job if sent from Moab, even if that plugin does not handle the job. This means that plugins must ensure they handle only actions or commands for resources which they report or handle.

6.2 Plugin Developer's Guide

Several plugin types are provided with Moab Web Services (see Plugin Types in the Quick Reference), but it is easy to create additional plugin types and add their functionality to web services. This involves using Groovy, which is based on the Java programming language. This section describes the general guidelines and specifics of implementing new plugin types.

API Classes and Interfaces

There are several packages and classes available to assist in creating plugin types. These can all be found in the API documentation under the com.ace.mws.plugins package.

6.2.1 Requirements

1. The class name must end in Plugin.
2. There must exist id field getter and setter methods:

* public String getId();
* public void setId(String id);

The id field may be stored in whichever way desired as long as the getter and setter are available as shown above, but will most likely be implemented as follows:

class BasicPlugin {
	String id
}

In this case, String id will be expanded by the Groovy compiler to the full getter and setter method definitions given above. In other words, no explicit method definitions are actually needed. Note that the BasicPlugin shown above is able to be uploaded as a plugin type to MWS, but does not actually do anything.

6.2.2 Dynamic Methods

// Full definitions can be found in [AbstractPlugin|api:com.ace.mws.plugins.AbstractPlugin]
public void start() throws PluginStartException;	// Equivalent to the start method in the [Plugin Control Service|Plugin Services]
public void stop() throws PluginStopException;  // Equivalent to the stop method in the [Plugin Control Service|Plugin Services]
public Log getLog();	// Discussed in [Logging|guide:pluginLogging]
public ConfigObject getAppConfig();	// Discussed in [Configuration|guide:pluginConfiguration]
// Full definition for these methods can be found in [AbstractPluginInfo|api:com.ace.mws.plugins.AbstractPluginInfo]
public String getPluginType();
public PluginState getState();
public Integer getPollInterval();
public Boolean getAutoStart();
public Map<String, Object> getConfig();	// See [Configuration|guide:pluginConfiguration]

Many of these methods are provided for convenience and are discussed in the linked pages or the following sections.

6.2.3 Logging

The logger may used to register messages to the MWS log at several levels (in order of severity):

1. trace
2. debug
3. info
4. warn
5. error
6. fatal

Each of these levels is available as a method on the logger, for example:

public void poll() {
	getLog().debug("getLog() is equivalent to just using 'log' in Groovy")
	log.debug("This is a debug message and is used for debugging purposes only")
	log.info("This is a informational message")
	log.warn("This is a warning")
	log.error("This is an error message")
}

Logger Name

Each logger in the MWS logging configuration has a name. In the case of plugins, it is comprised of the full class name, including the package, prepended by "plugins.". For example, a plugin class of "example.LoggingPlugin" will have access to a logger configured as "plugins.example.LoggingPlugin".

Logging Configuration

The logging configuration is done through the MWS configuration file. See the MWS Configuration page for more information on configuring loggers. A good configuration for developing plugin types may be to add "plugins" at the debug level. Be sure to set the log level threshold down for the desired appender.

log4j = {
  …
  // Appender configuration
  ...
  debug "plugins"
}

6.2.4 Configuration

Individual Plugin Configuration

The individual plugin configuration is separate for each instance of a plugin. This may be used to store current configuration information such as access information for linked resources. It should not be used to store cached information or non-configuration related data. The Individual Datastore should be used instead for these cases.

It is accessed by using the getConfig method discussed in Dynamic Methods.

public void poll() {
  def configFromMethod = getConfig()
  // OR an even simpler method…
  def configFromMethod = config
}

A common case is to retrieve the configuration in the configure method, verify that it matches predetermined criteria, and utilize it perform initial setup of the plugin (e.g. initialize libraries needed to communicate with external resources). For example, to verify that the configuration contains the keys "username" and "password", the following code may be used.

public void configure() throws InvalidPluginConfigurationException {
  def myConfig = config
  // This checks to make sure the key exists in the configuration Map and that the value is not empty or null
  if (!myConfig.containsKey("username") || !myConfig.username)
  	throw new InvalidPluginConfigurationException("The username configuration parameter must be provided")
  if (!myConfig.containsKey("password") || !myConfig.password)
    throw new InvalidPluginConfigurationException("The password configuration parameter must be provided")
}

Access MWS Configuration

The MWS application configuration can also be accessed in plugin types. This configuration is global for the entire application and can be modified by the administrator as shown in the MWS Configuration section.

It is accessed by using the getAppConfig method discussed in Dynamic Methods. This is demonstrated below.

public void poll() {
  // Retrieve the current MWS_HOME location
  def mwsHome = appConfig.mws.home.location
  // OR an even simpler method…
  def mwsHome = getAppConfig().mws.home.location
}

Any of the properties shown in the Configuration reference may be accessed. Custom properties may also be registered and accessed:

plugins.custom.property = "This is my custom property"

public void poll() {
  assert appConfig.plugins.custom.property=="This is my custom property"
}

6.2.5 Individual Datastore

• Data is persisted to the Mongo database and will be available even if the plugin or MWS is restarted.
• The data must be stored in groups of data called collections. These correspond directly to MongoDB collections.
• Each plugin may have an arbitrary number of collections.
• Collections are guaranteed not to collide if there are identically named collections between two plugin types or even two plugin instances.
• Each collection contains multiple objects or entries. These correspond directly to MongoDB documents.
• The values of entries may be any object which can be serialized to MongoDB: simple types (int or Integer), Maps, and Lists.
• A collection is automatically created whenever an entry is added to it, it does not need to be specifically initialized.

To utilize the datastore, the Plugin Datastore Service must be used. Operations are provided to add, query, and remove data from each collection.

Example

The example below demonstrates two web services. The first adds multiple entries containing various types of data to an arbitrarily named collection. The second retrieves the data and returns it to the user.

package example
import com.ace.mws.plugins.*
class DatastorePlugin extends AbstractPlugin {
	IPluginDatastoreService pluginDatastoreService
	def storeData(Map params) {
		def collectionName = params.collectionName
		def data = [[boolVal:true], [stringVal:"String"], [intVal:1], [nullVal:null]]
		if (pluginDatastoreService.addData(collectionName, data))
			log.info("Data successfully added")
		else
			log.info("There was an error adding the data")
	}
	def retrieveData(Map params) {
		def collectionName = params.collectionName
		return pluginDatastoreService.getCollection(collectionName)
	}
}

6.2.6 Exposing Web Services

1. The method must declare that it returns Object or def.
2. The method must define a single argument of type Map.
3. The method must actually return a List, Map, or a complex object; no simple types such as int or String can be returned.
4. The method must not be declared as private or protected; only public or unscoped methods will be recognized as web services.

Parameters and Request Body

The Map argument will contain all parameters passed into the web service by the client. See Accessing Plugin Web Services for additional details.

Parameters may be passed into the web service call as normal URL parameters such as ?param=value&param2=value2, as key-value pairs in the POST body of a request, or as JSON in the body.

For the first two cases, the parameters will be available on the Map argument passed into the web service call as key value pairs matching those of the request. Note that in these cases all keys and values will be interpreted as strings. However, the parameters object has several helper methods to convert from Strings to simple types, such as booleans, integers, doubles, floats, and lists. If the value is not a valid simple type, null is returned.

GET <webServiceUrl>?key=value&key2=true&key3=5&list=1&list=2
def serviceMethod(Map params) {
	assert params.key=="value"
	assert params.key2=="true"
	assert params.bool('key2')==true
	assert params.key3=="5"
	assert params.int('key3')==5
	assert params.list('list')==[1, 2]
	// Null is returned if the conversion is invalid
	assert params.int('key')==null
}

When the body possesses JSON, the parsed JSON object or array will be available within a parameter called body in the Map argument. In this scenario, the types of the values are preserved by the JSON format.

POST <webServiceUrl> with JSON body of
{"key":"value","key2":true,"key3":5}
def serviceMethod(Map params) {
	assert params.body.key=="value"
	assert params.body.key2==true
	assert params.body.key3==5
}

Unsecured Web Services

There are times when it is desirable to create a plugin with a publicly available web service that does not require a valid Application Account in order to access it. In these cases, the Unsecured annotation may be used on the plugin web service method. No authentication will be performed on Unsecured web services. An example of using the annotation is given below.

@Unsecured
def retrievePublicData(Map params) {
	return [data:["data item 1", "data item 2"]]
}

Be cautious in using this annotation as it may potentially present a security risk if sensitive data is returned from the web service.

Returning Errors

In order to signify an error occurred or invalid data was provided, the WebServiceException class may be thrown from any custom web service. This exception contains constructors and fields for a list of messages and a HTTP response code. For example, suppose that the user provided inadequate information. The web service could use the following code to notify the user and prompt them to take action with custom messages.

def service(Map params) {
	// Handle invalid input
	if (!params.int('a'))
		throw new WebServiceException("Invalid parameter 'a' specified, please specify an integer!", 400)
	// Use params.a correctly …
}

For the example above, a 400 response code (bad request) would be returned with a response body as follows:

{
  "messages":[
	"Invalid parameter 'a' specified, please specify an integer!"
  ]
}

If any other exception is thrown from a web service (ie Exception, IllegalArgumentException, etc), a 500 response code will be returned with the following response body:

{
  "messages":[
	"A problem occurred while processing the request",
	"Message provided in the exception constructor"
  ]
}

See the Responses and Return Codes section for more information on error formats in MWS.

Accessing the HTTP Request Method

The HTTP method used for the request is available from the Map parameters argument. The key used to access it is stored as a static field in PluginConstants called WEB_SERVICES_METHOD. The value is a string which can be GET, POST, PUT, or DELETE. The following example demonstrates how this could be used with the WebServiceException to create a REST API with a plugin.

def serviceMethod(Map params) {
	// Check to make sure that this request used the HTTP GET method
	// Throw a 405 error (method not supported) if not
	if (params[PluginConstants.WEB_SERVICES_METHOD]!="GET")
		throw new WebServiceException("Method is not supported", 405)
}

6.2.7 Reporting State Data

Generating Reports

To generate a report, simply create a new instance of a report depending on the type of object to be reported:

Each report has a single required parameter for creating a new instance - the ID of the object which is being reported. Once the report instance has been created, any property may be modified as shown in the API documentation links in the table above. The following example shows the creation of a simple node report and modification of a few properties:

public void poll() {
	NodeReport node = new NodeReport("node1")
	node.timestamp = new Date()
	node.image = "centos-5.4-stateless"
	… // Set other properties and persist the report
}

Special Cases in Field Values

All complex types, such as Lists, Maps, and objects (not including Enumerated values such as NodeReportState and JobReportState) have default values set for them and are not required to be instantiated before use. For example, the metrics property of a node report may be modified as follows:

public void poll() {
	NodeReport node = new NodeReport("node1")
	// The following assignments are equivalent in their functionality
	node.features.add("FEAT1")
	node.features << "FEAT2"
	// The following assignments are equivalent in their functionality
	node.metrics.METRIC1 = 4d
	node.metrics["METRIC2"] = 125.5
	… // Set other properties and persist the report
}

For the resources and requirements (jobs only) properties, assignments may be made easily without checking for previously existing values or null objects. For example, resources may be added to the resources property simply by accessing it as a Map:

public void poll() {
	NodeReport node = new NodeReport("node1")
	node.resources.RES1.total = 10
	node.resources.RES1.available = 3
	node.resources["RES2"].total = 10
	node.resources["RES2"].available = 10
	… // Set other properties and persist the report
}

The job report's requirements property has some additional handling to allow it to be accessed as a single JobReportRequirement object, such as in the following example:

public void poll() {
	JobReport job = new JobReport("job.1")
	job.nodeCountMinimum = 4
	job.processorCountMinimum = 2
	job.requiredNodeFeatures << "FEAT1"
	job.preferredNodeFeatures << "FEAT2"
	… // Set other properties and persist the report
}

Although multiple requirements may be added to the requirements list to provide consistent with the MWS Job resource, only the first requirement object's properties will be reported to MWM through the RM interface.

Managing Images for Nodes

In order to have Moab Workload Manager recognize a node as a virtual machine hypervisor, it must have a valid associated Image. In particular, the image property on a node report must set to a valid image name. The image's extensions.xcat.hvType and virtualizedImages properties are then used to report the correct hypervisor type and supported virtual machine images to MWM. If the image is invalid, it will be ignored and the node will not be recognized as a hypervisor.

Persisting a Report

After a report has been generated and all desired fields have been updated, the report must be sent to one of the three bundled RM services for persisting. If this is not done, the report will be discarded and will not be considered when reporting state information to MWM. The RM services are shown below according to the object type that they handle:

Each service has a save method that must be called with a list of reports to persist as shown in the following example.

INodeRMService nodeRMService
public void poll() {
	NodeReport node = new NodeReport("node1")
	// Change the state
	node.state = NodeReportState.BUSY
	// Persist
	nodeRMService.save([node])
}

Once this is done, the reports will be persisted to MongoDB and will be included in consolidation.

Report Consolidation

During each iteration of Moab Workload Manager's cycle, it will query MWS through the RM interface to access current node, virtual machine, and job information. At this point, all reports are loaded from the database and consolidated into a single report of each object as explained in the Data Consolidation section.

Several points must be noted when considering data consolidation:

• All unset, or null, values for properties on reports are ignored.
• Once consolidation is performed, node and virtual machine reports that were included are removed from the database (see delete-old below).
• Job reports are kept in the database until the Job state has been reported as a completed active (see JobReportState) and a configurable amount of time has passed. See the plugins.job.purge.duration Configuration parameter.

In some cases it may be desired to query MWS directly for the current consolidated node, virtual machine, and job reports. This may be done using the following URLs which return data in the Wiki interface format (see the Native plugin type for more information).

Additionally, URL parameters may be used to modify the output of the query as shown in the table below.

In effect, each iteration of MWM's cycle clears out all old reports using the delete-old parameter. Therefore it is recommended to use the default values (delete-old and previous both set to false) in between MWM scheduling iterations (after new reports are generated but before MWM queries for the consolidated report), and to use the previous parameter set to true after a MWM scheduling iteration has run but no new reports have been generated.

6.2.8 Controlling Lifecycle

• create plugin instances dynamically with specific configuration.
• retrieve plugin instances by ID or based on configuration properties.
• start or stop plugin instances.
• verify plugin instance configuration.

Creating Plugins

Several methods are provided to allow on-the-fly creation of new plugins. Generally, they allow a plugin with a specific ID and plugin type (as a string or as a Groovy Class) to be created with optional configuration properties. These properties should match the fields in the Plugin API.

If any configuration properties are omitted, the defaults will be used as described in the Setting Default Plugin Configuration section. A boolean value is also returned indicating whether the creation succeeded or not.

Note that the createPlugin methods will initialize the plugin for retrieval or usage and attempt to start the plugin if the autoStart property is true.

Retrieving Plugins

Plugins may be retrieved by using an ID, querying by plugin type, or even querying based on configuration parameters. Several methods are provided to perform these functions as shown on the Plugin Control Service page.

Starting and Stopping Plugins

Plugins may also be started or stopped on demand. These two methods are exposed directly as start and stop on the plugin control service. Although each method does not return any data, exceptions are thrown if errors are encountered.

Verifying Plugin Configuration

Finally, the plugin control service may be used to verify plugin configuration at any point instead of just when the plugin is started or modified. This may be useful to attempt to modify plugin configuration directly through the setConfig dynamic method and then verify that the new configuration is valid for the plugin. Exceptions are thrown if the plugin or the configuration is invalid.

Examples

If an error state is detected it may be necessary to stop the current plugin instance until corrective action can be taken. This may be done using the following code:

package example
import com.ace.mws.plugins.*
class ErrorPlugin {
	IPluginControlService pluginControlService
	public void poll() {
		// Error is detected, stop plugin instance!
		try {
			log.warn("An error was detected, trying to stop the plugin ${id}")
			pluginControlService.stop(id)
			log.warn("The plugin was successfully stopped")
		} catch(PluginStopException e) {
			log.error("Plugin instance ${id} could not be stopped", e)
		}
	}
}

6.2.9 Accessing MWS REST Resources

All accesses to resources require a HTTP method to use (such as GET, POST, PUT, or DELETE) and a relative URL (such as /rest/jobs). Although it mimics the REST resource interface, no actual requests are made and no data is transmitted through the network.

Authentication

All resources are available to the Moab REST Service, and no authentication or Application Accounts are needed.

Caution must be used when developing plugin types, as there are no restrictions to what may be done with the Moab REST Service. This is especially true when not utilizing hooks as discussed below.

Hooks

If Pre and Post-Processing Hooks are utilized in MWS, the plugin type developer may choose whether or not they are executed when performing a "request" through the Moab REST service. This is done through the hooks option as documented on the Moab Rest Service page.

Examples

This code retrieves a list of all nodes, and is equivalent to the Get All Nodes task.

package example
import com.ace.mws.plugins.*
import net.sf.json.*
class RestPlugin {
	IMoabRestService moabRestService
	public void poll() {
		def result = moabRestService.get("/rest/nodes")
		// OR with the hook enabled…
		def result = moabRestService.get("/rest/nodes", hooks:true)
		assert result instanceof MoabRestResponse
		assert nodes instanceof List
		log.debug("Nodes list:")
		nodes.each { JSON node ->
			log.debug(node.id)
		}
	}
}

This code adds a flag to a job, and is equivalent to the Modify Job Attributes task. This request also enables the hook (if one is configured) for the "request" and uses a URL parameter. This is the equivalent of making a call to /rest/jobs/job.1?proxy-user=adaptive.

package example
import com.ace.mws.plugins.*
import net.sf.json.*
class RestPlugin {
	IMoabRestService moabRestService
	public void poll() {
		def jobId = "job.1"
		def result = moabRestService.put("/rest/jobs/"+jobId, hooks:true, params:['proxy-user':'adaptive']) {
			[flags:["RESTARTABLE"]]
		}
		assert result.isSuccess()
	}
}

6.2.10 Handling Events

The Polling Event

To maintain current information, each plugin is polled at a specified time interval. The following method definition is required to utilize the polling event.

void poll() { … }

Typically this polling method is used to report node and virtual machine information. By default, the polling interval is set to 30 seconds, but can be modified for all or individual plugins as explained in Plugin Management.

When a polling event occurs, the poll method on the target plugin is called. This method may perform any function desired and should typically make calls to the Node RM Service, the Virtual Machine RM Service, and the Job RM Service services to report the current state of nodes and virtual machines. For example, the poll method in the Native plugin type is implemented as follows:

This is an extremely simplified version of what is actually implemented in the Native plugin type.

INodeRMService nodeRMService;
IVirtualMachineRMService virtualMachineRMService;
public void poll() {
	nodeRMService.save(getNodes());
	virtualMachineRMService.save(getVirtualMachines());
}

This simple poll method calls two other helper methods called getNodes and getVirtualMachines to retrieve node and virtual machine reports. These reports are then sent to the appropriate RM service. See Reporting State Data for more information on the RM services, but the objective of this example is to demonstrate one possible use of the poll event handler. Other plugin types, on the other hand, may use the poll event to update internal data from pertinent resources or make calls to external APIs.

Lifecycle Events

Events are also triggered for certain lifecycle state changes. The following method definitions are required to receive lifecycle events.

public void configure() throws InvalidPluginConfigurationException { … }
public void beforeStart() { … }
public void afterStart() { … }
public void beforeStop() { … }
public void afterStop() { … }

Each event is described in the table below with the associated state change when the event is triggered.

Currently, no events are triggered for pausing, resuming, erroring, or clearing errors for plugins.

RM Events

When MWS is configured as a Moab Resource Manager (see Configuring Moab), RM events are sent from Moab to each plugin according to the Routing page. The following method definitions are required to receive these events.

public boolean jobCancel(List<String> jobs) { … }
public boolean jobModify(List<String> jobs, Map<String, String> properties) { … }
public boolean jobRequeue(List<String> jobs) { … }
public boolean jobResume(List<String> jobs) { … }
public boolean jobStart(String jobId, String taskList, String username) { … }
public boolean jobStart(String jobId, String taskList, String username, Map<String, String> properties) { … }
public boolean jobSubmit(Map<String, String> properties) { … }
public boolean jobSuspend(List<String> jobs) { … }
public boolean nodeModify(List<String> nodes, Map<String, String> properties) { … }
public boolean nodePower(List<String> nodes, NodeReportPower state) { … }
public boolean resourceCreate(String type, String id, Map<String, String> attributes) { … }
public boolean systemModify(Map<String, String> properties) { … }
public List<String> systemQuery(List<String> attributes) { … }
public boolean virtualMachineMigrate(String vmId, String hypervisorId, String operationId) { … }

These calls are equivalent to the Moab RM URLs as described in the Native "Native Plugin Interface Comparison" section. All method definitions are documented in the AbstractPlugin API documentation.

6.2.11 Handling Exceptions

There are several specific cases where Exceptions should or can be used:

• The reload method on the Plugin Control Service can throw the InvalidPluginConfigurationException to signify that the configuration contains errors.
• Various methods on the Plugin Control Service throw plugin exceptions which must be caught to diagnose errors when creating plugin types.
• Any exception (including the Exception class) can be thrown from a custom web service to display a 500 Internal Server Error to the client requesting the service with the given error message.

6.2.12 Managing SSL Connections

Several points should be noted when using the SSL Service:

• Certificate files may be in the PEM file format and do not need to be in the DER format (as is typical of Java security).
• Each method returns an instance of SSLSocketFactory, which may then be used to create simple sockets or, in combination with another client library of choice, create a connection.
• If the client certificate password is non-null, it will be used to decrypt the protected client certificate.
• This service is not needed when performing SSL communications with trusted certificates, such as those for HTTPS enabled websites that do not have a self-signed certificate.
• If the file name of the certificate file (client or server) is relative (no leading '/' character), it will be loaded from the mws.certificates.location Configuration parameter.
• The default value of mws.certificates.location is MWS_HOME/etc/ssl.crt.
• Both the client certificate alias and password may be null. In this case, the client certificate must not be encrypted and the client certificate's default alias (the first subject CN) will be used.
• The lenient socket factory and hostname verifier automatically trust all server certificates. Because of this, they present a large security hole. Only use these methods in development or in fully trusted environments.

Example

To create a socket to a server that requires a client certificate, the following code may be used.

package example
import com.ace.mws.plugins.*
class SSLConnectionPlugin extends AbstractPlugin {
	ISslService sslService
	public void poll() {
		// This certificate is not encrypted and will be the only certificate presented to the
		// connecting end of the socket.
		// This file will be loaded from MWS_HOME + mws.certificates.location + my-cert.pem.
		String clientCert = "my-cert.pem"
		def socketFactory = sslService.getSocketFactory(clientCert, null, null)
		def socket = socketFactory.createSocket("hostname.com", 443)
		// Write and read from the socket as desired…
	}
}

To create a HTTPS URL connection to a server that has a self-signed certificate, the following code may be used. Note that this is very typical of client libraries - they have a method to set the SSL socket factory used when creating connections.

package example
import com.ace.mws.plugins.*
class SSLConnectionPlugin extends AbstractPlugin {
	ISslService sslService
	public void poll() {
		// This certificate represents either the server public certificate or the CA's certificate.
		// Since the path is absolute it will not be loaded from the MWS_HOME directory.
		String serverCert = "/etc/ssl/certs/server-cert.pem"
		def socketFactory = sslService.getSocketFactory(serverCert)
		// Open connection to URL
		HttpsURLConnection conn = "https://hostname.com:443/test".toURL().openConnection()
		conn.setSSLSocketFactory(socketFactory)
		// Retrieve page content and do with as desired…
		def pageContent = conn.getInputStream().text
	}
}

6.2.13 Utilizing Services or Custom "Helper" Classes

1. Bundled services such as the Moab Rest Service.
2. Custom built translators loaded by convention of their name.
3. Other custom built helper classes registered with Annotations.

These will each be described in this section.

6.2.13.1 Bundled Services

More information may be found on each bundled service in the Quick Reference section under "Plugin Services". See especially the "Usage" page under "Plugin Services" to understand generally how they are to be used.

6.2.13.2 Using Translators

Simply end any class name with "Translator", and it will be automatically injected just as bundled services onto plugin types, other translators, or even custom registered components. The injection occurs only if a field exists on the class matching the name of the translator with the first letter lower-cased. For example, a translator class called "MyTranslator" would be injected on plugin types, other translators, and custom components that define a field called "myTranslator" as def myTranslator or MyTranslator myTranslator.

Do not use two upper-case letters to start the class name of a Translator. Doing this may cause injection to work improperly. i.e. use RmTranslator instead of RMTranslator as the class name.

Be careful not to declare translator and custom component injection such that a cyclic dependency is created.

Example

Suppose that a translator needs to be created to handle a connection to access an external REST resource. The translator could be defined as follows:

package example
class ExampleTranslator {
	public int getExternalNumber() {
		def number = … // Make call to external resource
		return number
	}
}

A plugin type can then use the translator by defining a field called "exampleTranslator". Note that an instance does not need to be explicitly created.

package example
class ExamplePlugin {
	def exampleTranslator
	// OR …
	//ExampleTranslator exampleTranslator
	public void poll() {
		// Use the translator
		log.info("The current number is "+exampleTranslator.getExternalNumber())
	}
}

To extend the example, the translator may also be injected into another translator:

package example
class AnotherTranslator {
	def exampleTranslator
	public int modifyNumber(int number) {
		return number + exampleTranslator.getExternalNumber()
	}
}

This translator may be used in the plugin type just as the other translator.

6.2.13.3 Registering Custom Components

All annotations are available in the dependencies declared by the plugins-commons artifact.

Do not use two upper-case letters to start the class name of a custom component. Doing this may cause injection to work improperly. i.e. use RmUtility instead of RMUtility as the class name.

Changing Scope

By default, when a custom component is injected, only a single instance is created for all classes which inject it. This is referred to as the 'singleton' scope. Another scope that is available is 'prototype', which creates a new instance every time it is injected. This is useful when the class contains state data or fields that are modified by multiple methods. To change the scope, use the org.springframework.context.annotation.Scope on the class with a single String parameter specifying 'singleton' or 'prototype'.

Injecting Translators or Components

The need may arise to inject translators or other custom components onto custom components. This is done using the org.springframework.beans.factory.annotation.Autowired or javax.annotation.Resource annotations. The Autowired annotation is used to inject class instances by the type (i.e. MyTranslator myTranslator) while the Resource annotation is used to inject class instances by the name (i.e. def myTranslator). Add the desired annotation to the field that needs to be injected.

There is a known issue with dynamically updating plugin types with typed field injection, such as that required when using the Autowired annotation. See the Add or Update Plugin Types section for more information.

Note that using the Autowired annotation does injection by type which differs from translator and plugin type injection. These are done by name just as the Resource annotation allows. Due to this fact, a type of "def" cannot be used when doing injection onto custom components using the Autowired annotation. See the example below.

Injection of custom components onto translators and plugin types are still done by name, only fields injected using the Autowired annotation are affected.

Be careful not to declare translator and custom component injection such that a cyclic dependency is created.

Example

Suppose that a custom utility class is needed to perform complex logic. A custom component could be defined as follows (notice the optional use of the Scope annotation):

package example
import org.springframework.stereotype.Component
import org.springframework.context.annotation.Scope
@Component
@Scope("prototype")
class ComplexLogicHandler {
	def handleLogic() {
		… // Perform complex logic and return
	}
}

A plugin type or translator could then be defined to inject this component:

package example
class CustomPlugin {
	def complexLogicHandler
	public void poll() {
		complexLogicHandler.handleLogic()
	}
}

Now suppose another custom component needs to use the ComplexLogicHandler in its code. It can inject it using the Autowired annotation:

package example
import org.springframework.stereotype.Component
import org.springframework.beans.factory.annotation.Autowired
@Component
class AnotherHandler {
	// Note that this is injected by type, so 'def' may not be used
	@Autowired
	ComplexLogicHandler complexLogicHandler
	def wrapLogic() {
		complexLogicHandler.handleLogic()
	}
}

To perform the same injection but by name (as translators and plugin types are injected), use the Resource annotation:

package example
import org.springframework.stereotype.Component
import javax.annotation.Resource
@Component
class AnotherHandler {
	// Note that this is injected by name based solely on the name defined in
	//	the annotation.  The name of the field itself does not affect the injection.
	@Resource(name="complexLogicHandler")
	def complexLogicHandler
	def wrapLogic() {
		complexLogicHandler.handleLogic()
	}
}

6.2.14 Packaging Plugins

1. A simple Groovy file containing a single plugin type definition.
2. A JAR file containing one or more plugin types, translators, and custom components.

While each may be uploaded to MWS using the REST API or the User Interface as described in Add or Update Plugin Types, using a JAR file is recommended. Using a simple Groovy file is useful for testing and generating proof of concept work, but does not allow the use of several features of plugins.

The principles of packaging a plugin type or set of plugin types in a JAR file are very simple. Simply compile the classes and package in a typical JAR file. All classes ending in "Plugin" are automatically attempted to be loaded as a plugin type, all classes ending in "Translator" are attempted to be loaded as a translator, and all classes annotated as a custom component will be attempted to be loaded. It is recommended that a build framework is used to help with compiling and packaging the JAR file, such as Gradle. This makes it easy to declare a dependency on the necessary JAR files used in plugin development and to debug, compile, and test plugin code.

In addition to using utility services such as translators, packaging plugin types in JAR files allows the creation of a single project for multiple related plugin types and bundling of external dependencies. These two features are discussed in the following sections.

6.2.14.1 Plugin Projects and Metadata

To define a project file, simply add a class to JAR file that ends in "Project". This file will attempted to be loaded as the project file. Every field on a project file, and even the file itself, is optional. All available fields are shown in the example below.

class SampleProject {
	// Plugin information
	String title = "Sample"
	String description = "Sample plugin types"
	String author = "Adaptive Computing Enterprises, Inc."
	String email = "[email protected]"
	// Versioning properties
	String version = "0.1"
	String mwsVersion = "7.1 > *"
	String license = "APACHE"
	// Documentation properties
	String issueManagementLink = "http://example.com/ticket-system/sample-plugins"
	String documentationLink = "http://example.com/docs/sample-plugins"
	String scmLink = "http://example.com/git/sample-plugins"
	// Plugins that are to be created with these properties only when they do NOT exist
	// This does not override any existing plugin instance configuration
	def initialPlugins = {
		/*
		// Multiple instances of plugins may be defined here.
		// In this case, 'sample' is the id of the plugin
		sample {
				pluginType = "Sample"
				// All properties except for "pluginType" are optional
				pollInterval = 30
				autoStart = true
				config {
					configParam = "value"
				}
			}
		}
		// Another plugin with an ID of 'sample2'
		sample2 {
			…
		*/
	}
}

As can be seen, metadata information about the plugin type(s), versions, and documentation are available. These are displayed when viewing plugin information in the User Interface or through the REST API.

Any of these properties except for initialPlugins may be overwritten by the plugin type class itself by using static properties. A simple example is shown below.

package example
class SamplePlugin {
	// Properties may be typed, untyped, final, or otherwise,
	// but they MUST be static
	static version = "0.2"
	static title = "Sample plugin"
	static description = "This sample plugin is used to demonstrate metadata information"
	static author = "Separate Division"
	… // Rest of the plugin type definition
}

Initial Plugins

The initial plugins closure provides the flexibility to insert plugin instances when the JAR is loaded. This occurs at two points: when the plugin JAR is first uploaded to MWS, and when MWS is restarted. As shown in the example above, the ID, pluginType, and other properties may be configured for multiple plugins.

The nature of Groovy closures means that programmatic definition of initial plugins is possible. This may even be based on the MWS application configuration. Two properties are automatically available in the initialPlugins closure:

• appConfig - Contains the MWS application configuration. Any configuration parameter is available for access as documented on the Configuration page.
• suite - Contains the currently configured suite that MWS is running in. This is equivalent to the mws.suite configuration parameter, and is an instance of Suite.

Native Plugin Case Study

The Native JAR file utilizes many of the features discussed above. In the root of the JAR file, a compiled class called NativeProject exists which defines all of the metadata fields, including initialPlugins. Trying to create an initial plugin presents two distinct problems:

• The plugin should only be initialized if the suite is CLOUD.
• The plugin type configuration must contain an entry referencing the configured mws.home.location parameter, or the configured MWS_HOME location.

The initialPlugins closure is defined as follows:

import com.ace.mws.plugins.Suite
class NativeProject {
	… // Metadata fields
	def initialPlugins = {
		// Only initialize the cloud-native plugin if the suite is CLOUD
		if (suite==Suite.CLOUD) {
			'cloud-native' {
				pluginType = "Native"
				pollInterval = 30
				config {
					// Use the appConfig property to retrieve the current MWS_HOME location
					getCluster = "file://${appConfig.mws.home.location}/etc/nodes.txt"
				}
			}
		}
	}
}

6.2.14.2 Managing External Dependencies

1. The plugin type must bundle all external dependency JARs in the root of the plugin type JAR file.
2. An entry must be included in the MANIFEST.MF file that references each of these bundled JAR files as a space separated list:

Class-Path: dependency1.jar dependency2.jar dependency3.jar

Assuming that these rules are followed and that the plugin type is uploaded using the REST API or the User Interface, the dependent JARs will first be loaded and then the new plugin type and associated files will be loaded.

6.2.15 Example Plugin Types

Please see the Plugin Types item in the Quick Reference menu for all bundled plugin types.

A sample plugin type in Groovy would resemble the following:

package sample
import com.ace.mws.plugins.*
class SamplePlugin extends AbstractPlugin {
	static author = "Adaptive Computing"
	static description = "A simple plugin in groovy"
	static version = "0.1"
	INodeRMService nodeRMService
	public void configure() throws InvalidPluginConfigurationException {
		def myConfig = config	// "config" is equivalent to getConfig() in groovy
		def errors = []
		if (!myConfig.arbitraryKey)
			errors << "Missing arbitraryKey!"
		if (errors)
			throw new InvalidPluginConfigurationException(errors)
	}
	public void poll() {
		NodeReport node = new NodeReport("node1")
		node.resources.RES1.total = 5
		node.resources.RES1.available = 5
		node.state = NodeReportState.IDLE
		nodeRMService.save([node])
	}
	// Access at /rest/plugins/<id>/services/example-service
	public def exampleService(Map params) {
		return [success:true]
	}
}

6.3 Plugin Type Management

Plugin Type JAR or groovy files should never be manually copied into the MWS_HOME/plugins directory. They must be managed using the methods shown in this section or through the REST API.

6.3.1 Listing Plugin Types

6.3.2 Displaying Plugin Types

6.3.3 Add or Update Plugin Types

When a plugin type is updated, by default all corresponding plugins created from the plugin type will be recreated. If this behavior is not desired, clear the "Do you want to reload all plugins to use this new version?" checkbox before uploading the plugin type.

There is a known issue when dynamically updating plugin types that use typed fields for injected classes, such as:

MyTranslator myTranslator

instead of

def myTranslator

Change all injections to 'def' in order to work around the issue. Note that this issue makes injection of instances on custom components using the Autowired annotation work improperly since they must be typed. Use the Resource annotation as documented in the Registering Custom Components section instead.

In all cases, and as a last resort, restarting MWS after updating a plugin type resolves the issue.

Single Class File

Groovy files containing a single plugin type may be uploaded at the /mws/admin/plugin-types/create URL.

Simply click Add files..., select the .groovy class file, and click the Start upload button. If the plugin type was successfully uploaded and initialized, the size of the file uploaded will be displayed along with the name of the plugin loaded.

If the upload failed or an error occurred during initialization of the plugin, an error message will be displayed.

JAR File

A JAR file as described in the Packaging Plugins section containing one or more plugins may also be uploaded using the same process as the Groovy file.

Click Add files..., select the .jar file, and click the Start upload button. If the upload failed or an error occurred during initialization of the plugin(s), an error message will be displayed.

The JAR upload process differs from the single file in that if successful, the name of the JAR file itself is displayed instead of the plugin name(s).

Code

To paste or type code directly into MWS and have it be loaded as a single class file, click Type or Paste Code and type or paste the code into the presented text box.

When the code is in the box, click Create. If the upload succeeded and the code was able to be compiled as Groovy, the browser will be redirected to the Show Plugin Type page. If the upload failed or an error occurred during compilation or initialization of the plugin, an error message will be displayed.

The MWS log file file may need to be referred to for additional details and error messages in the case of a failure.

6.4 Plugin Management

6.4.1 Listing Plugins

6.4.2 Creating a Plugin

6.4.3 Displaying a Plugin

6.4.4 Modifying a Plugin

6.4.5 Deleting a Plugin

6.4.6 Monitoring and Lifecycle Controls

If plugins are created from plugin types which do not have a poll method, their lifecycle controls will be limited. Any information below which mentions polling does not apply to the 'no-polling' plugin shown in the screenshots.

Active Plugins

Active plugins are those which are in the Started or Paused states. These are available to receive events such as polling. If paused, a plugin will not receive events but is not actually stopped, therefore no stop events are triggered.

The following images demonstrate the status of plugins in the active states.

Started plugins which can include the relative time of the last poll as well as the time of the next poll in a countdown format. Action buttons are available to stop or pause the plugin as well as trigger an immediate poll event.

Paused plugins which can include only the last polling time. Action buttons are available to stop or resume the plugin, as well as trigger an immediate poll event.

Disabled Plugins

Disabled plugins are those which are in the Stopped or Errored states. These plugins do not receive events such as polling. If errored, a plugin may either be stopped, which represents a "clearing" of the error, or started normally. However, if no action is taken on an errored plugin, it likely will not start due to the fact that most plugins are put into the errored state during startup of the plugin.

The following images demonstrate the representation of plugins in the disabled states.

Stopped plugins. A single action button is available to attempt to start the plugin.

An errored plugin. As mentioned previously, action buttons are available to stop the plugin or clear the error as well as attempt to start the plugin. If the start fails, an error message will be displayed.

6.4.7 Setting Default Plugin Configuration

The parameters to configure are documented on the MWS Configuration page and comprise most values starting with plugins.