(Quick Reference)

5 MWS Plugins (Beta)

5 MWS Plugins (Beta)

This section describes MWS Plugins, their use, and their creation in Moab Web Services.

MWS Plugins are currently in beta. Interfaces may change significantly in future releases.

5.1 Plugin Overview

This section provides an overview of the plugin layer in web services. The following areas will be covered:
  • An introduction to the concept of MWS plugins
  • How to configure Moab Workload Manager to interact with MWS plugins
  • A description of the plugin lifecycle
  • How plugins are driven by events
  • How to expose web services from a plugin
  • How data collisions between plugins are resolved
  • How calls from Moab are routed to MWS plugins

5.1.1 Introduction

Moab Web Services plugins provide a highly extensible interface to interact with Moab, MWS, and external resources. Plugins can perform some of the same functions as Moab Resource Managers, while also providing many other features not available to RMs. This section will discuss the main features of plugins, some basic terminology, and how MWS plugins can interact with Moab.

Features

Plugins can

  • be created, modified, and deleted without restarting Moab or MWS.
  • be defined in Groovy and uploaded to MWS without restarting.
  • have individual data storage space and configuration.
  • be polled at a regular interval (configured on a per-plugin basis)
  • be informed of important system events.
  • be individually stopped, started, paused, and resumed.
  • expose custom web services for external use.
  • be manipulated via a full RESTful API (see Resources for more information).

Terminology

There are two distinct terms in the plugin layer: plugin types and plugins (or plugin instances).

Plugin Types

Plugin Types can be considered plugin templates with built-in logic. In object-oriented programming languages, this relates to the concept of a class. They possess certain abilities, or methods, that can be called by Moab Web Services to query information about a certain resource. They also can define methods which will be exposed to external clients as web services. They do not contain any configuration or current data, but they are often tied to a type of component, such as components that communicate with Moab's WIKI Protocol or those that are built on a certain product.

They define several types of methods:

  1. Query methods such as getNodes, getVirtualMachines, and getNodes that retrieve the current state of the resources that the plugin monitors.
  2. The poll method (optional) that is called at a configured interval.
  3. Instance methods that return information about the current plugin, such as getState. While these are defined in the plugin type, the plugin type itself does not have a state.
  4. Lifecycle methods of plugins created from the plugin type, such as beforeStart and afterStart.
  5. Web service methods that expose custom functionality as public web services.

Some examples of plugin types include the Native plugin type, the MSM plugin type, and the CSA plugin type.

Plugins (Instances)

Plugins (also called plugin instances) are created from plugin types. They contain current data or configuration and use the plugin type methods to interact with resources.

Interactions with Moab as a Resource Manager

The plugin layer in MWS is integrated with Moab via the Native Resource Manager (RM) interface. When utilizing plugins, MWS is configured as a RM in Moab as explained in the next section. Events from Moab are pushed through the RM interface to MWS which is then pushed to each plugin in turn. The relationship between Moab Web Services, Moab, and plugins is shown in the following image:

In the diagram above, the MWS RM signifies that MWS is configured as a Moab Resource Manager.

5.1.2 Configuring Moab

To use the full functionality of MWS plugins, Moab must be configured to use MWS as a resource manager. The following lines must be in the /opt/moab/etc/moab.cfg file or one of its included files:

RMCFG[mws]              TYPE=NATIVE
RMCFG[mws]              FLAGS=UserSpaceIsSeparate
RMCFG[mws]              CLUSTERQUERYURL=exec://$TOOLSDIR/mws/cluster.query.mws.pl
RMCFG[mws]              WORKLOADQUERYURL=exec://$TOOLSDIR/mws/workload.query.mws.pl
RMCFG[mws]              JOBCANCELURL=exec://$TOOLSDIR/mws/job.cancel.mws.pl
RMCFG[mws]              JOBMIGRATEURL=exec://$TOOLSDIR/mws/vm.migrate.mws.pl
RMCFG[mws]              JOBMODIFYURL=exec://$TOOLSDIR/mws/job.modify.mws.pl
RMCFG[mws]              JOBREQUEUEURL=exec://$TOOLSDIR/mws/job.requeue.mws.pl
RMCFG[mws]              JOBRESUMEURL=exec://$TOOLSDIR/mws/job.resume.mws.pl
RMCFG[mws]              JOBSTARTURL=exec://$TOOLSDIR/mws/job.start.mws.pl
RMCFG[mws]              JOBSUBMITURL=exec://$TOOLSDIR/mws/job.submit.mws.pl
RMCFG[mws]              JOBSUSPENDURL=exec://$TOOLSDIR/mws/job.suspend.mws.pl
RMCFG[mws]              NODEMODIFYURL=exec://$TOOLSDIR/mws/node.modify.mws.pl
RMCFG[mws]              NODEPOWERURL=exec://$TOOLSDIR/mws/node.power.mws.pl
RMCFG[mws]              RESOURCECREATEURL=exec://$TOOLSDIR/mws/resource.create.mws.pl
RMCFG[mws]              SYSTEMMODIFYURL=exec://$TOOLSDIR/mws/system.modify.mws.pl
RMCFG[mws]              SYSTEMQUERYURL=exec://$TOOLSDIR/mws/system.query.mws.pl

The next step is to edit the MWS values in /opt/moab/etc/cloud.cfg. Here are the default values:

CONFIG[default]                 MWS_URL=http://localhost:8080/mws
CONFIG[default]                 MWS_USERNAME=admin
CONFIG[default]                 MWS_PASSWORD=adminpw

MWS_USERNAME and MWS_PASSWORD must match the values of auth.defaultUser.username and auth.defaultUser.password, respectively, found in /opt/mws/etc/mws-config.groovy.

The *.mws.pl scripts should be located in the tools/mws of the Moab home directory. The Moab/WebServices.pm module must also be available to the scripts. All of these files may be found in the tools/mws and lib/perl5 directories of the Moab tar file. They are automatically installed if Moab is configured with the --with-mws flag or they can be copied directly from there to the tools folder in your Moab home directory.

To enable such actions as submitting jobs as different users, the ENABLEPROXY=TRUE option must be present in the ADMINCFG configuration line and the OSCREDLOOKUP option must be set to NEVER as follows:

ADMINCFG[1]           USERS=root      ENABLEPROXY=TRUE
OSCREDLOOKUP          NEVER

5.1.3 Lifecycle States

During the course of a plugin's use, the state of the plugin may change many times. Plugins have four possible states: Stopped, Started, Paused, and Errored. The flow of a plugin through the states is shown in the following image:

To see descriptions of each state, see the PluginState API.

Events that occur during lifecycle state changes may be found in the Events section.

5.1.4 Events

Plugins use an event based model in that methods are called on the plugin when certain criteria are met or situations arise. Plugin types may be created to handle certain events by implementing or not implementing certain methods. Events currently exist for polling and certain lifecycle state changes.

The Polling Event

To maintain current information, each plugin is polled for node, job, and virtual machine information at a specified time interval. By default, this interval is set to 30 seconds, but can be modified for all or individual plugins as explained in Plugin Management.

When a polling event occurs, the poll method on the target plugin is called. This method may perform any function desired and should typically make calls to the Plugin Persistence Service to make updated to nodes, jobs, or virtual machines. For example, the poll method in the Native plugin type is implemented as follows:

This is an extremely simplified version of what is actually implemented in the Native plugin type.

public void poll() {
	getPluginPersistenceService().updateNodes(getNodes());
	getPluginPersistenceService().updateVirtualMachines(getVirtualMachines());
	getPluginPersistenceService().updateJobs(getJobs());
}

This simple poll method calls three other helper methods called getNodes, getVirtualMachines, and getJobs to retrieve node, job, and virtual machine objects. These results are each sent to the appropriate method in the plugin persistence service. While the specific details of the plugin persistence service are not important to understand at this point, the objective of this example is to demonstrate one possible use of the poll event handler. The CSA plugin type, on the other hand, uses the poll event to retrieve update internal data from its pertinent resources and to update node and virtual machine information. It does not query or persist any job information.

Lifecycle Events

Events are also triggered for certain lifecycle state changes. These are documented in the table below with the associated method that must be implemented on a plugin type to handle the event.

State ChangeEventDescription
StartbeforeStartTriggered just before starting a plugin.
StartafterStartTriggered just after a plugin has been started.
StopbeforeStopTriggered just before stopping a plugin.
StopafterStopTriggered just after stopping a plugin.

Currently, no events are triggered for pausing, resuming, erroring, or clearing errors for plugins.

5.1.5 Custom Web Services

Although the events interface typically serves most cases, there are some instances where an event is not supported that is desired. This is especially true when an external resource is the source of the event. To address these issues, plugins can expose custom web services to external resources. These web services may be named freely and do anything they wish in the plugin framework.

For example, suppose a resource needs to notify a plugin that provisioning of a virtual machine has been completed. Instead of having the plugin poll the resource to verify that the provisioning was finished, the plugin could expose a custom web service to handle notification from the resource itself.

Sample custom web service
def vmProvisionFinished(Map params) {
	// Handle event
	return [messages:["Event successfully processed"]]
}

A full explanation of the syntax and creation of custom web services may be seen on the Plugin Type Guidelines page under "Exposing Web Services".

For information how resources can access plugin web services, see Accessing Plugin Web Services.

5.1.6 Data Collision Detection

At times, plugins can report differing or even contradictory data for nodes, jobs, and virtual machines. This is called a data "collision". Currently, when data from one plugin "collides" with another, the last plugin to report (or persist using the plugin persistence service) the data will be considered the authoritative source for information.

For example, suppose two plugins exist, pluginA and pluginB. These plugins both report data for a node with an ID of node1. However, each reports a different node power state. Plugin A reports the power as ON, while plugin B reports the power as OFF. The data collision that occurs due to these two plugins persistence contradictory data is resolved by the timing of their polling. If plugin A is polled first and plugin B second, the node will be reported as OFF until plugin A is polled again and vice versa.

The simple workaround for this issue is to ensure that no two plugins report the same resource or that they report different properties of the same resource. For example, if plugin A only modified the power state and plugin B only modified the available disk, these two plugins would work in harmony to provide a consistent view of the node resource.

5.1.7 Routing

Due to the fact that Moab Web Services is configured as a Resource Manager (RM) in Moab Workload Manager, events are sometimes triggered by Moab through the RM interface. These actions could be migrating a virtual machine, starting a job, submitting a job, modifying a node, and so forth. The decisions of which plugins are affected and notified is termed routing .

Currently all plugins receive all commands from Moab. This means that each plugin will receive the command to start a job if sent from Moab, even if that plugin does not handle the job. This means that plugins must ensure they handle only actions or commands for resources which they report or handle.

5.2 Plugin Type Management

Plugin types comprise the methods by which Moab may communicate with resource managers or other external components. They define all operations that can be performed for a "type" or "class" of plugins.

Several plugin types are provided with web services, but it is easy to create additional plugin types and add their functionality to web services.

5.2.1 Bundled Plugin Types

Several plugin types are provided by Adaptive Computing for use in Moab Web Services. Examples of these include the Native and MSM plugin types.

Please see the Bundled Plugin Types item in the Quick Reference menu for all bundled types.

5.2.2 Creating Plugin Types

Creating a plugin type involves using Groovy, which is based on the Java programming language. This section describes the general guidelines and specifics of implementing a simple plugin.

5.2.2.1 Plugin Type Guidelines

The com.ace.mws.plugins.AbstractPlugin abstract class is provided to assist in creating plugin types. However, this class need not be extended to provide a fully functional plugin type. In fact, there are only two methods that must be implemented to provide a working plugin type:
  • public String getId();
  • public void setId(String id);

These may be stored in whichever way desired, but will most likely be implemented as follows:

In the following Groovy example, String id will be expanded by the compiler to the full method definitions given above. Thus no explicit method definitions are actually needed.

Basic Groovy Implementation
class BasicPlugin {
	String id
}

To pass the checks to be able to add the class as a plugin, there are two requirements:

  1. The ID getter and setter must be fully implemented (as described above).
  2. The class name must end in "Plugin".

Dynamic Methods on Plugins

Several methods are dynamically inserted onto each plugin. These methods do not need to be included in the plugin class, and in fact are preferred not to as they will simply be overwritten.

These methods are shown below:

// Defined in com.ace.mws.plugins.AbstractPlugin
public void start() throws PluginStartException;	// Equivalent to [pluginControlService.start(String id)|guide:pluginControlLifecycle]
public void stop() throws PluginStopException;  // Equivalent to [pluginControlService.stop(String id)|guide:pluginControlLifecycle]

// Defined in com.ace.mws.plugins.AbstractPluginInfo public String getPluginType(); // Equivalent to [pluginConfigurationService.getPluginType(String id)|guide:pluginConfigurationService] public PluginState getState(); // Equivalent to [pluginConfigurationService.getState(String id)|guide:pluginConfigurationService] public Integer getPollInterval(); // Equivalent to [pluginConfigurationService.getPollInterval(String id)|guide:pluginConfigurationService] public Boolean getAutoStart(); // Equivalent to [pluginConfigurationService.getAutoStart(String id)|guide:pluginConfigurationService] public Map<String, Object> getConfig(); // Equivalent to [pluginConfigurationService.getConfig(String id)|guide:pluginConfigurationService]

Plugin Metadata

Metadata may be included in plugin classes by defining static properties on the classes. Currently, the metadata available is author and description. These may be defined in the following manner:

The following example does not implement the ID property and therefore would not pass as a valid plugin.
Groovy plugin with Metadata
class ExamplePlugin {
	static author = "Adaptive Computing"
	static description = "A basic example for a plugin with metadata"
}

Exposing Web Services

Any number of methods may be exposed as public web services by following two simple rules:

  1. The method must return a list, map, or a complex object.
  2. It must define a single argument of a Map.

The argument will contain all parameters passed into the web service by the client. See Accessing Plugin Services for additional details.

Parameters may be passed into the web service call as normal URL parameters such as ?param=value&param2=value2, as key-value pairs in the POST body of a request, or as JSON in the body. For the first two cases, the parameters will be available on the Map argument passed into the web service call as key value pairs matching those of the request. Note that in these cases all keys and values will be interpreted as strings.

GET PLUGIN_SERVICE_URL?key=value&key2=true&key3=5

def serviceMethod(Map params) { assert params.key=="value" assert params.key2=="true" assert params.key3=="5" }

In the latter case, the parsed JSON properties will be available within a parameter called body in the Map argument. In this scenario, the types of the values are preserved by the JSON format.

POST PLUGIN_SERVICE_URL with JSON body of
{"key":"value","key2":true,"key3":5}

def serviceMethod(Map params) { assert params.body.key=="value" assert params.body.key2==true assert params.body.key3==5 }

Events

For events that trigger method calls on plugins, these methods may be implemented on custom plugin types to handle the event. For more information, see the Plugin Events section.

External Dependencies

External dependencies (e.g. JAR files) may be included and referenced in custom plugin types. However, certain rules must be followed in order to have these load correctly:

  1. The plugin type must be bundled and uploaded as a JAR file.
  2. The plugin type must bundle all external dependency JARs in the root of the plugin type JAR file.
  3. An entry must be included in the MANIFEST.MF file that references each of these bundled JAR files as a space separated list:

Class-Path: dependency1.jar dependency2.jar dependency3.jar

Assuming that these rules are followed, and that the plugin type is uploaded using the REST API or the User Interface, the dependent JARs will first be loaded and then the new plugin type and associated files will be loaded.

5.2.2.2 API Classes and Interfaces

There are several packages and classes available to assist in creating plugin types. These can all be found in the API documentation under the com.ace.mws.plugins package.

Here is a brief synopsis of the classes that can and should be used:

Interfaces

The com.ace.mws.plugins package contains the interfaces AbstractPluginInfo and AbstractPlugin that should form the basis of any new plugin type.

Only the getId() and setId() functions must be implemented for a fully operational plugin. All other methods will be inserted dynamically if they do not exist on startup.

Services

The com.ace.mws.plugins.services package contains interfaces for all services available to plugin types. These may be used as discussed in Services.

Exceptions

The com.ace.mws.plugins.exceptions package contains several exceptions that may be used and in some cases, should be caught.

5.2.2.3 Plugin Type Example

A sample plugin type in Groovy would resemble the following:

package test

import com.ace.mws.plugins.* import com.ace.mws.plugins.exceptions.*

class UploadTestPlugin { static author = "Adaptive Computing" static description = "A simple plugin in groovy"

String id

public void verifyConfiguration() throws InvalidPluginConfigurationException { def myConfig = config def errors = [] if (!myConfig.arbitraryKey) errors << "Missing arbitraryKey!" if (errors) throw new InvalidPluginConfigurationException("Invalid plugin ${id} configuration", errors) } }

5.2.3 Plugin Services

Several services are available for use by any plugin type. To use services, they must be declared within the class of the plugin type. For example, to use the plugin control service, a pluginControlService property of type IPluginControlService or "def" must be declared on the plugin type. The actual service will be inserted or injected into the plugin class when the plugin is used.

Injected typed service
package example

import com.ace.mws.plugins.services.IPluginControlService

public class ExamplePlugin { IPluginControlService pluginControlService

public void someMethod() { // Use the control service pluginControlService.[method](); } }

Injected untyped service
package example

public class ExamplePlugin { def pluginControlService

public void someMethod() { // Use the control service pluginControlService.[method](); } }

Do not attempt to create a new instance of the services before use, such as in a constructor. The services will be automatically injected before any methods are called on the plugin.

The injected service property must be named correctly to use it, regardless of the type used.

5.2.3.1 Configuration Service

The configuration service controls all configuration options for plugins. Typically this service does not need to be called directly as methods are provided on all plugins which are routed to the configuration service as explained in the guidelines under Dynamic Methods.

The pluginConfigurationService property will be injected with a class of type IPluginConfigurationService.

5.2.3.2 Control Service

The control service allows lifecycle management operations to be performed on plugins. It also provides methods to create and retrieve plugins. Note that the plugin control service may be used by other plugins, allowing one plugin to dynamically create, retrieve, start, or stop plugins. The CSA plugin does exactly this by creating a new plugin (SA for example) for each supported provider in CSA.

The pluginControlService property will be injected with a class of type IPluginControlService.

Creating Plugins

Several methods are provided to allow on-the-fly creation of new plugins. Generally, they allow a plugin with a specific ID and plugin type (as a string or as a Groovy Class) to be created with optional configuration properties. These properties should match the fields in the Plugin API. If specific or all configuration properties are omitted, the defaults will be used as described in the Plugin Management with Configuration file section.

In each case, a boolean value is returned indicating whether the creation succeeded or not. Additionally, the createPlugin methods will initialize the plugin for retrieval or usage and attempt to start the plugin if the autoStart property is true.

Create plugin with default configuration
try {
	if (pluginControlService.createPlugin("myPlugin", "Native"))
		println "myPlugin was created successfully!"
	else
		println "There was an error creating myPlugin"
} catch(PluginStartException e) {
	println "There was a problem starting the new plugin: ${e.message}"
} catch(InvalidPluginConfigurationException e) {
	println "There were errors with the plugin's configuration: ${e.errors}"
}

Create plugin with custom configuration
if (pluginControlService.createPlugin("myPlugin", "Native", [autoStart:false, pollInterval:600]))
	println "myPlugin was created successfully!"
else
	println "There was an error creating myPlugin"

Retrieving Plugins

Retrieving plugins requires either a unique identifier or the type and configuration option(s).

Retrieving by Unique Identifier

Get plugin by ID
IPlugin plugin = pluginControlService.getPluginById("plugin1");

Retrieving by Type and Configuration Properties

The second method of retrieving Plugins involves sending a type and configuration properties as a map. Both parameters are required; however, the configuration map may be empty as in the following example.

Get plugin by Type Only
Map<String, String> config = new HashMap<String, String>();
IPlugin plugin = pluginControlService.getPlugin("Native", config);

In this case, the first plugin with a type of Native will be returned. If no Plugins of this type exist, null is returned.

If the configuration properties map is filled with any properties, all keys and values in it must be matched for a plugin to be successfully retrieved. For example, if the current plugin list looks like the following:

test {
	pluginType = "Native"
	config = [test:"true"]
}
test2 {
	pluginType = "Native"
	config = [test2:"true"]
}

Then the following calls would result:

IPlugin plugin;
Map<String, String> config = new HashMap<String, String>();

config.put("test", "true"); plugin = pluginControlService.getPlugin("Native", config); assert "test"==plugin.getId();

config.put("test2", "true"); plugin = pluginControlService.getPlugin("Native", config); assert plugin==null;

config.remove("test"); plugin = pluginControlService.getPlugin("Native", config); assert "test2"==plugin.getId();

Starting or Stopping Plugins

Plugins may be started or stopped on demand. These two methods are exposed directly as start and stop on the plugin control service. Although each method does not return any data, exceptions are thrown if errors are encountered.

These methods correctly handle lifecycle events and changing plugin state. These should never be modified directly!

Start Plugin
try {
	pluginControlService.start("myPlugin")
} catch(PluginStartException e) {
	println "There was a problem starting the plugin: ${e.message}"
} catch(InvalidPluginException) {
	println "The plugin 'myPlugin' is invalid"
} catch(InvalidPluginConfigurationException e) {
	println "The plugin has an invalid configuration: ${e.errors}"
}

Stop Plugin
try {
	pluginControlService.stop("myPlugin")
} catch(PluginStopException e) {
	println "There was a problem stopping the plugin: ${e.message}"
} catch(InvalidPluginException) {
	println "The plugin 'myPlugin' is invalid"
}

Verifying Plugin Configuration

Finally, the plugin control service may be used to verify plugin configuration at any point instead of just when the plugin is started or modified. This may be useful to attempt to modify plugin configuration directly through the Configuration Service and then verify that the new configuration is valid for the plugin. Exceptions are thrown if the plugin or the configuration is invalid.

Verify plugin configuration
try {
	pluginControlService.verifyConfiguration("myPlugin")
} catch(InvalidPluginException) {
	println "The plugin 'myPlugin' is invalid"
} catch(InvalidPluginConfigurationException e) {
	println "The plugin has an invalid configuration: ${e.errors}"
}

5.2.3.3 Data Persistence Service

The data persistence service is provided to ease the storage of Moab state data such as nodes, jobs, and virtual machines. Objects passed to the service are saved to the Moab Web Services database. It also handles data collisions as explained in the Overview.

If the plugin uses the getNodes, getJobs, or getVirtualMachines methods exclusively for handling polling, the service will likely never be used directly. This is due to the fact that the default AbstractPlugin implementation of the poll method uses the persistence service with the results from these methods. The persistence service, however, is used in all plugins that persist job, node, or virtual machine data.

The pluginPersistenceService property will be injected with a class of type IPluginPersistenceService.

All examples use a custom web service to create events.

Note that in all cases, the Node, Job, and VirtualMachine objects are intentionally not saved before being passed to the persistence service.

Persisting Data to the Database

In this most typical use case of the persistence service, it may be used to persist node, job, and virtual machine data to the database.

Persisting Nodes

Persisting Nodes in Groovy
package example

import com.ace.mws.nodes.Node

public class ExamplePlugin { def pluginPersistenceService

public def updateNodesService(Map params) { def nodes = ...// create Node objects here if (pluginPersistenceService.updateNodes(nodes)) log.info("Nodes successfully updated") else log.info("There was an error updating nodes") } }

Persisting Jobs

Persisting Jobs in Groovy
package example

import com.ace.mws.jobs.Job

public class ExamplePlugin { def pluginPersistenceService

public def updateJobsService(Map params) { def jobs = ...// create Job objects here if (pluginPersistenceService.updateJobs(jobs)) log.info("Jobs successfully updated") else log.info("There was an error updating jobs") } }

Persisting Virtual Machines

Persisting Virtual Machines in Groovy
package example

import com.ace.mws.vms.VirtualMachine

public class ExamplePlugin { def pluginPersistenceService

public def updateVirtualMachinesService(Map params) { def vms = ...// create Virtual Machine objects here if (pluginPersistenceService.updateVirtualMachines(vms)) log.info("VMs successfully updated") else log.info("There was an error updating VMs") } }

Removing Data from the Database

On the other hand, the plugin persistence service may also be used to remove state data from the database by using the remove* methods.

Removing Nodes

Removing Nodes in Groovy
package example

import com.ace.mws.nodes.Node

public class ExamplePlugin { def pluginPersistenceService

public def removeNodesService(Map params) { def nodes = ...// load Node objects here

if (pluginPersistenceService.removeNodes(nodes)) log.info("Nodes successfully removed") else log.info("There was an error removing nodes") } }

Removing Jobs

Removing Jobs in Groovy
package example

import com.ace.mws.jobs.Job

public class ExamplePlugin { def pluginPersistenceService

public def removeJobsService(Map params) { def jobs = ...// load Job objects here

if (pluginPersistenceService.removeJobs(jobs)) log.info("Jobs successfully removed") else log.info("There was an error removing jobs") } }

Removing Virtual Machines

Removing Virtual Machines in Groovy
package example

import com.ace.mws.vms.VirtualMachine

public class ExamplePlugin { def pluginPersistenceService

public def removeVirtualMachinesService(Map params) { def vms = ...// load VirtualMachine objects here

if (pluginPersistenceService.removeVirtualMachines(vms)) log.info("Virtual machines successfully removed") else log.info("There was an error removing virtual machines") } }

5.2.3.4 Individual Datastore Service

The individual datastore service is provided to allow a plugin to persist data to the database that is isolated from all other persistent data. It is not designed to store Moab data such as nodes, jobs, or virtual machines, but custom, arbitrary data pertinent only to the individual plugin.

The pluginDatastoreService property will be injected with a class of type IPluginDatastoreService.

Persisting Custom Data

The datastore service may be used to persist custom, arbitrary data to the database. Multiple collections may be used by a single plugin and can be named arbitrarily. Although non-alphanumeric characters may be used, it is not recommended as it could cause loss of data between collections.

Always use the id of the current plugin when calling the pluginDatastoreService methods. Failure to do so will cause issues with other plugins.

Adding A Single Entry

Persisting Custom Entry in Groovy
package example

public class ExamplePlugin { def pluginDatastoreService

public def addDataEntryService(Map params) { def collectionName = "collection1" def data = [:] // Add data here to the Map if (pluginDatastoreService.addData(id, collectionName, data)) log.info("Data successfully added") else log.info("There was an error adding the data") } }

Adding Multiple Entries

Persisting Multiple Entries in Groovy
package example

public class ExamplePlugin { def pluginDatastoreService

public def addDataEntriesService(Map params) { def collectionName = "collection1" def dataList = [] dataList.add( /* Custom Map of data here */) dataList << // Custom Map of data here if (pluginDatastoreService.addData(id, collectionName, dataList)) log.info("Data entries successfully added") else log.info("There was an error adding the data entries") } }

Updating A Single Entry

Updating Entry in Groovy
package example

public class ExamplePlugin { def pluginDatastoreService

public def updateDataEntryService(Map params) { def collectionName = "collection1" def data = [:] // Add data here to the Map if (pluginDatastoreService.updateData(id, collectionName, "key", "value", data)) log.info("Data successfully updated") else log.info("There was an error updating the data") } }

Querying Data

The datastore service may also be used to query for collections and specific entries in each collection.

Find If A Collection Exists

Collection Exists in Groovy
package example

public class ExamplePlugin { def pluginDatastoreService

public def addDataEntryService(Map params) { def collectionName = "collection1" if (pluginDatastoreService.exists(id, collectionName)) log.info("Collection exists") else log.info("The collection does not exist") } }

Get Contents Of A Collection

Get Collection in Groovy
package example

public class ExamplePlugin { def pluginDatastoreService

public def addDataEntriesService(Map params) { def collectionName = "collection1" def dataList = pluginDatastoreService.getCollection(id, collectionName) if (dataList!=null) log.info("Collection successfully queried") else log.info("There was an error querying the collection") } }

Get A Single Entry From A Collection

Get Single Entry in Groovy
package example

public class ExamplePlugin { def pluginDatastoreService

public def updateDataEntryService(Map params) { def collectionName = "collection1" def data = pluginDatastoreService.getData(id, collectionName, "key", "value") if (data!=null) log.info("Data successfully retrieved") else log.info("There was an error retrieving the data") } }

Removing Data

The data in the individual datastore may also be cleared out or removed on a collection or single entry basis.

Removing A Collection

Removing Collection in Groovy
package example

public class ExamplePlugin { def pluginDatastoreService

public def addDataEntryService(Map params) { def collectionName = "collection1" def data = pluginDatastoreService.clearCollection(id, collectionName) // Data now contains the collection that was cleared if (data!=null) log.info("Collection successfully cleared") else log.info("There was an error clearing the collection") } }

Removing A Single Entry

Remove Single Entry in Groovy
package example

public class ExamplePlugin { def pluginDatastoreService

public def addDataEntriesService(Map params) { def collectionName = "collection1" if (pluginDatastoreService.removeData(id, collectionName, "key", "value")) log.info("Data entry successfully removed") else log.info("There was an error removing the data entry") } }

5.2.4 Uploading Plugin Types

Plugin types can be uploaded into Moab Web Services using the user interface or REST API.

5.2.4.1 Upload with the User Interface

The user interface may be used to upload plugins using a file, a Java Archive (JAR) file, or pasted Groovy code.

Single Class File

Groovy files containing a single plugin class may be uploaded at the /mws/admin/plugin-types/create URL.

Simply click Add files..., select the .groovy class file, and click the Start upload button. If the plugin type was successfully uploaded and initialized, the size of the file uploaded will be displayed.

If the upload failed or an error occurred during initialization of the plugin, an error message will be displayed. See the log file for additional details and error messages.

JAR File

A JAR file containing one or more plugins may also be uploaded using the same process as the Groovy file.

Navigate to the /mws/admin/plugin-types/create URL. Click Add files..., select the .jar class file, and click the Start upload button. If the upload failed or an error occurred during initialization of the plugin(s), an error message will be displayed. See the log file for additional details and error messages.

The JAR upload process differs from the single file in that if successful, the name of each successfully loaded plugin class will be displayed.

There are two ways that the plugins are extracted from the JAR file: the manifest file and autodetection.

Manifest File

The manifest file, located at META-INF/MANIFEST.MF, will be loaded and an attribute named MWS-Plugin-Types will be used. This attribute's value should be a comma-separated list of full class names of all plugin types, including the package.

example.jar/META-INF/MANIFEST.MF Example
Manifest-Version: 1.0
MWS-Plugins: example.package.ExamplePlugin, example.package.AnotherExamplePlugin

Autodetect

If no manifest attribute is specified, or the manifest file does not exist, then MWS will search in the JAR for file names that end with Plugin. If it finds any, it will attempt to load them as plugin classes.

Code

Code may also be written dynamically in the browser which is then uploaded and compiled as Groovy code. Make sure to refer to the plugin type Guidelines before finishing the upload process.

Navigate to the /mws/admin/plugin-types/create URL and click Paste Source Code to open the text area where code should be placed.

Paste or type the code into the field and click Create. If the upload succeeded, the user interface will be redirected to the plugin type show page. If the upload failed or an error occurred during initialization of the plugin, an error message will be displayed. See the log file for additional details and error messages.

5.2.4.2 Uploading with REST API

Alternatively, the same file formats may be uploaded to Moab Web Services using a REST API. The URLs, payloads, and responses are fully documented in the Updating Plugin Types section.

When using the REST API, the code and single Class files use the same operation.

5.2.5 Listing Plugin Types

Finally, it is possible to list the available plugin types with their associated authors and descriptions through either the REST API or the user interface.

Listing in REST API

Retrieving all or specific plugin types is fully documented in the Getting Plugin Types resource section.

Listing in User Interface

To retrieve a list of all plugin types, navigate to /mws/admin/plugin-types/list.

The ID of each plugin type may be clicked to navigate to a page with more information concerning the type, including the current instances using it. A link is also provided to create a new plugin from the currently displayed type.

5.3 Plugin Management and Usage

Plugins may be managed and accessed with Moab Web Services dynamically, even while running. This includes plugin instance configuration, controlling plugin lifecycle, and accessing custom web services.

5.3.1 Configuring Plugins

Configuring plugins may be done by any of these methods:
  1. Using the MWS configuration file which is read during the MWS startup process.
  2. Using the user interface through a web browser.
  3. Using the REST API through scripts or other web client utilities.

5.3.1.1 Managing with Configuration File

Only new plugins (those with IDs that do not exist in the database) are loaded on startup. The database is considered the authoritative source for all current plugin configuration.

Configuration of plugins with a file involves setting the Configuration fields in the Moab Web Services configuration file. See the MWS configuration guide for more information on the configuration file.

Two areas can be configured within the file: default values and plugin configuration.

Changing Default Values

Configuration may be specified for default values for all new plugins as follows:

All settings are optional for the default configuration. If no values are specified, the default values will be used as shown in the Configuration reference guide.

Default plugin configuration
plugins {
	pollInterval = 30
	pollEnabled = true
	autoStart = true
	config {
		arbitraryKey = "arbitrary value"
		username = "admin"
		password = "pass"
	}
}

With these settings, any new plugins would be created with polling enabled, auto start enabled, a polling interval of 30 seconds, and three config entries.

Additionally, the pluginType and id fields may be given a default value that will be used in the User Interface for creating new plugins as follows:

Setting UI Defaults
plugins {
	id = "companyId00"
	pluginType = "Native"
}

Instance Configuration

New plugins can be created by using the configuration file. Please note, however, that if a plugin already exists in the database with the same ID when the configuration file is read, the configuration file settings will be ignored. In other words, the database data is taken over all configuration file data.

To define plugins, simply include an instances block in the configuration file. Each new block within instances is the ID of a new plugin and contains all desired configuration for it.

Sample plugin 'native1'
plugins {
	instances {
		native1 {
			pluginType = "Native"
			pollInterval = 25
			autoStart = false
		}
	}
}

It should be reiterated that all configuration entries for a plugin, excluding the id and pluginType, are optional.

Default vs Instance Config

Any config entries defined in the instances block will be merged with the default config entries with the plugin entries taking precedence. For example, for the following configuration:

Config Entries
plugins {
	config {
		key = "defaultKey"
		defaultKey = "defaultValue"
	}
	instances {
		native1 {
			pluginType = "Native"
			config {
				key = "pluginKey"
				pluginKey = "pluginValue"
			}
		}
	}
}

A plugin would be configured with a combined configuration of:

config {
	key = "pluginKey"
	defaultKey = "defaultValue"
	pluginKey = "pluginValue"
}

5.3.1.2 Managing with User Interface

Plugins may be listed, created, modified, and deleted by navigating to /mws/admin/plugins.

New plugins may be created by navigating to /mws/admin/plugins/create. This interface exposes the same configuration options that are in the External File configuration. The same validation occurs through the user interface for required and optional fields.

5.3.1.3 Managing with REST API

The URLs, payloads, and responses of managing plugins through the REST API are fully documented in the Plugins Resource sections.

5.3.2 Controlling Plugin Lifecycle

Monitoring and lifecycle control of plugins may be performed on a single page located at /mws/admin/plugins/control/list. This page displays the current state of all plugins as well as their polling status.

Active Plugins

Active plugins are those which are in the Started or Paused states. These are available to receive events such as polling. If paused, a plugin will not receive events but is not actually stopped, therefore no stop events are triggered.

The following images demonstrate the status of plugins in the active states.

A started plugin which includes the relative time of the last poll as well as the time of the next poll in a countdown format. Action buttons are available to stop or pause the plugin as well as trigger an immediate poll event.

A paused plugin which includes only the last polling time. Action buttons are available to stop or resume the plugin, as well as trigger an immediate poll event.

Disabled Plugins

Disabled plugins are those which are in the Stopped or Errored states. These plugins do not receive events such as polling. If errored, a plugin may either be stopped, which represents a "clearing" of the error, or started normally. However, if no action is taken on an errored plugin, it likely will not start due to the fact that most plugins are put into the errored state during startup of the plugin.

The following images demonstrate the representation of plugins in the disabled states.

A stopped plugin. A single action button is available to attempt to start the plugin.

An errored plugin. As mentioned previously, action buttons are available to stop the plugin or clear the error as well as attempt to start the plugin. If the start fails, an error message will be displayed.

5.3.3 Accessing Web Services

As mentioned in the Overview, custom web services may be available in plugins. These web services may be called externally by resources and arbitrary consumers or internally by other plugins.

Access Web Services Externally

To access the custom web services defined by the plugin, navigate to or call /mws/rest/plugins/ ID /services/ SERVICE_METHOD where ID is the unique identifier for the plugin, and SERVICE_METHOD is the method name of the exposed service.

Parameters may be passed into the web service call as normal URL parameters such as ?param=value&param2=value2, in the POST body of a request, or as JSON in the body.

Additionally, translation is done to map CamelCase service names to dash-separated names in the URL. For example, a web service method named notifyEvent on a plugin with an ID of notifications can be called with the following URLs.

// Camel case
/mws/rest/plugins/notifications/services/notifyEvent
// Dash separated
/mws/rest/plugins/notifications/services/notify-event

Web Service Calls from Internal Plugins

In some cases, it may be desirable to access the custom web services from another plugin internally. To do so, simply retrieve the plugin using the plugin control service and call the desired method directly.

For example, if a plugin exists with an ID of "yourPlugin", and another plugin identified as "myPlugin" wants to access a custom web service defined as the following:

yourPlugin web service
def notifyEvent(Map params) {
	// Handling of the event
	return [processed:true]
}

The plugin "myPlugin" would simply retrieve "yourPlugin" using the plugin control service and call the method. The return value can be used directly without any translation to or from JSON.

Call plugin's custom service
IPluginControlService pluginControlService

void poll() { // This plugin is "myPlugin" assert id=="myPlugin"

// Retrieve "yourPlugin" def yourPlugin = pluginControlService.getPluginById("yourPlugin") assert yourPlugin.id=="yourPlugin"

// Call custom web service internally def result = yourPlugin.notifyEvent([:]) assert result.processed==true }