Conventions

21.1 Policy-based VM Migration

One of the unique features of Moab Adaptive Computing Suite is policy-based VM migration. Using information about data center applications and server load, Moab can aim to keep VMs in the data center optimally distributed across hypervisors.

There are two types of policy-based VM migration:

• Performance-based Migration
• Consolidation-based Migration (formerly known as "Green Migration")

These two types have differing goals that at times may seem at odds with one another. Fortunately, Moab's scheduling engine is, in most cases, able to resolve and satisfy both sets of goals:

• Performance goal: To equalize loads across hypervisors as migrations are queued due to overcommit conditions. This places VMs to be migrated on the least-loaded HV available.
• Consolidation goal: To load hypervisors as close to thresholds as possible, without exceeding them. This policy places VMs to be migrated on the most loaded HV possible, within these constraints. A second loop of this policy will select lightly-loaded hypervisors to be evacuated completely.

This chapter will explain each type of migration in more detail and the general steps taken by Moab to find the best VM placement. It will also discuss how to configure Moab to perform automatic migration.

• Overcommit Factor and Threshold
• Overutilization Migration
• Green Migration and Consolidation

21.1-A Throttling VM Migration

When the event occurs that triggers the migration you will want to prevent your infrastructure from being overloaded with migrations. In the following example we use GRES to limit the migration of VMs to 10.

Example of throttling in the Moab.cfg:

# Limit for reservation-based migration throttle Moab.cfg
#Throttle the number of VMs that can be migrated at any given time to 10
VMMigrateThrottle 10

21.1-B General Notes

• If the RM reports a VM's resources as undefined or 0, or the VM's reported resources do not meet the following criteria, Moab prevents the VM from migrating.
  • The VM's state must not be "Unknown." 
  • The VM must have a reported CPULOAD greater than 0.
  • The VM must have an AMEMORY less than its CMEMORY. This indicates that some memory is currently in use and tells Moab that the RM is reporting memory correctly.
• The RM must report a hypervisor's HVType and a positive CPU and memory load for Moab to consider it for migration; however, Moab will allow migration to hypervisors without a reported CPU or memory load if VMMIGRATETOZEROLOADNODES is TRUE.
• The mvmctl -f command, run with the eval flag, does not submit migration jobs, but returns a report detailing the actions that would occur if you configured a VM migration policy.

Example moab.cfg

SCHEDCFGMoab SERVER=server:42559

#Note for this you would need to have a xcat and msm configuration. For convenience I have removed those to keep this example short
#VM Management

HideVirtualNodes Transparent
AllowVMMigration TRUE

VMMigrateThrottle 10

PARCFG[ALL] VMMigrateDuration=0:40:00
PARCFG[ALL] VMCreateDuration=01:00:00

PARCFG[ALL] VMDeleteDuration=30:00

#List of images available

IMAGECFG[xcat] VMOSLIST=suse11,win2008,rhel54

#Set the powerpolicy for each node

NODECFG[DEFAULT] POWERPOLICY=green

#Number of systems to keep active for a resource in the pool. This example keeps one system available if VM migration is needed.

MaxGreenStandByPoolSize 1

PARCFG[xcat] NODEPOWEROFFDURATION=20:00
PARCFG[xcat] NODEPOWERONDURATION=20:00

#Max time to wait before a hypervisor is powered off if that hypervisor is idle. This is a green setting
NodeIdlePowerThreshold 40:00:00

#Overcommit settings and green settings

NODEAVAILABILITYPOLICY UTILIZED
VMMigrationPolicy OVERCOMMIT,CONSOLIDATION@10:00:00
VMOCThreshold PROC:0.7,MEM:0.95
NODECFG[DEFAULT] OVERCOMMIT=PROC:2.0,MEM:2.0

GEVENTCFG[disk_free] ACTION=fail SEVERITY=4
NODECFG[DEFAULT] TRIGGER=atype=exec,etype=threshold,failoffset=1:00,threshold=gmetricdisk_free>90,action="/opt/moab/tools/filesize_fault.py $OID $METRICTYPE"