Upgrading the xCAT connector from 7.2.x

To upgrade the RPM suite

1. Shut down all Adaptive Computing applications.

[root]# service moab stop
[root]# service mam stop
[root]# service tomcat6 stop

2. Untar the tarball.

[root]# tar xzf <connector-type>-<version>.tar.gz

3. Change directory into the extracted directory.
4. Install the repository.

[root]# ./install-rpm-repos.sh [<repository-directory>]



For a description of the options of the repository installer script, run:

# ./install-rpm-repos.sh -h

The [<repository-directory>] option is the directory where you want to copy the RPMs. If no argument is given, [<repository-directory>] defaults to /opt/adaptive-rpm-repository/rpm. If the [<repository-directory>] already exists, RPMs will be added to the existing directory. No files are overwritten in [<repository-directory>]. A repository file is also created in /etc/yum.repos.d/ and points to the [<repository-directory>] location.

For ease in repository maintenance, the install script fails if Adaptive Computing RPMs are copied to different directories. If a non-default [<repository-directory>] is specified, please use the same directory for future updates.

The script installs the createrepo package and its dependencies. You must answer "y" to all the questions in order for the RPM install to work. Additionally, the script installs the EPEL, xCAT, and 10gen repositories.

5. Merge the new .repo files in /etc/yum.repos.d/ with the existing ones.

The install-rpm-repos.sh script will not overwrite existing RPM, GPG key or .repo files. Because some .repo files may have changed from previous releases, some merging of the .repo files is necessary. The newest files will have the .new extension.

Please compare older .repo files with the newer ones to ensure that the latest changes are reflected. In some cases, there is no change, and you can remove the new file. In most cases, however, it is safe to overwrite the old .repo file with the new one. For example:

[root]# mv /etc/yum.repos.d/AC.repo.new /etc/yum.repos.d/AC.repo



The latest xCAT-core.repo file points to xCAT version 2.7. If you are not ready to upgrade xCAT to 2.7, then do not use the xCAT-core.repo.new file.

After making changes in the /etc/yum.repos.d directory, make sure to run the following command to update the yum cache:

[root]# yum clean all

6. Update all the moab RPMs to the latest version.

[root]# yum update 'moab*'



The Moab and MWS RPMs automatically create a backup of all relevant files. These backups are stored in /var/tmp/backup-<rpmName>-<timestamp>.tar.gz.

If changes are detected between any existing configuration files and new configuration files, a version of the new configuration file will be saved under <configurationFileLocation>/<fileName>.rpmnew.

7. (Optional, but recommended) Adaptive Computing strongly recommends upgrading MongoDB to version 2.4. Support for environments using MongoDB 2.0 is now deprecated and will be removed in future releases. Please refer to docs.mongodb.org for instructions on how to upgrade MongoDB. Note that you must pay close attention to the information regarding instances with auth enabled (as this is the recommended setup for Moab Cloud Suite).

Removing the mongo20-10gen and mongo20-10gen-server packages may force a removal of the moab-cloud-suite package (see example below). It is okay to allow this; moab-cloud-suite is a virtual RPM package used only for convenience in installing, and removing it will not harm the current installation.

[root]# service mongod stop
[root]# yum remove mongo20-10gen mongo20-10gen-server
[root}# yum install mongo-10gen-server
[root]# service mongod start



Note that the settings in the /etc/mongod.conf file were saved in /etc/mongod.conf.rpmsave while removing MongoDB 2.0. You may need to be restore any custom settings after MongoDB 2.4 is installed in the new /etc/mongod.conf file (for example, "auth = true").

After upgrading to MongoDB 2.4, you should verify that the MongoDB credentials were preserved:

[root]# mongo -u mws_user mws -p
MongoDB shell version: 2.4.8
connecting to: mws
> show collections
counters
event
...

8. Upgrade the schema of the mws database in MongoDB.


You must perform this step, regardless of whether you upgraded MongoDB to version 2.4 or not. (See previous step.)



Before updating this database, you should perform a full backup. This can be done by using the mongodump utility documented in the MongoDB documentation.

Run the database migration script provided with MWS. (It is safe to run this script more than once. If for any reason, errors occur during the execution of the script, run it again.)

[root]# mongo -u mws_user mws /opt/mws/utils/db-migrate.js -p



Depending on the number of events and services in the system, the script may take several minutes to execute.

9. (Optional, but recommended) You must update the moab-verify-oracle-java RPM package before you can proceed with the Java 7 upgrade. You may need to remove Java 6 before upgrading to Java 7.


Oracle Java 7 Runtime Environment is the recommended Java environment, but Java 6 is also supported. All other versions of Java, including OpenJDK/IcedTea, GNU Compiler for Java, etc. cannot run Moab Web Services.

Do the following:

1. Update the moab-verify-oracle-java RPM package.

[root]# yum update moab-verify-oracle-java

2. Download the Linux x64 RPM version of Oracle Java SE 7 JRE. (Go to the Oracle Java 7 download page, copy the URL to the Linux x64 RPM version, then run the following command.)

[root]# wget <URL> -O jre-7-linux-x64.rpm

To verify that the download was successful, run the following on the RPM before installation:

[root]# rpm -qip jre-7-linux-x64.rpm

3. Run the following to install Java 7:

[root]# rpm -Uh jre-7-linux-x64.rpm

10. Merge the configuration files.


Whether or not to start with the old configuration file and add newer configuration options (or vice versa) depends on the amount of customization you previously made in earlier versions. In instances where you have modified very little, you should consider using the newer configuration and merging site-specific settings from the old file into the new one. The following steps highlight important changes between the 7.2.x default configuration and the 7.5.0 default configuration. Also note that new configuration files may have auto-generated content for secret keys and default passwords—be careful to ensure that secret keys shared between components are configured correctly.



The recommended layout for the /opt/moab/etc/ directory appears as follows:

# ls -al /opt/moab/etc/
total 92
lrwxrwxrwx. 1 root root    14 Nov 13 15:45 cloud.cfg -> cloud.xcat.cfg
-rw-r--r--. 1 root moab  4166 Nov 13 15:45 cloud.xcat.cfg
-rw-r--r--. 1 root moab  2323 Nov 13 13:41 config.moab.pl
-rw-r--r--. 1 root moab   989 Nov 13 13:41 config.sql.pl
-rw-r--r--. 1 root moab  1966 Nov 13 13:41 log4perl.cfg
lrwxrwxrwx. 1 root root    14 Nov 13 15:46 moab.cfg -> moab.cloud.cfg
-rw-r--r--. 1 root moab 23500 Nov 13 15:43 moab.cloud.cfg
drwxr-xr-x. 2 root moab  4096 Nov 13 15:41 moab.d
-rw-r--r--. 1 root moab   391 Nov 13 13:41 moab.dat
-r--r--r--. 1 root root   493 Nov  6 16:14 moab.lic
-rw-------. 1 root moab   288 Nov 13 15:39 moab-private.cfg
-rw-r--r--. 1 root moab  2058 Nov 13 13:41 nagios.cfg
lrwxrwxrwx. 1 root root    14 Nov 13 15:46 nami.cfg -> nami.cloud.cfg
-rw-r--r--. 1 root moab  2141 Nov 13 13:41 nami.cloud.cfg
-rw-r--r--. 1 root moab  2089 Nov  6 16:17 vlan.cfg

 Do the following:

1. Merge the /opt/moab/etc/moab-private.cfg file. Make sure that unique items in /opt/moab/etc/moab-private.cfg.rpmnew are added to the existing /opt/moab/etc/moab-private.cfg file. This includes the new MWS RM credentials:

CLIENTCFG[RM:mws] USERNAME=moab-admin PASSWORD=changeme!



The default MWS credentials in 7.2.x were admin:adminpw. These have been changed in 7.5.0 to moab-admin:changeme! in order to mirror the Viewpoint default credentials. Use whatever credentials you have configured in /opt/mws/etc/mws-config.groovy.

2. Merge customizations from /opt/moab/etc/moab.cfg and /opt/moab/etc/moab.d/* into /opt/moab/etc/moab.cloud.cfg.

Some important updates were made to the default configuration file between 7.2.x and 7.5.0, and it is important that these changes are made to the moab.cfg file you are using. Although there are several ways to configure and merge changes into the /opt/moab/etc/moab.cfg file, the following instructions outline the recommended best practices. Deviations from these best practices may result in unexpected behavior or added difficulty in future upgrades.

It is best to use the new default configuration file (/opt/moab/etc/moab.cloud.cfg) and merge changes from previous files into that one. You will notice that content from the /opt/moab/etc/moab.d/ directory has been merged into /opt/moab/etc/moab.cloud.cfg. Ensure that custom configuration options in all files located in /opt/moab/etc/moab.d/ directory get merged in to /opt/moab/etc/moab.cloud.cfg. This is especially important for the /opt/moab/etc/moab.d/policy.cfg file.

You should avoid #include configurations.

Keep in mind the following changes between the 7.2.x configuration file (/opt/moab/etc/moab.cfg) and the 7.5.0 configuration file (/opt/moab/etc/moab.cloud.cfg). Differences not listed below should be configured in the 7.5.0 configuration file as they were configured in the 7.2.x file.

• The "evacvms" reservation profile is no longer needed. It is harmless if left configured, but does not need to exist in moab.cloud.cfg in 7.5.0.

RSVPROFILE[evacvms] TRIGGER=EType=start,AType=internal,action=node:$(HOSTLIST):evacvms

• Moab now integrates natively with MWS. You must change the RMCFG[mws] lines to reflect the change.

How it looks in 7.2.x:

##############################################
#
# Resource Manager Configuration - mws
#
##############################################
RMCFG[mws]                      TYPE=NATIVE
RMCFG[mws]                      ENV=PERL5LIB=/opt/moab/lib/perl5
RMCFG[mws]                      ClusterQueryURL=exec://$TOOLSDIR/mws/cluster.query.mws.pl
RMCFG[mws]                      POLLINTERVAL=00:30

How it should look in 7.5.0:

################################################################################
#
# Resource Manager Configuration - mws
# NOTE: This RM must come after the storage RM in order
#       for storage to work correctly
#
################################################################################
RMCFG[mws]                      TYPE=MWS
RMCFG[mws]                      POLLINTERVAL=00:30
RMCFG[mws]                      BASEURL=http://localhost:8080/mws
# Credential information is stored in moab-private.cfg
						

• Make sure to add the NOCREATERESOURCE flag to the "monitoring" resource manager. For example:

RMCFG[monitoring]                  ClusterQueryURL=exec://$TOOLSDIR/nagios/cluster.query.nagios.pl
RMCFG[monitoring]                  POLLINTERVAL=00:30
RMCFG[monitoring]                  FLAGS=nocreateresource

• The following addition was also made to the /opt/moab/etc/moab.cloud.cfg file in 7.5.0:

# How often Moab should check Node/VM features to see if a configuration change
# has caused a breach in policy
NodeVMFeatureCheckTime           10:00

• Unless they are absolutely necessary for your environment, the following configurations should not be enabled:

AssignVLanFeatures               TRUE
NODESETPOLICY                    ONEOF
NODESETATTRIBUTE                 FEATURE
NODESETLIST                      vlan0,vlan1,vlan2,vlan3,vlan4,vlan5
RMCFG[filerm]                    TYPE=NATIVE
RMCFG[filerm]                    ClusterQueryURL=file://$TOOLSDIR/file.rm.dat

Although the upgrade should have created a backup of the moab.cfg file (in /var/tmp/backup-<rpmName>-<timestamp>.tar.gz), it is best to create your own backup until you can confirm the updated configuration behaves as expected.

[root]# mv /opt/moab/etc/moab.cfg /opt/moab/etc/moab.cloud.cfg.bak

Once the changes have been merged to /opt/moab/etc/moab.cloud.cfg, configure Moab to use the new file. The recommended configuration is to use a symlink called /opt/moab/etc/moab.cfg that points to /opt/moab/etc/moab.cloud.cfg.

[root]# ln -s /opt/moab/etc/moab.cloud.cfg /opt/moab/etc/moab.cfg

3. Make sure the cloud.cfg and the nami.cfg symlinks are created.

[root]# ln -s /opt/moab/etc/cloud.xcat.cfg cloud.cfg

[root]# ln -s /opt/moab/etc/nami.cloud.cfg nami.cfg

4. Merge the /opt/moab/etc/vlan.cfg file.

/opt/moab/etc/vlan.cfg should be the original vlan.cfg file used in 7.2.x. There are no significant changes between 7.2.x and 7.5.0 regarding the format or content of this file. Instructions for properly configuring this file can be found in Configuring Moab Cloud Suite for xCAT.

5. Merge the /opt/mws/etc/mws-config.groovy file.

Merge the /opt/mws/etc/mws-config.groovy.rpmnew file with the old /opt/mws/etc/mws-config.groovy file by editing /opt/mws/etc/mws-config.groovy. (Note the addition of the "auditAppender" in the default logging configuration of /opt/mws/etc/mws-config.groovy.rpmnew.)

log4j = {
...
   // Configure an appender for the audit log.
   def auditAppender = new org.apache.log4j.rolling.RollingFileAppender(
         name: 'audit',
         layout: new com.ace.mws.logging.ACPatternLayout("%j\t\t\t%c{1}\t\t\t%m%n"))
   def auditRollingPolicy = new org.apache.log4j.rolling.TimeBasedRollingPolicy(
         fileNamePattern: '/opt/mws/log/audit.%d{yyyy-MM-dd}',
         activeFileName: '/opt/mws/log/audit.log')
   auditRollingPolicy.activateOptions()
   auditAppender.setRollingPolicy(auditRollingPolicy)

   appenders {
         ...
         appender auditAppender
    }
    ...
    // Logs audit information to the audit log, not the rootLog
    trace additivity:false, audit:'mws.audit'
}

Note that the "mws.suite" parameter and the "mam.*" parameters have been moved to a suite-specific file in /opt/mws/etc/mws.d/ and do not need to exist in /opt/mws/etc/mws-config.groovy.

Also note the new "*messageQueue" parameters in /opt/mws/etc/mws-config.groovy.rpmnew. These are required and the moab.messageQueue.secretKey value should match the value located in /opt/moab/etc/moab-private.cfg.

11. Perform the following actions to upgrade the Viewpoint configuration.


The following step deletes all content in the $VIEWPOINT_HOME directory. This includes Viewpoint configuration information (for example, MWS and LDAP connection information), page customizations, and any other customizations made in any Viewpoint XML configuration files. These customizations can be restored from the data backed up during RPM installation.

The RPM installation should have backed up the contents of the $VIEWPOINT_HOME directory. However, before proceeding, you must verify that the backup was created in /var/tmp/backup-moab-viewpoint-<timestamp>.tar.gz. If it was not automatically created by the RPM installation, create a tarball from the contents of the $VIEWPOINT_HOME directory before proceeding.

1. In versions prior to 7.2.0, user information was stored in the Viewpoint MySQL or Oracle database. Now users and groups are stored in LDAP. If you haven't already, move users and groups over to OpenLDAP™ or Microsoft Active Directory®. For more information, see Setting up OpenLDAP on CentOS 6.


Viewpoint does not depend on MySQL™ or Oracle® databases that have been used in previous versions. Beginning with version 7.2.0, Viewpoint uses a new embedded database called "h2." Database files are automatically created within your $VIEWPOINT_HOME directory under data/h2/. This database contains only the current configuration from the Viewpoint Configuration page.



The MySQL or Oracle database you used for previous versions of Viewpoint is not the same as the MySQL database used for Moab. Do not remove the Moab database.

2. Remove the contents of the $VIEWPOINT_HOME directory.

[root]# rm -rf /opt/viewpoint/*

12. Start the moab service.

[root]# service moab start

13. Verify that the version has changed.

[root]# moab --about

14. Start the tomcat service.

[root]# service tomcat6 start

15. Log in to Viewpoint with the default super user username and password (moab-admin/changeme!).

The Viewpoint Configuration displays.

16. Do the following:
1. Set the Moab Web Services connection. (For more information, see "Setting the MWS configuration" in the Viewpoint Management and User Guide.)


You no longer configure the MWS connection in the ViewpointConfig.groovy file. This configuration will be stored in the h2 database.

2. Authenticate the LDAP connection by specifying the bind user password. (For more information, see "Setting the LDAP configuration" in the Viewpoint Management and User Guide.)
3. Change the super user password. (For more information, see "Changing the super user password" in the Viewpoint Management and User Guide.)
4. Check the MWS Diagnostics to verify that the MWS connections are active. (For more information, see "Fields: MWS Diagnostics" in the Viewpoint Management and User Guide.)
17. Permissions and roles (in versions prior to 7.2.0 stored in core.xml) are now stored in MWS. Use the Role Management page in Viewpoint to manage roles (groups of permissions) as you need. (For more information, see "About role management" in the Viewpoint Management and User Guide.)
18. Restart the moab service.

[root]# service moab stop

# Wait for Moab Services Manager to shut down (usually about 20 seconds)
[root]# service moab start

19. (Optional, if Moab Accounting Manager was previously installed) Start the mam service.

[root]# service mam start