• Switching Operating Systems with xCAT

Install a TFTP server. Then, to set up PXE booting, do the following:

1. Download the SYSLINUX binaries from the following location: http://www.kernel.org/pub/linux/utils/boot/syslinux/

   These binaries are also included with MSMHPC Tools. See Installing the MSMHPC Tools for more information.

2. Unpack at least chain.c32, pxelinux.0, and pxelinux.cfg/default in the TFTP server root.

   chain.c32 will not work with a mismatching version of pxelinux.0. If you are using automated deployment tools, replace pxelinux.0 with the latest version or find the matching version of chain.c32 for your version of SYSLINUX.

   The default PXE boot configuration file is pxelinux.cfg/default, a sample of which follows:

   DEFAULT windows
   PROMPT 0
   TIMEOUT 100
   LABEL windows
       KERNEL chain.c32
       APPEND hd0 1
   LABEL linux
       KERNEL chain.c32
       APPEND hd0 0

3. The "Linux" label boots the MBR bootloader, which should be set to boot into Linux by default. The "Windows" label boots the OS on the first partition. The previous example defaults to Windows. Reboot the machine and test PXE booting to ensure proper function. Then, modify the partitions as needed to reflect your setup.
4. After confirming that you can PXE boot both partitions by switching the DEFAULT, create two files in pxelinux.cfg/: "windows" and "linux". Make a list of MAC addresses for the compute nodes. Create a symlink from each node to one of the OSes it will boot:
ln -fs windows NODE001
ln -fs linux NODE002
...
ln -fs linux NODEXXX

5. Create a symlink from each MAC address (that you will boot) to its hostname (note: prepend the MAC address with 01- and use lowercase A-F characters):
ln -fs NODE001 01-00-03-47-43-3f-73
...
ln -fs NODEXXX 01-00-03-47-43-4f-92

6. Add a provisioning resource manager to the Moab configuration file (moab.cfg):

#####################################################################
#
#  OS Switching Resource Manager                                    
#
#####################################################################
RMCFG[prov] TYPE=NATIVE RESOURCETYPE=PROV
# This signifies the resource manager is a provisioning RM.
RMCFG[prov] ENV=OSSTRING=windows;RMNAME=HPC;PUBKEY=moabpublickey;DOMAIN=HPCDOMAIN;PROXY=http://WINDOWSHEADNODE:5343/MSMHPC
# RMNAME refers to the RM configured in 4.0.
# More information for this line is found in the moab.cfg file generated in section 1.1.1.
RMCFG[prov] PROVDURATION=5:00
# This tells Moab how long it takes the node to reboot.
RMCFG[prov] NODEMODIFYURL=exec://$HOME/scripts/os.switch.pl
# This is the script Moab calls to switch the operating system.

Example os.switch.pl:

#!/usr/bin/perl
#
# Copyright (c) 2007-2010 Adaptive Computing Enterprises, Inc.
#
use strict;
use warnings;
use FindBin;
use lib "$FindBin::Bin";
use Moab::MSMHPC;
my ($host, undef, $newos, undef) = @ARGV;
my $os;
my $pxe_cfg_dir = '/tftpboot/pxelinux.cfg';
if (defined $newos)
  {
  (undef, $os) = split '=', $newos;
  }
die "Usage: os.switch.pl  --set OS=\n"
  . "   os: 'linux' or 'windows'"
  unless defined $host and defined $os;
$host = uc($host);
if($os eq 'linux')
  {
  # Check that node is not already booting to the new OS
  chdir($pxe_cfg_dir) or die("Failed to change to $pxe_cfg_dir");
  exit 0 unless readlink($host) ne $os;
  #switch boot os
  chdir($pxe_cfg_dir) or die ("Failed to change to $pxe_cfg_dir");
  my $rc = system("ln -fs $os $host") >> 8;
  die "ln -fs $host $os FAILED with rc: $?" unless $rc == 0;
  # Reboot the node
  my $MSMHPC = Moab::MSMHPC->new();
     $MSMHPC->remoteReboot($host);
  }
elsif($os eq 'windows')
  {
  # Check that the node is not already being booted to the new OS
  chdir($pxe_cfg_dir) or die("Failed to change to $pxe_cfg_dir");
  exit 0 unless readlink($host) ne $os;
  chdir($pxe_cfg_dir) or die ("Failed to change to $pxe_cfg_dir");
  #switch boot os
  my $rc = system("ln -fs $os $host") >> 8;
  die "ln -fs $host $os FAILED with rc: $?" unless $rc ==0;
  #reboot the node
  $rc = system("ssh root\@$host shutdown -r now $os") >> 8;
  die "ssh root\@$host shutdown -r now FAILED with rc: $rc" unless $rc == 0;
  }
else
  {
  die "Usage: os.switch.pl  --set OS=\n"
        . "   os: 'linux' or 'windows'";
  }
print "pending\n";
exit 0;

The script is called when Moab needs to change the operating system on a node. It changes the operating system running on a node by taking the destination operating system and compute node name as shell parameters.

Modify the examples supplied in this section as needed.

6. After you install MSMHPC on the Windows head node and the MSMHPC Tools Perl scripts in the "scripts" directory of your Moab installation, verify PXE booting by running os.switch.pl NODENAME --set OS=windows (or linux) for each node.

To test switching from Windows to Linux, you need to fully configure Moab for switching. See section 3.0.

3.2.1 Switching Operating Systems with xCAT

The optional os.switch.pl.xcat script included in MSMHPC tools can be used in place of the PXE boot process, allowing you control MSMHPC using xCAT commands. The script only works with a full xCAT setup, including TFTP and DHCP servers. xCAT 2.6x is recommended.

To configure xCAT to work correctly with the os.switch.pl.xcat script, do the following:

1. Install xCAT and define all your nodes in it.
2. Deploy Windows/Linux on the nodes.
3. Copy chain.32 from to /tftpboot. To do so:
   1. Locate the chain.c32 file included in the MSMHPC tools.
4. Add a noderes.netboot entry to make the node boot via PXE instead of XNBA. A sample looks like this:

   "vm4",,"pxe","000.00.00.000"...

5. Add the OS entries to the boottargets table in noderes.netboot, adaptiving it to your disk or partition layout.
6. Ensure that the os.switch.pl.xcat script is copied into the $MOAB_HOME/tools directory and is configured in moab.cfg as the NODEMODIFYURL for the provisioning resource manager. See the Provisioning and Load Balancing documentation for more information.