Starting in TORQUE version 3.0 users can run multiple MOMs on a single node. The initial reason to develop a multiple MOM capability was for testing purposes. A small cluster can be made to look larger since each MOM instance is treated as a separate node.
When running multiple MOMs on a node each MOM must have its own service and manager ports assigned. The default ports used by the MOM are 15002 and 15003. With the multi-mom alternate ports can be used without the need to change the default ports for pbs_server even when running a single instance of the MOM.
There are three steps to setting up multi-mom capability:
The attributes mom_service_port and mom_manager_port were added to the nodes file syntax to accommodate multiple MOMs on a single node. By default pbs_mom opens ports 15002 and 15003 for the service and management ports respectively. For multiple MOMs to run on the same IP address they need to have their own port values so they can be distinguished from each other. pbs_server learns about the port addresses of the different MOMs from entries in the server_priv/nodes file. The following is an example of a nodes file configured for multiple MOMs:
hosta np=2 hosta-1 np=2 mom_service_port=30001 mom_manager_port=30002 hosta-2 np=2 mom_service_port=31001 mom_manager_port=31002 hosta-3 np=2 mom_service_port=32001 mom_manager_port=32002
Note that all entries have a unique host name and that all port values are also unique. The entry hosta does not have a mom_service_port or mom_manager_port given. If unspecified, then the MOM defaults to ports 15002 and 15003.
18.104.22.168 hosta hosta-1 hosta-2 hosta-3
Even though the host name and all the aliases resolve to the same IP address, each MOM instance can still be distinguished from the others because of the unique port value assigned in the server_priv/nodes file.
To start multiple instances of pbs_mom on the same node, use the following syntax:
pbs_mom -m -M <port value of mom_service_port> -R <port value of mom_manager_port>
Continuing based on the earlier example, if you want to create four MOMs on hosta, type the following at the command line:
# pbs_mom -m -M 30001 -R 30002 # pbs_mom -m -M 31001 -R 31002 # pbs_mom -m -M 32001 -R 32002 # pbs_mom
Notice that the last call to pbs_mom uses no arguments. By default pbs_mom opens on ports 15002 and 15003. No arguments are necessary because there are no conflicts.
Terminate pbs_mom by using the momctl -s command. For any MOM using the default manager port 15003, the momctl -s command stops the mom. However, to terminate moms with a manager port value not equal to 15003, you must use the following syntax:
momctl -s -p
The -p option sends the terminating signal to the MOM manager port and the MOM is terminated.