Table 3-1. Visual Cluster Example

	Slot 1	Slot 2	Slot 3
Rack 1	Node A		Node D
Rack 2		Node B
Rack 3			Node C

A rack is a physical frame that holds a node. The slot is the location of the node inside the rack. The racks make up the first column of the table. The slot locations increase from left to right. For example, Node A is located on Rack 1 in Slot 1. Node D is also located on Rack 1 but instead of Slot 1 it's located in Slot 3. In the visual cluster Node A through Node D are displayed as icons. The different icons can represent node state, node attributes, reservations, jobs, and/or nodes. The subpanel sections below describe these states in more detail. Further information can be gathered about nodes by hovering the mouse over any nodes.

It should be noted that the visual cluster is for display purposes only and the location of the node does not play any part in how Workload Manager schedules.

3.5.1.1 Node Attribute Selector

The node attribute selector gives the user the power to see various attributes of the nodes displayed in the Visual Cluster. This allows the user to compare and contrast attributes of interest. Node attributes include standard categories such as architecture, OS, hardware metrics (memory, disk, swap, etc.), as well as any metric read in through Moab as a generic metric (a node's GMETRIC). The default display for node attributes is the corresponding color of the outer rim of each node cell. This can be changed in the section titled "Node Display Preferences".

The Clear Attribute button clears any current selection and node attribute.

Once a node attribute is selected, Moab Cluster Manager determines the number of nodes and processors that describe each attribute and displays a corresponding key value that matches the Visual Cluster by color. Each attribute's display can be individually controlled via the check box next to each node attribute name and color.

If the node attribute is a numerical value - for example, a generic metric - then Moab Cluster Manager will attempt to place the values into a reasonable range as to effectively categorize the values.

3.5.1.2 Highlight Jobs and/or Nodes

This section provides a user with the option of highlighting resources in the visual cluster table. The three sections are divided into jobs and nodes. Each section can be simultaneously displayed by giving each border a different color. The white box displays the names of the selected resources with the headers "Job: " and "Node: " respectively. The Select buttons open the list jobs or list nodes window depending on which resource the user has selected. The visual cluster window appears with the desired resources highlighted. The Clear buttons remove the highlight from the visual cluster table and erase the names from the colored box. The Color button changes the highlight color for the specified resource. The new highlight color is displayed in the colored box.

3.5.2 Processor Usage

Summary

This graph displays how the cluster's processors are being used over time. The left bar, or y-axis, displays the number of processors. The bottom bar, or x-axis, displays time. The light yellow color displays the total available processors on the cluster. The dark yellow color displays the processors used by jobs and job reservations. The blue color displays the processors used by reservations other than job reservations.

The switch statistics option allows for the Available Processors and Jobs Reservations colors to be switched.

3.5.3 Event Calendar

Summary

The events calendar helps you view the events you are interested in while filtering out those you are not interested in. This can be much more convenient than searching through logs and event files. Once you have finished filtering, you can view the details of the remaining events and correlate this with your nodes' state history.

Filtering

The first filter you should probably apply is the time filter. After setting a start and end time, you can then select what type of events you wish to see. Use the tree control at the top left of this window to click the event types and/or subtypes you desire. For example, you might choose to see all events, all job events, or job end events only. After making your selection, click the Apply Filter button.

If you are looking for events for a specific object, and you know the object's name, you can filter out all other events by specifying the object type and ID in the fields at the lower left. For example, you might wish to see only events relating to the job "moab.2". To do this, you would specify an object type of "Job" and an object ID of "moab.2".

Viewing Event Details

To view the details of desired events, simply click their colored icon representations in the timeline. Note that on busy schedulers, an icon may represent multiple events. You will be able to see the event type, ID, time, object type, object ID, and any messages from the scheduler pertaining to this event.

The Node State/Node Category Chart

Below the timeline is a colored line chart. Vertically below any point on the timeline one can see the percentage of nodes that were active, idle, down, etc. This can be useful in determining the events you might want to filter. For example, if you see that 30% of your nodes suddenly went idle, yet you know there was a large backlog during that time, you might want to view all reservation start events. This might help you find an unused reservation responsible for the idle nodes.