(Click to open topic with navigation)
This topic describes the motivation and purpose behind the creation of Nitro.
In this topic:
Until recently, Adaptive Computing's traditional market has been scheduling batch jobs for HPC systems, colloquially known as "supercomputers". These HPC systems are typically composed of hundreds to tens of thousands of "compute nodes" (servers designed for fast computations and large amounts of I/O), which are increasingly "off-the-shelf" servers. Part of scheduling workloads on such large HPC systems is optimizing the use of all the system's resources. This can be a complex process involving a large amount of computation which can take a significant amount of time.
Complicating the scheduling of HPC systems is the increasing use of workloads consisting of small jobs. These workloads do not need multiple servers but can execute on a single server or even on a single core, and may execute in a short amount of time, sometimes just seconds or even sub-seconds. Such workloads typically fall under the category of HTC and do not need the optimization performed by HPC schedulers.
When an HPC scheduler must schedule large quantities of HTC workloads, these workloads severely slow down the HPC scheduler and reduce its scheduling throughput. To counteract the effects of HTC workloads on HPC schedulers, Adaptive Computing created Nitro. Nitro schedules and runs HTC workloads on any HPC cluster or in any datacenter without affecting the throughput of the HPC or the datacenter scheduling software.
Nitro runs large quantities of HTC workloads as a single job submitted to any HPC or datacenter scheduler (it is scheduler-agnostic) using the resources allocated to it by the HPC or datacenter scheduler. This permits the scheduler to operate normally and eliminates the impact the scheduling of thousands or millions of HTC workloads would have on it.
Nitro runs workloads small enough to execute on a single compute node or server, regardless whether HTC in nature, with as little overhead as possible in order to speed up the execution of HTC-like workloads through the elimination of regular scheduler overhead. This is its primary intended purpose.
Nitro runs a single workload submitted to any HPC or datacenter scheduler and executes hundreds, thousands, or even millions of HTC-like workloads using the resources allocated to it by the HPC or datacenter scheduler. This permits the scheduler to operate normally by eliminating the impact that scheduling of thousands or millions of HTC workloads would have on the scheduler.