(Click to open topic with navigation)

You are here: Nitro Overview > Introduction to Nitro

1.1 Introduction to Nitro

This Nitro User Guide documentation explains how to define tasks for a task file, the kinds of information you can give to Nitro via your job script and/or job submission to customize your Nitro job's execution, and the information Nitro will give you as it executes your tasks and after it has finished.

Nitro lets you execute many workloads quickly using a single job.

This topic introduces you to Nitro and provides some basic understanding about how to use Nitro.

In this topic

1.1.1 What is Nitro?

Nitro is a "high-throughput task scheduler" product that quickly executes user jobs or "tasks" that fit the criterion of being able to fit and execute on a single node.

A job or task can be a

While Nitro can be used for workloads that execute in short (sub-second) or long (hours) periods of time, the shorter a workload executes, the better performance gain Nitro will give you over a normal job scheduler; meaning the shorter the workload, the higher the performance gain.

Notwithstanding the advantages for small and short jobs, some sites where users may submit many (1000s) small jobs use Nitro to improve their normal job scheduler's performance, which degrades when the site has a large queue containing tens or hundreds of thousands, or even millions, of jobs. These sites have these users convert their many small jobs into a single Nitro job, which can greatly reduce the job queue size and speeds the response time for the users since the normal job scheduler can schedule a single job that then executes thousands or millions of small jobs as "tasks".

1.1.2 How Nitro Works

To execute many workloads quickly, you give Nitro a text-based "task" file containing commands that execute application programs you want to run. The task file has no limit on the number of application program commands, which means the task file may contain 10, 10 thousand, or 10 million tasks; it does not matter to Nitro.

If you have many small jobs, you convert them into Nitro tasks by extracting the jobs' commands to execute application programs and then putting the extracted commands into a single Nitro task file.

To use Nitro, you must do these four things:

  1. Create a task file.
  2. Create a job script.
  3. Submit your job script to a job scheduler and give it the location of your task file.
  4. Examine the Nitro job output to see the results.

1.1.3 About a Nitro Job

The following image identifies and illustrates the different items involved with a Nitro job, its submission to a job scheduler, its execution on the job's nodes, and the results it generates and returns to you.

Click to enlarge

A Nitro job is a user job script that defines where the Nitro task file is and launches the Nitro application. The user job script can tailor the Nitro application's execution through the use of pre-defined Nitro environment variables.

When executed, the Nitro application takes the information from the Nitro task file and generates output to four files, the Nitro Job Log File and Nitro Task Log File, and the job's Standard Output File and Standard Error File.

In addition, while the Nitro job is executing, the Nitro application saves checkpoint information in a Nitro Job Checkpoint File that permits a Nitro job to restart later and resume from where it stopped because it was cancelled, preempted, etc. When the Nitro application finishes processing all tasks, it deletes the checkpoint file.

1.1.4 How to Create and Run a Nitro Job

A Nitro job is a user job script that defines where the Nitro task file is and launches the Nitro application.

You can create the task file on any system that permits you to create and edit a text file. (for example, Windows or Linux systems).

Typically, your administrator will give you a sample job script to use as the starting point for your user job script. You can pass your task file name/path to your Nitro job either by

After creating the task file, and possibly customizing the job script, you submit your Nitro job script to your site's job scheduler to request a quantity of resources (for example, five nodes) on which to execute your Nitro job. The job scheduler schedules and allocates the requested resources to your job and then starts your Nitro job when the resources become available.

Your job script then executes the Nitro application via the launch_nitro.sh script and the Nitro application reads your task file and executes its tasks on your job's resources as quickly and with as little overhead as possible.

While tasks are executing, the Nitro application updates its Nitro Job Log File every five seconds to indicate its progress processing the tasks in your Nitro task file. In addition, for each task it processes, it outputs a record of the task, its status, and performance information, as well as possible task output, to the Nitro Task Log File.

When the Nitro job finishes,

1.1.5 Factors Affecting Nitro Job Performance

Your execution time will vary based on these factors:

For example:

If a node has 16 cores, it can execute 16 single-core tasks at once (simultaneously); however it can only execute 2 multi-threaded tasks that require 8 cores each. You can expect a faster tasks per second rate with the first scenario (16 single-core tasks), than with the 2 multi-threaded tasks scenario.

© 2017 Adaptive Computing