HP XC System Software Administration Guide Version 4.0

Table Of Contents
directly to the job. The SLURM srun command supports an --input option (also available in
its short form as the -i option) that provides input to all tasks.
16.2.1.1 Job Starter Scripts
LSF with SLURM dispatches all jobs locally. The default installation of LSF with SLURM on the
HP XC system provides a job starter script that is configured for use by all LSF queues. This job
starter script adjusts the LSB_HOSTS and LSB_MCPU_HOSTS environment variables to the correct
resource values in the allocation. Then, the job starter script uses the srun command to launch
the user task on the first node in the allocation.
If this job starter script is not configured for a queue, the user jobs begin execution locally on the
LSF execution host. In this case, it is recommended that the user job uses one or more srun
commands to make use of the resources allocated to the job. Work done on the LSF execution
host competes for core time with the LSF with SLURM daemons, and could affect the overall
performance of LSF with SLURM on the HP XC system.
The bqueues -l command displays the full queue configuration, including whether or not a
job starter script has been configured. See the Platform LSF documentation or bqueues(1) for more
information on the use of this command.
For example, consider an LSF with SLURM configuration in which node n20 is the LSF execution
host and nodes n[1-10] are in the SLURM lsf partition. The default normal queue contains
the job starter script, but the unscripted queue does not have the job starter script configured.
Example 16-2 Comparison of Queues and the Configuration of the Job Starter Script
$ bqueues -l normal | grep JOB_STARTER
JOB_STARTER: /opt/hptc/lsf/bin/job_starter.sh
$ bqueues -l unscripted | grep JOB_STARTER
JOB_STARTER:
$ bsub -Is hostname
Job <66> is submitted to the default queue <normal>.
<<Waiting for dispatch...>>
<<Starting on lsfhost.localdomain>>
n10
$ bsub -Is -q unscripted hostname
Job <67> is submitted to the default queue <unscripted>.
<<Waiting for dispatch...>>
<<Starting on lsfhost.localdomain>>
n20
This release of the HP XC System Software provides an LSF queue JOB_STARTER script, which
is configured for all default queues during HP XC installation. This JOB_STARTER script performs
three tasks:
It creates an accurate LSB_HOSTS environment variable.
It creates an accurate LSB_MCPU_HOSTS environment variable.
It uses a SLURM srun command to launch a user's interactive job on the first allocated
compute node.
The LSB_HOSTS and LSB_MCPU_HOSTS environment variables, as initially established by LSF
with SLURM, do not accurately reflect the host names of the HP XC system nodes that SLURM
allocated for the user's job. This JOB_STARTER script corrects these environment variables so
that existing applications compatible with LSF can use them without further adjustment.
The SLURM srun command used by the JOB_STARTER script ensures that every interactive
job submitted by a user begins on the first allocated node. Without the JOB_STARTER script, all
interactive user jobs would start on the LSF execution host. This behavior is not consistent with
192 Managing LSF