HP XC System Software Administration Guide Version 2.1

# controllsf show current
LSF is currently up, and assigned to node n16
All LSF administration must be don e from the LSF-HPC Executio n H
ost. The lsadmin and
badmin commands can only be run on this host; they are not intend
ed to be run on any other
nodes in the HP X C system and may p roduce f alse results if they a
re.
When the LSF-HPC scheduler determines that it is time to dispatch a job, it re
quests an
allocation of nodes from SLURM. After the successful allocation, LS
F-HPC prepares the
job environm ent with the necessary SLURM allocatio n variables, th
at is, SLURM_JOBID
and SLURM_NPROCS, and launches the user’s job locally on the LSF-HP
C Execution Host.
The user’s job must contain SLURM srun commands to pick up the alloc
ation enviro nment
variables and dispatch the job to the allocated compute nodes.
12.1.1 Job Starter Script
This release of the HP XC System Software provides an LSF queue JOB_STARTER
script, which is configured for all default queues during HP XC system installation. This
JOB_STARTER script performs three tasks:
• It creates an accurate LSB_HOSTS environment variable.
• It creates an accurate LSB_MCPU_HOSTS environment variable.
•itusesansrun command to launch the user’s job on the first allocated compute node.
The LSB_HOSTS and LSB_MCPU_HOSTS environment variables as initially established by
LSF-HPC do not accurately reflect the host names of the HP XC system nodes that have been
allocated by SLURM for the user’s job. This JOB_STARTER script corrects these environment
variables so that existing LSF-ready applications can use them without further adjustment.
The srun command used by the JOB_STARTER script ensures that every job submitted by a
user begins on the first allocated node. Without the JOB_STARTER script, all user jobs would
start on the LSF-HPC Execution Host; this is not consistent with standard LSF behavior and
creates the potential for a bottleneck in perf orm ance as b oth the LSF-HPC daem ons and local
user tasks compete for CPU cycles.
The JOB_STARTER script has two drawback s:
•TheJOB_STARTER script does not support the bsub command’s -i input file option.
The work-aro und for this is to provide the input file directly to the application. The SLURM
srun command has its o wn --input o ption, which should suffice.
• All I/O runs through the srun comm and in the JOB_STARTER script. This mean s full
tty support is not available for interactive s ession s, resulting in no prompting when
launching a shell.
The work-aro und is to set your display to s upp ort launching an xterm instead of a shell.
The JOB_STARTER script is located at /opt/hptc/lsf/bin/job_starter.sh,andis
pre-configured for a ll of the queues created during the default LSF-HPC installation on XC. It’s
recommended that the JOB_STARTER script be configured for all queues.
To disable the JOB_STARTER script, simply remove it or comment it out from the lsb.queues
configuration file. See the manu al titled Administering Platform LSF from Platform C o mputing
for more inform a tion on the JOB_STARTER option and configurin g queues.
See the standard LSF documentatio n for more information on configuring JOB_STARTER
scripts and how they work.
12.1.2 SLURM External Scheduler
The integration of LSF-HPC with SLURM in clud es the addition of a SLURM-based external
scheduler. Users can submit SLURM parameters in the context of their jobs. This enables users
12-2 LSF-HPC Administration