LSF Version 7.3 - Using Platform LSF HPC

bsub -n 3 -R "span[ptile=1]" -I -a mpich_gm mpirun.lsf my_job
Job <564> is submitted to queue <hpc_linux>.
<<Waiting for dispatch ...>>
<<Starting on hostA>>
...
Planning your task geometry specification
You should plan their task geometry in advance and specify the job resource
requirements for LSF to select hosts appropriately.
Use
bsub -n and -R "span[ptile=]" to make sure LSF selects appropriate hosts
to run the job, so that:
◆ The correct number of nodes is specified
◆ All exceution hosts have the same number of available slots
◆ The ptile value is the maximum number of CPUs required on one node by task
geometry specifications.
LSB_PJL_TASK_GEOMETRY only guarantees the geometry but does not guarantee
the host order. You must make sure each host selected by LSF can run any group of tasks
specified in LSB_PJL_TASK_GEOMETRY.
You can also use
bsub -x to run jobs exclusively on a host. No other jobs share the
node once this job is scheduled.
Usage notes and limitations
◆ MPICH-P4 jobs:
MPICH-P4
mpirun requires the first task to run on local node OR all tasks to run
on remote node (
-nolocal). If the LSB_PJL_TASK_GEOMETRY environment
variable is set,
mpirun.lsf makes sure the task group that contains task 0 in
LSB_PJL_TASK_GEOMETRY runs on the first node.
◆ LAM/MPI jobs:
You should not specify
mpirun n manually on command line; you should use
LSB_PJL_TASK_GEOMETRY for consistency with other Platform LSF MPI
integrations. LSB_PJL_TASK_GEOMETRY overrides the
mpirun n option.
◆ OpenMPI jobs:
Each thread of an OpenMPI job is counted as a task. For example, task geometry
specification is:
setenv LSB_PJL_TASK_GEOMETRY "{(1), (2,3,4) (0,5)}"
and task 5 is an openmp job that spawns 3 threads. From this specification, the job
spans 3 nodes, and maximum number of CPUs required is 4 (because
(0,5)
requires 4 cpus). The job should be submitted as:
bsub -n 12 -R "span[ptile=4]" -a openmp mpirun.lsf myjob
Examples
For the following task geometry:
setenv LSB_PJL_TASK_GEOMETRY "{(2,5,7)(0,6)(1,3)(4)}"
The job submission should look like: