LSF Version 7.3 - Using Platform LSF HPC

Running Jobs with Task Geometry
Specifying task geometry allows you to group tasks of a parallel job step to run together
on the same node. Task geometry allows for flexibility in how tasks are grouped for
execution on system nodes. You cannot specify the particular nodes that these groups
run on; the scheduler decides which nodes run the specified groupings.
Task geometry is supported for all Platform LSF MPI integrations including IBM POE,
LAM/MPI, MPICH-GM, MPICH-P4, and Intel® MPI.
Use the LSB_PJL_TASK_GEOMETRY environment variable to specify task geometry
for your jobs. LSB_PJL_TASK_GEOMETRY overrides any process group or
command file placement options.
The environment variable LSB_PJL_TASK_GEOMETRY is checked for all parallel
jobs. If LSB_PJL_TASK_GEOMETRY is set users submit a parallel job (a job that
requests more than 1 slot), LSF attempts to shape LSB_MCPU_HOSTS accordingly.
The
mpirun.lsf script sets the LSB_MCPU_HOSTS environment variable in the job
according to the task geometry specification. The PJL wrapper script controls the actual
PJL to start tasks based on the new LSB_MCPU_HOSTS and task geometry.
Syntax
setenv LSB_PJL_TASK_GEOMETRY "{( task_ID ,...) ...}"
For example, to submit a job to spawn 8 tasks and span 4 nodes, specify:
setenv LSB_PJL_TASK_GEOMETRY "{(2,5,7)(0,6)(1,3)(4)}"
◆ Tasks 2,5, and 7 run on one node
◆ Tasks 0 and 6 run on another node
◆ Tasks 1 and 3 run on a third node
◆ Task 4 runs on one node alone
Each task_ID number corresponds to a task ID in a job, each set of parenthesis
contains the task IDs assigned to one node. Tasks can appear in any order, but the entire
range of tasks specified must begin with 0, and must include all task ID numbers; you
cannot skip a task ID number. Use braces to enclose the entire task geometry
specification, and use parentheses to enclose groups of nodes. Use commas to separate
task IDs.
For example.
setenv LSB_PJL_TASK_GEOMETRY "{(1)(2)}"
is incorrect because it does not start from task 0.
setenv LSB_PJL_TASK_GEOMETRY "{(0)(3)}"
is incorrect because it does not specify task 1and 2.
LSB_PJL_TASK_GEOMETRY cannot request more hosts than specified by the
bsub -n option.
For example:
setenv LSB_PJL_TASK_GEOMETRY "{(0)(1)(2)}"
specifies three nodes, one task per node. A correct job submission must request at least
3 hosts: