LSF Version 7.3 - Using Platform LSF HPC

◆ SGI Altix Linux ProPack 4 and ProPack 5 do not support
CPUSET_OPTIONS=CPUSET_MEMORY_MANDATORY or
CPUSET_OPTIONS=CPUSET_CPU_EXCLUSIVE attributes. If the cpuset job
runs on an Altix host, the cpusets created on the Altix system will have their
memory usage restricted to the memory nodes containing the CPUs assigned to the
cpuset. The CPUSET_MEMORY_MANDATORY and
CPUSET_CPU_EXCLUSIVE attributes are ignored.
◆ CPUSET_OPTIONS=CPUSET_MEMORY_MANDATORY implies node-level
allocation
◆ CPUSET_OPTIONS=CPUSET_MEMORY_MANDATORY cannot be used
together with MAX_CPU_PER_NODE=max_num_cpus
The scheduler will not use CPU 0 when determining an allocation on IRIX or TRIX.
You must not include CPU 0 in the list of CPUs specified by CPU_LIST.
You should not use the MPI_DSM_MUSTRUN=ON environment variable. If a job is
suspended through preemption, LSF can ensure that cpusets are recreated with the same
CPUs, but it cannot ensure that a certain task will run on a specific CPU. Jobs running
with MPI_DSM_MUSTRUN cannot migrate to a different part of the machine.
MPI_DSM_MUSTRUN also interferes with job checkpointing.
Including memory nodes in the allocation (Altix ProPack4 and Propack
5)
When you specify a list of memory node IDs with the cpuset external scheduler option
MEM_LIST, LSF creates a cpuset for the job that includes the memory nodes specified
by MEM_LIST in addition to the local memory attached to the CPUs allocated for the
cpuset. For example, if
"CPUSET[MEM_LIST=30-40]" , and a 2-CPU parallel job is
scheduled to run on CPU 0-1 (physically located on node 0), the job is able to use
memory on node 0 and nodes 30-40.
Unavailable memory nodes listed in MEM_LIST are ignored when LSF allocates the
cpuset. For example, a 4-CPU job across two hosts (hostA and hostB) that specifies
MEM_LIST=1 allocates 2 CPUs on each host. The job is scheduled as follows:
◆ CPU 0 and CPU 1 (memory=node 0, node 1) on hostA
◆ CPU 0 and CPU 1 (memory=node 0, node 1) on hostB
If hostB only has 2 CPUs, only node 0 is available, and the job will only use the memory
on node 0.
MEM_LIST is only available for dynamic cpuset jobs at both the queue level and the
command level.
When both MEM_LIST and CPUSET_OPTIONS=CPUSET_MEMORY_LOCAL
are both specified for the job, the root cpuset nodes are included as the memory nodes
for the cpuset. MEM_LIST is ignored, and CPUSET_MEMORY_LOCAL overrides
MEM_LIST.