LSF Version 7.3 - Using Platform LSF HPC

You should not use a CXFS file system for LSB_RLA_WORKDIR.
◆ LSF_PIM_SLEEPTIME_UPDATE=Y
On Altix hosts, use this parameter to improve job throughput and reduce a job’s
start time if there are many jobs running simultaneously on a host. This parameter
reduces communication traffic between
sbatchd and PIM on the same host.
When this parameter is defined:
❖ sbatchd does not query PIM immediately as it needs information—it will only
query PIM every LSF_PIM_SLEEPTIME seconds.
❖ sbatchd may be intermittently unable to retrieve process information for jobs
whose run time is smaller than LSF_PIM_SLEEPTIME.
❖ It may take longer to view resource usage with bjobs -l.
By default, Linux sets the maximum file descriptor limit to 1024. This value is too small
for jobs using more than 200 processes. To avoid MPI job failure, specify a larger file
descriptor limit. For example:
# /etc/init.d/lsf stop
# ulimit -n 16384
# /etc/init.d/lsf start
Any host with more than 200 CPUs should start the LSF daemons with the larger file
descriptor limit. SGI Altix already starts the
arrayd daemon with the same ulimit
specifier, so that MPI jobs run without LSF can start as well.
See the Platform LSF Configuration Guide for information about the lsb.queues
and
lsf.conf files.
Resources for dynamic and static cpusets
If your environment uses both static and dynamic cpusets or you have more than one
static cpuset configured, you must configure decreasing numeric resources to represent
the cpuset count, and use
-R "rusage" in job submission. This allows preemption,
and also lets you control number of jobs running on static and dynamic cpusets or on
each static cpuset.
Edit lsf.shared and configure resources for cpusets and configure resources for
static cpusets and non-static cpusets. For example:
Begin Resource
RESOURCENAME TYPE INTERVAL INCREASING DESCRIPTION # Keywords
...
dcpus Numeric () N
scpus Numeric () N
End Resource
Where:
❖ dcpus is the number CPUs outside static cpusets (that is the total number of
CPUs minus the number of CPUs in static cpusets).
❖ scpus is the number of CPUs in static cpusets. For static cpusets, configure a
separate resource for each static cpuset. You should use the cpuset name as the
resource name.