LSF Version 7.3 - Administering Platform LSF

Administering Platform LSF 615
Achieving Performance and Scalability
Increase the file
descriptor limit
1 To achieve efficiency of performance in LSF, follow the instructions in your
operating system documentation to increase the number of file descriptors on
the LSF master host.
TIP: To optimize your configuration, set your file descriptor limit to a value at least as high as
the number of hosts in your cluster.
The following is an example configuration. The instructions for different
operating systems, kernels, and shells are varied. You may have already
configured the host to use the maximum number of file descriptors that are
allowed by the operating system. On some operating systems, the limit is
configured dynamically.
Your cluster size is 5000 hosts. Your master host is on Linux, kernel version 2.4:
1 Log in to the LSF master host as the
root user.
2 Add the following line to your
/etc/rc.d/rc.local startup script:
echo -n "5120" > /proc/sys/fs/file-max
3 Restart the operating system to apply the changes.
4 In the
bash shell, instruct the operating system to use the new file limits:
# ulimit -n unlimited
Tuning LSF for Large Clusters
To enable and sustain large clusters, you need to tune LSF for efficient querying,
dispatching, and event log management.
In this section
◆ Managing scheduling performance on page 615
◆ Limiting the number of batch queries on page 617
◆ Improving the speed of host status updates on page 617
◆ Managing your user’s ability to move jobs in a queue on page 618
◆ Managing the number of pending reasons on page 618
◆ Achieving efficient event switching on page 618
◆ Automatic load updating on page 619
◆ Managing the I/O performance of the info directory on page 619
◆ Processor binding for LSF job processes on page 620
◆ Increasing the job ID limit on page 622
Managing scheduling performance
For fast job dispatching in a large cluster, configure the following parameters:
LSB_MAX_JOB_DISPATCH_PER_SESSION in lsf.conf
The maximum number of jobs the scheduler can dispatch in one scheduling session