Platform LSF Administration Guide Version 6.2

Optimizing Performance in Large Sites
Administering Platform LSF
542
Optimizing Performance in Large Sites
As your site grows, you must tune your LSF cluster to support a large number of hosts
and an increased workload.
This chapter discusses how to efficiently tune querying, scheduling, and event logging
in a large cluster that scales to 5000 hosts and 100,000 jobs at any one time.
To target performance optimization to a cluster with 5000 hosts and 100,000 jobs, you
must:
◆ Configure your operating system. See “Tuning UNIX for Large Clusters” on
page 543
◆ Fine-tune LSF. See “Tuning LSF for Large Clusters” on page 544
What’s new in LSF performance?
LSF provides some new parameters for tuning your cluster, which you will learn about
in this chapter. However, before you calculate the values to use for tuning your cluster,
consider the following enhancements to the general performance of LSF daemons, job
dispatching, and event replaying:
◆ Both scheduling and querying are much faster
◆ Switching and replaying the events log file, lsb.events, is much faster. The length
of the events file no longer impacts performance
◆ Restarting and reconfiguring your cluster is much faster
◆ Job submission time is constant. It does not matter how many jobs are in the system.
The submission time does not vary.
◆ The scalability of load updates from the slaves to the master has increased
◆ Load update intervals are scaled automatically
The following graph shows the improvement in LIM startup after the LSF performance
enhancements:
Y axis: # of hosts
x axis: Time in seconds