LSF Version 7.3 - Administering Platform LSF

Administering Platform LSF 329
Resource Preemption
Dynamic Resources
Specify duration If the preemption resource is dynamic, you must specify the duration part of the
resource reservation string when you submit a preempting or preemptable job.
Resources outside the
control of LSF
If an ELIM is needed to determine the value of a dynamic resource (such as the
number of software licenses available), LSF preempts jobs as necessary, then waits
for ELIM to report that the resources are available before starting the high-priority
job. By default, LSF waits 300 seconds (5 minutes) for resources to become
available. This time can be increased (PREEMPTION_WAIT_TIME in
lsb.params).
If the preempted jobs do not release the resources, or the resources have been
intercepted by a non-LSF user, the ELIM does not report any more of the resource
becoming available, and LSF might preempt more jobs to get the resources.
Requirements for Resource Preemption
◆ Resource preemption depends on all these conditions:
◆ The preemption resources must be configured
(PREEMPTABLE_RESOURCES in
lsb.params).
◆ Jobs must reserve the correct amount of the preemption resource, using
resource reservation (the rusage part of the resource requirement string).
◆ For dynamic preemption resources, jobs must specify the duration part of the
resource reservation string.
◆ Jobs that use the preemption resource must be spread out among multiple
queues of different priority, and preemptive scheduling must be configured so
that preemption can occur among these queues (preemption can only occur if
jobs are in different queues).
◆ Only a releaseable resource can be a preemption resource. LSF must be
configured to release the preemption resource when the job is suspended
(RELEASE=Y in
lsf.shared, which is the default). You must configure this no
matter what your preemption action is.
◆ LSF’s preemption behaviour must be modified. By default, LSF’s default
preemption action does not allow an application to release any resources,
except for job slots and static shared resources.
Custom Job Controls for Resource Preemption
Why you have to customize LSF
By default, LSF’s preemption action is to send a suspend signal (SIGSTOP) to stop
the application. Some applications do not release resources when they get SIGSTOP.
If this happens, the preemption resource does not become available, and the
preempting job is not successful.
You modify LSF’s default preemption behaviour to make the application release the
preemption resource when a job is preempted.