Platform LSF Administration Guide Version 6.2

Chapter 28
Running Parallel Jobs
Administering Platform LSF
449
2 Shortly afterwards, a parallel job (job2) requiring all 4 CPUs is submitted. It cannot
start right away because
job1 is using one CPU, so it reserves the remaining 3
processors (figure b).
3 At 8:30 am, another parallel job (job3) is submitted requiring only two processors
and with a run limit of 1 hour. Since
job2 cannot start until 10:00am (when job1
finishes), its reserved processors can be backfilled by
job3 (figure c). Therefore
job3 can complete before job2's start time, making use of the idle processors.
4 Job3 will finish at 9:30am and job1 at 10:00am, allowing job2 to start shortly
after 10:00am.
In this example, if
job3's run limit was 2 hours, it would not be able to backfill job2's
reserved slots, and would have to run after
job2 finishes.
Limitations
◆ A job will not have an estimated start time immediately after mbatchd is
reconfigured.
◆ Jobs in a backfill queue cannot be preempted (a job in a backfill queue might be
running in a reserved job slot, and starting a new job in that slot might delay the start
of the big parallel job):
❖ A backfill queue cannot be preemptable.
❖ A preemptive queue whose priority is higher than the backfill queue cannot
preempt the jobs in backfill queue.
Backfilling and job
slot limits
A backfill job borrows a job slot that is already taken by another job. The backfill job
will not run at the same time as the job that reserved the job slot first. Backfilling can
take place even if the job slot limits for a host or processor have been reached.
Backfilling cannot take place if the job slot limits for users or queues have been reached.
Configuring backfill scheduling
Backfill scheduling is enabled at the queue level. Only jobs in a backfill queue can backfill
reserved job slots. If the backfill queue also allows processor reservation, then
backfilling can occur among jobs within the same queue.
Configuring a
backfill queue
To configure a backfill queue, define BACKFILL in lsb.queues.
Specify
Y to enable backfilling. To disable backfilling, specify N or blank space.
Example
BACKFILL=Y
Enforcing run limits
Backfill scheduling works most efficiently when all the jobs in a cluster have a run limit
specified at the job level (
bsub -W). You can use the external submission executable,
esub, to make sure that all users specify a job-level run limit.
Otherwise, you can specify ceiling and default run limits at the queue level (RUNLIMIT
in
lsb.queues).
Viewing information about job start time
Use bjobs -l to view the estimated start time of a job.