Platform LSF Administration Guide Version 6.2

Chapter 26
Chunk Job Dispatch
Administering Platform LSF
411
Configuring a Chunk Job Dispatch
CHUNK_JOB_SIZE (lsb.queues)
To configure a queue to dispatch chunk jobs, specify the CHUNK_JOB_SIZE
parameter in the queue definition in
lsb.queues.
For example, the following configures a queue named
chunk, which dispatches up to 4
jobs in a chunk:
Begin Queue
QUEUE_NAME = chunk
PRIORITY = 50
CHUNK_JOB_SIZE = 4
End Queue
After adding CHUNK_JOB_SIZE to lsb.queues, use badmin reconfig to
reconfigure your cluster.
By default, CHUNK_JOB_SIZE is not enabled.
Chunk jobs and
job throughput
Throughput can deteriorate if the chunk job size is too big. Performance may decrease
on queues with CHUNK_JOB_SIZE greater than 30. You should evaluate the chunk
job size on your own systems for best performance.
CHUNK_JOB_DURATION (lsb.params)
If CHUNK_JOB_DURATION is set in lsb.params, jobs submitted to a chunk job
queue are only chunked if the job has a CPU limit or run limit set in the queue
(CPULIMIT or RUNLMIT) or specified at job submission (
-c or -W bsub options)
that is less than or equal to the value of CHUNK_JOB_DURATION.
Jobs are not chunked if:
◆ CPU limit or a run limit is greater than the value of CHUNK_JOB_DURATION.
or
◆ No CPU limit and no run limit are specified in the queue (CPULIMIT and
RUNLIMIT) or at job submission (
-c or -W bsub options).
The value of CHUNK_JOB_DURATION is displayed by
bparams -l.
After adding CHUNK_JOB_DURATION to
lsb.params, use badmin reconfig
to reconfigure your cluster.
By default, CHUNK_JOB_DURATION is not enabled.
Restrictions on chunk job queues
CHUNK_JOB_SIZE is ignored and jobs are not chunked for the following queues:
◆ Interactive (INTERACTIVE = ONLY parameter)
◆ CPU limit greater than 30 minutes (CPULIMIT parameter)
If CHUNK_JOB_DURATION is set in
lsb.params, the job is chunked only if
it is submitted with a CPU limit that is less than or equal to the value of
CHUNK_JOB_DURATION (
bsub -c)
◆ Run limit greater than 30 minutes (RUNLIMIT parameter)