LSF Version 7.3 - Using Platform LSF HPC

Submitting IBM POE Jobs over InfiniBand
Platform LSF installation adds a shared nrt_windows resource to run and monitor POE
jobs over the InfiniBand interconnect.
Begin Resource
RESOURCENAME TYPE INTERVAL INCREASING DESCRIPTION
...
poe Numeric 30 N (poe availability)
dedicated_tasks Numeric () Y (running dedicated
tasks)
ip_tasks Numeric () Y (running IP tasks)
us_tasks Numeric () Y (running US tasks)
nrt_windows Numeric 30 N (free nrt windows on
IBM poe over IB)
...
End Resource
Begin ResourceMap
RESOURCENAME LOCATION
poe [default]
nrt_windows [default]
dedicated_tasks (0@[default])
ip_tasks (0@[default])
us_tasks (0@[default])
End ResourceMap
Job Submission
Run bsub -a poe to submit an IP mode job:
bsub -a poe mpirun.lsf job job_options -euilib ip poe_options
Run bsub -a poe to submit a US mode job:
bsub -a poe mpirun.lsf job job_options -euilib us poe_options
If some of the AIX hosts do not have InfiniBand support (for example, hosts that still
use HPS), you must explicitly tell LSF to exclude those hosts:
bsub -a poe -R "select[nrt_windows>0]" mpirun.lsf job job_options poe_options
Job monitoring
Run lsload to display the nrt_windows and poe resources:
lsload -l
HOST_NAME status r15s r1m r15m ut pg io ls it tmp swp mem nrt_windows poe
hostA ok 0.0 0.0 0.0 1% 8.1 4 1 0 1008M 4090M 6976M 128.0 1.0
hostB ok 0.0 0.0 0.0 0% 0.7 1 0 0 1006M 4092M 7004M 128.0 1.0