Product specifications

QLogic OFED+ Host Software Page 19 of 20
The following error may occur on rare occasions with Intel MPI/uDAPL:
unexpected DAPL event 4008 from ...
This problem is caused by a limitation in Intel MPI/uDAPL's dynamic connection
mechanism when MPI processes are not paying sufficient attention to incoming
interconnect traffic. To work around this problem, add the following to the mpirun
command:
genv I_MPI_USE_DYNAMIC_CONNECTIONS 0
An SRP write performance bug between QLE7240/QLE7280 and LSI (Engenio)
storage systems was fixed. To apply this fix, modify the srp_sg_tablesize and
max_sect parameters. To modify srp_sg_tablesize, do one of the following:
o Add the following line to /etc/modprobe.conf:
options ib_srp srp_sg_tablesize=80
o Define srp_sg_tablesize when adding the ib_srp module (as root):
modprobe ib_srp srp_sg_tablesize=80
o If using srp_daemon, modify max_sect by adding the following command to
/etc/srp_daemon.conf file:
a id_ext=200500A0B81146A1,ioc_guid=00a0b80200402bef,max_sect=2048
To ensure proper operation of MPI, the QLogic OFED+ INSTALL sets the memory
locking limits in the /etc/security/limits.conf file to unlimited, due to the large
amounts of memory needed to run the MPIs. Open MPI recommends using unlimited
ulimits for lockable memory. For more information, see:
http://www.open-mpi.org/faq/?category=openfabrics#ib-locked-pages
An alternative is to use the --mca mpi_leave_pinned 1 option with mpirun.
When users increase the number of processes beyond the number of open files
allowed by ulimit, mpirun prints an error message. The ulimit for the number of open
files is typically 1024 on both Red Hat and SLES. The message will look similar to the
following:
MPIRUN.up001: Warning: ulimit for the number of open files is only
1024, but this mpirun request requires at least number of files open
files (sockets). The shell ulimit for open files needs to be increased.
This is due to limit:
descriptors 1024
The ulimit can be increased; QLogic recommends an increase of approximately 20
percent over the number of CPUs. For example, in the case of 2048 CPUs, ulimit
could be increased to 2500: ulimit -n 2500. Increase the ulimit only on the host
where mpirun was started, unless the mode of operation allows mpirun from any
node.