Installation guide

Release Notes
The modulefiles have pathnames /opt/scyld/modulefiles/openmpi/compiler/version, where version is a file that
amends $PATH, $LD_LIBRARY_PATH, and $MANPATH with pathnames that point into the associated compiler-specific
/opt/scyld/openmpi/version/compiler/ subdirectories.
Supporting only one openmpi-scyld version on the master node tends to cause problems because OpenMPI applications
linked against an earlier version may break when a new version updates and replaces the old, until the applications are
rebuilt against the new version. If a new openmpi-scyld version is installed to co-exist with previous version, vs. updated
to replace earlier versions, then (for example) the default module load openmpi/gnu references the newest version, and
the version-specific module load openmpi/gnu/1.5 references the older openmpi-scyld version 1.5, thereby allowing for a
grace period for users to continue to execute applications that are linked to an older version, without needing to immediately
rebuild every application to work with the new version. Version 1.5.1 (and beyond) can co-exist with earlier versions,
although co-existence is problematic:
• Version 1.5 libraries, executable binaries, and manpages reside in directories /opt/scyld/openmpi/gnu/, etc.,
whereas version 1.5.1 files reside in directories /opt/scyld/openmpi/1.5.1/gnu/, etc. This causes no problems
with co-existence.
However, version 1.5 modulefiles reside in (for example) file /opt/scyld/modulefiles/openmpi/gnu, whereas ver-
sion 1.5.1 (and beyond) modulefiles use those same pathnames as directories, and use version-specific names for the
modulefiles themselves, e.g., /opt/scyld/modulefiles/openmpi/gnu/1.5.1. If and when version 1.5.1 installs to
co-exist with version 1.5, it silently transforms the preexisting version 1.5 modulefiles into the same directory and file
paradigm employed by version 1.5.1. In other words, the co-existing gnu modulefiles become:
/opt/scyld/modulefiles/openmpi/gnu/1.5
/opt/scyld/modulefiles/openmpi/gnu/1.5.1
• Version 1.4.3 (and earlier) uses libraries located in /usr/lib64/OMPI/compiler/, executable binaries located in
/usr/openmpi/compiler/bin/, and manpages located in /usr/openmpi/man/. While version 1.5.1 can co-exist
with 1.4.3, in order to employ module load openmpi/gnu/1.4.3 the cluster administrator must manually create version
1.4.3 modulefiles as (for example) files:
/opt/scyld/modulefiles/openmpi/gnu/1.4.3
following the same paradigm employed by the 1.5.1 modulefiles.
Issues with Spanning Tree Protocol and portfast
Network switches with Spanning Tree Protocol (STP) enabled will block packets received on a port for the first 30 seconds
after the port comes online, giving the switch and the Spanning Tree algorithm time to determine if the device on the new
link is a switch, and to determine if Spanning Tree will block or forward packets from this port. This is done to prevent
"loops" which can cause packets to be endlessly repeated at a high rate and consume all network bandwidth. Each time
the link goes down and comes back up, another 30-second blocking delay occurs. This delay can prevent PXE/DHCP from
obtaining an IP address, or can prevent the node’s initial kernel from downloading its initial root filesystem, which results in
the node endlessly iterating in the early boot sequence, or can delay the node’s ongoing filecache provisioning of libraries
to the node.
We recommend disabling STP if feasible. If not feasible, then we recommend reconfiguring the switch to use
Rapid STP or portfast, which avoids the 30-second delay, or employing some other port mode that will forward
packets as a port comes up. There is no generic procedure for enabling these options. For Cisco switches, see
http://www.cisco.com/en/US/products/hw/switches/ps700/products_tech_note09186a00800b1500.shtml. For other switch
models, see the model-specific documentation.
21