Service Manual

Installation Guide 23

Configuring IPoIB

Ensure that there is a subnet manager running on the InfiniBand network.

This can either be the embedded subnet manager on the switch or Open

Subnet Manager (OpenSM) that is packaged with WinOF.

If you decide to run the subnet manager on the head node, ensure that the

subnet manager on the switch is disabled, then click

Start



Administrative Tools



Services

and start the OpenSM service

on the head node.

To set the compute nodes IP addresses automatically, ensure that

DHCP service is enabled and running on the head node.

Also ensure that the head node is connected to the InfiniBand fabric.

Use the

HPC Cluster Manager

to select the desired network topology.

NOTE: The recommended network topology is topology 3, where the compute

nodes are isolated on private and application networks.

Enter the IP address range using the

Network Configuration

wizard.

NOTE: If the InfiniBand network is not discovered by the HPC Cluster Manager,

close the application and restart the HPC Management Service by clicking

StartAdministrative Tools Services. Run the HPC Cluster Manager again.

NOTE: After configuring the topology, it may be necessary to restart the DHCP

Server so that the head node will assign IP addresses to the compute nodes

through the application network. You can restart the DHCP service using the

Services window, by clicking StartAdministrative ToolsServices.

You may have to wait for a few minutes for DHCP to assign the addresses.

Installing Network Direct

By default, once the InfiniBand drivers are installed, IPoIB is configured but

Network Direct is not enabled.

In order to get the maximum performance from the InfiniBand fabric,

you must enable the Network Direct Interface. To do so, run the following

command on the head node and compute nodes (using clusrun):

ndinstall -i

book.book Page 23 Monday, September 20, 2010 1:36 PM