User Manual

Rev 2.1-1.0.6

Mellanox Technologies

For the changes in openib.conf to take effect, run:

/etc/init.d/openibd restart

4.1.2.5 Multiple Connections from Initiator InfiniBand Port to the Target

Some system configurations may need multiple SRP connections from the SRP Initiator to the

same SRP Target: to the same Target IB port, or to different IB ports on the same Target HCA.

In case of a single Target IB port, i.e., SRP connections use the same path, the configuration is

enabled using a different initiator_ext value for each SRP connection. The initiator_ext value is a

16-hexadecimal-digit value specified in the connection command.

Also in case of two physical connections (i.e., network paths) from a single initiator IB port to

two different IB ports on the same Target HCA, there is need for a different initiator_ext value on

each path. The conventions is to use the Target port GUID as the initiator_ext value for the rele-

vant path.

If you use srp_daemon with -n flag, it automatically assigns initiator_ext values according to this

convention. For example:

Notes:

1. It is recommended to use the -n flag for all srp_daemon invocations.

2. ibsrpdm does not have a corresponding option.

3. srp_daemon.sh always uses the -n option (whether invoked manually by the user, or automat-

ically at startup by setting SRP_DAEMON_ENABLE to yes).

4.1.2.6 High Availability (HA)

Overview

High Availability works using the Device-Mapper (DM) multipath and the SRP daemon. Each

initiator is connected to the same target from several ports/HCAs. The DM multipath is responsi-

ble for joining together different paths to the same target and for fail-over between paths when

one of them goes offline. Multipath will be executed on newly joined SCSI devices.

Each initiator should execute several instances of the SRP daemon, one for each port. At startup,

each SRP daemon detects the SRP Targets in the fabric and sends requests to the ib_srp module

to connect to each of them. These SRP daemons also detect targets that subsequently join the fab-

ric, and send the ib_srp module requests to connect to them as well.

Operation

When a path (from port1) to a target fails, the ib_srp module starts an error recovery process. If

this process gets to the reset_host stage and there is no path to the target from this port, ib_srp

will remove this scsi_host. After the scsi_host is removed, multipath switches to another path to

this target (from another port/HCA).

When the failed path recovers, it will be detected by the SRP daemon. The SRP daemon will then

request ib_srp to connect to this target. Once the connection is up, there will be a new scsi_host

for this target. Multipath will be executed on the devices of this host, returning to the original

state (prior to the failed path).

id_ext=200500A0B81146A1,ioc_guid=0002c90200402bec,\

dgid=fe800000000000000002c90200402bed,pkey=ffff,\

service_id=200500a0b81146a1,initiator_ext=ed2b400002c90200