User Manual

Features Overview and ConfigurationRev 2.3-1.0.1
Mellanox Technologies
182
3.4.1.6.3 Multi-GUID Support in InfiniBand
As of MLNX_OFED v2.2-1.0.0, Infiniband VFs in SR-IOV setting can have more than a single
GUID to be used for their purposes. Totally there are 128 GUIDs per port, where the PF occupies
2 entries, and the remaining GUIDs are divided equally between all the VFs. If there are any
remainders, those GUIDs are given to the VFs with the lowest IDs.
For example in case that are 5 VFs, the division will be as follows: PF – 2, slave 1 will have 26
and last 4 slaves will have 25. (2 + 1 * 26 + 4 * 25 = 128).
To find the mapping between VF's GUID entry to its physical one, use the sysfs mechanism
<gid_idx> directory.
The example below shows the mapping between “entry 0” of
<PCI FUNCTION #1> to its physical
one on port number 1.
cat /sys/class/infiniband/mlx4_0/iov/<PCI FUNCTION #1>/ports/1/gid_idx/0
Initial GUIDs' values depend on the mlx4_ib module parameter 'sm_guid_assign' as follows:
Mode Type Description
sm assigned Asks SM for values for GUID entry 0 per VF. Other entries will have
value 0 in the port GUID table, and ffffffffffffffff under their matching
admin_guids entry.
This is the default mode (value=1).
For example, as GUID entry 1 is not the base gid index of the HYP, upon
startup it will have the below values under the sysfs entries:
cat /sys/class/infiniband/mlx4_0/iov/ports/1/admin_guids/1
ffffffffffffffff
cat /sys/class/infiniband/mlx4_0/iov/ports/1/gids/1
fe80:0000:0000:0000:0000:0000:0000:0000
admin assigned Once sm_guid_assign is set to 0, the driver works in admin assigned
mode which result in having only valid value for GUID entry 0 for the
PF. All other entries under the port will be 0 and corresponding
admin_guids will be ffffffffffffffff.
Port Up/Down
When moving to multi-guid mode, the port is assumed to be up when the base GUID index per
VF/PF (entry 0) has a valid value. Setting other GUID entries does not af
fect the port status.
However, any change in a GUID will cause a GUID change event for its VF/PF even if it is not
the base one.
Persistency Support
Once admin request is rejected by the SM, a retry mechanism is set. Retry time is set to 1 second,
and for each retry it is multiplied by 2 until reaching the maximum value of 60 seconds.
Addi-
tionally, when looking for the next record to be updated, the record having the lowest time to be
executed is chosen.
Any value reset via the
admin_guid interface is immediately executed and it resets this entry
timer.