User Manual

Driver FeaturesRev 2.0-3.0.0
Mellanox Technologies
92
To configure the GUID at index <n> on port <port_num>:
Example:
Step 3. Read the administrative status of the GUID index.
To read the administrative status of GUID index
m on port n:
Step 4. Check the operational state of a GUID.
The values indicate what gids are actually configured on the firmware/hardware, and all the
entries are R/O.
Step 5. Compare the value you read under the "admin_guids" directory at that index with the value
under the
"gids" directory, to verify the change requested in Step 3 has been accepted by the
SM, and programmed into the hardware port GID table.
If the value under
admin_guids/<m> is different that the value under gids/<m>, the request is
still in progress.
4.13.7.2.3Partitioning IPoIB Communication using PKeys
PKeys are used to partition IPoIB communication between the Virtual Machines and the Dom0
by mapping a non-default full-membership PKey to virtual index 0, and mapping the default
PKey to a virtual pkey index other than zero.
The below describes how to set up two hosts, each with 2 Virtual Machines. Host-1/vm-1 will be
able to communicate via IPoIB only with Host2/vm1,and Host1/vm2 only with Host2/vm2.
In addition, Host1/Dom0 will be able to communicate only with Host2/Dom0 over ib0. vm1 and
vm2 will not be able to communicate with each other, nor with Dom0.
This is done by configuring the virtual-to-physical PKey mappings for all the VMs, such that at
virtual PKey index 0, both vm-1s will have the same pkey and both vm-2s will have the same
PKey (different from the vm-1's), and the Dom0's will have the default pkey (different from the
vm's pkeys at index 0).
OpenSM must be used to configure the physical Pkey tables on both hosts.
• The physical Pkey table on both hosts (Dom0) will be configured by OpenSM to be:
• The vm1's virt-to-physical PKey mapping will be:
cd /sys/class/infiniband/mlx4_0/iov/ports/<port_num>/admin_guids
echo <your desired guid> > n
cd /sys/class/infiniband/mlx4_0/iov/ports/1/admin_guids
echo
1
"0x002fffff8118" > 3
1. echo "0x0" means let the SM assign a value to that GUID
echo "0xffffffffffffffff" means delete that GUID
echo <any other value> means request the SM to assign this GUID to this index
cat /sys/class/infiniband/mlx4_0/iov/ports/<n>/admin_guids/<m>
/sys/class/infiniband/mlx4_0/iov/ports/<n>/gids (where n = 1 or 2)
index 0 = 0xffff
index 1 = 0xb000
index 2 = 0xb030
pkey_idx 0 = 1
pkey_idx 1 = 0