User Manual

Driver FeaturesRev 2.1-1.0.6
Mellanox Technologies
102
(the most significant bit indicates if a PKey is a full PKey).
Step 2. Configure (on Dom0) the virtual-to-physical PKey mappings for the VMs.
Step a. Check the PCI ID for the Physical Function and the Virtual Functions.
Step b. Assuming that on Host1, the physical function displayed by lspci is "0000:02:00.0", and that
on Host2 it is "0000:03:00.0"
On Host1 do the following.
Do not touch the Dom0 mapping table (under <nnnn>:<nn>:00.0). Modify only
tables under 0000:02:00.1 and/or 0000:02:00.2. We assume that vm1 uses VF
0000:02:00.1 and vm2 uses VF 0000:02:00.2
Step c. Configure the virtual-to-physical PKey mapping for the VMs.
vm1 pkey index 0 will be mapped to physical pkey-index 1, and vm2 pkey index 0
will be mapped to physical pkey index 2. Both vm1 and vm2 will have their pkey
index 1 mapped to the default pkey.
Step d. On Host2 do the following.
Step e. Once the VMs are running, you can check the VM's virtualized PKey table by doing (on the
vm).
Step 3. Start up the VMs (and bind VFs to them).
Step 4. Configure IP addresses for ib0 on the host and on the guests.
The ",ipoib" causes OpenSM to pre-create IPoIB the broadcast group for the indicated
PKeys.
lspci | grep Mel
cd /sys/class/infiniband/mlx4_0/iov
0000:02:00.0 0000:02:00.1 0000:02:00.2 ...
1
1. 0000:02:00.0 contains the virtual-to-physical mapping tables for the physical func-
tion.
0000:02:00.X contain the virt-to-phys mapping tables for the virtual functions.
echo 0 > 0000:02:00.1/ports/1/pkey_idx/1
echo 1 > 0000:02:00.1/ports/1/pkey_idx/0
echo 0 > 0000:02:00.2/ports/1/pkey_idx/1
echo 2 > 0000:02:00.2/ports/1/pkey_idx/0
cd /sys/class/infiniband/mlx4_0/iov
echo 0 > 0000:03:00.1/ports/1/pkey_idx/1
echo 1 > 0000:03:00.1/ports/1/pkey_idx/0
echo 0 > 0000:03:00.2/ports/1/pkey_idx/1
echo 2 > 0000:03:00.2/ports/1/pkey_idx/0
cat /sys/class/infiniband/mlx4_0/ports/[1,2]/pkeys/[0,1]