Designing a High Performance Network File Server

It is extremely important to ensure the configuration of the APA link aggregates match the

configuration of the link partner (network switch or server). In particular, the server and switch should

specify the same trunking protocol and use the same load balancing algorithm. Proper configuration

is critical to ensure that the inbound traffic to the NFS server is evenly balanced across all the ports in

the APA link aggregates. Refer to your specific network switch documentation for information on how

to properly configure the link aggregation trunks on your switch.

Bind nfsktcpd Service Threads to the CPUs Processing Interrupts

By default, nfsktcpd threads are able to execute on any CPU in the system and they can migrate to

other CPUs at any time. While this behavior works fine in most environments, it can result in

decreased throughput in environments where the nfsktcpd threads are migrating to CPUs in other

cells, resulting in cell-to-cell memory transfers.

If the newly introduced rpc_svc_nfs_cpu_bind_method kernel parameter is enabled (1), when the

server receives an inbound NFS/TCP request it will assign that request to an nfsktcpd thread which

has been bound to the same CPU that is servicing the interrupt of the network interface where the

request arrived. This allows a single CPU to process the entire NFS request from the time it arrives on

the server’s network interface until the outbound reply is sent. Also, by keeping nfsktcpd threads

bound to the same CPU for the life of the thread, it guarantees the same pool of nfsktcpd threads will

be used to service consecutive requests from the same client systems and those requests will continue

to be processed on the same CPU instead of migrating to other CPUs, potentially in other cells.

Again, in most environments these precautions may not be necessary. However, in the case of very

large NFS servers configured with many CPUs and multiple network interface cards, the ability to

bind specific nfsktcpd threads to specific CPUs (associated with specific network interface cards) can

result in a huge performance gain, as it did in this engagement.

lan900

lan901

ure 5 – APA Ports Confined to a Sin

le I/O Cell