Managing NFS and KRPC Kernel Configurations in HP-UX 11i v2 Contents 1 Introduction ........................................................................................3 2 Managing Kernel Tunables Using Kctune................................................4 2.1 NFS Client Tunables........................................................................................... 4 2.1.1 nfs_async_read_avoidance_enabled ............................................................................... 4 2.
.1.23 2.2 2.3 2.4 nfs3_new_acache .......................................................................................................... 21 NFS Server Tunables ....................................................................................... 23 2.2.1 nfs_portmon .................................................................................................................. 23 2.2.2 nfs_exports_rwlock ........................................................................................
1 Introduction NFS is a network-based application that offers transparent file access across a network. The behavior and performance of NFS depends on numerous kernel tunables. Tunables are variables that control the behavior of the HP-UX kernel. To achieve optimal performance, the system administrator can modify the values of the tunables. This white paper discusses the configurable Network File System (NFS) and Kernel Remote Procedure Call (KRPC) tunables in HP-UX 11i v2 and describes how to manage them.
2 Managing Kernel Tunables Using Kctune 2.1 NFS Client Tunables Table 2.1-1 lists the NFS client tunables. The last column specifies which NFS patch first introduced the tunable. Table 2.
to continue issuing NFS READ calls even when all the biods are busy servicing asynchronous write requests. In many situations this may improve application performance. In some situations with NFSv2 mounts, NFS read performance may regress by enabling this tunable. If application performance is negatively affected with this tunable enabled, set the tunable back to the default value (disabled).
2.1.3 nfs_do_purge_rddir_cache Description The nfs_do_purge_rddir_cache tunable controls how an NFS client manages the readdir cache contents. If an NFS mounted filesystem has a huge directory hierarchy with a large number of files, and this mounted directory is referenced, large numbers of directory cache entries are created. The huge directory cache can build pressure on system memory. When this tunable is enabled, it frees the directory cache. This eases the load on memory.
Modifying the value Enabling this tunable provides lock cancellation interoperability between Solaris and HP-UX 11i v3 servers and HP-UX 11i v2 clients. With Solaris or HP-UX 11i v3 NFS servers, this tunable must be enabled on HP-UX 11i v2 clients. If this tunable is not enabled, the 11i v2 NFS client could incorrectly grab a file lock on a Solaris or HP-UX 11i v3 NFS server. This tunable will not be available in the next release of HP-UX.
Note: The nfs_fine_grain_fs_lock tunable was first introduced with patch PHNE_34550 and accepted only two values: 0 and 1. Beginning with patch PHNE_35118, the tunable is modified to accept three values: 0, 1, and 2. If you set the value to 2, verify that you have installed NFS patch PHNE_36257 for a potential race condition fix (SR:8606477797) Restrictions on changing The nfs_fine_grain_fs_lock tunable is dynamic. System reboot is not required to activate a change made to this tunable.
Note: If there are no NFS mount points in the system, any change made to the value of the tunable is effective immediately. If there are NFS mount points in the system, a system reboot is required for the change to be activated. Modifying the value Enable this tunable so your application experiences better performance when working with locked files on NFS filesystems. This tunable is not available in the next release of HP-UX.
Tested values Default: 0 Min: 0 Max: 40000 Note: If the value of nfs_nrnode tunable is set to default, the value displayed is 0. However, the actual value is that of the ncsize tunable. If the tunable is set to a value greater than 40000, an informational warning is issued at boot time. This value is outside the tested limit. Restrictions on changing The nfs_nrnode tunable is static. System reboot is required to activate a change made to this tunable.
This tunable is not available in the next release of HP-UX. Starting with the HP-UX 11i v3 release, conformity with POSIX standards is the default behavior and is not user-configurable. 2.1.10 nfs_wakeup_one Description The nfs_wakeup_one tunable enables you to specify whether the NFS client activates all NFS threads or only a single NFS thread waiting for the same rnode lock.
2.1.11 nfs2_do_symlink_cache Description The nfs2_do_symlink_cache tunable caches the contents of symbolic links in NFSv2 mounted filesystems. If the server changes the contents of a symbolic link, and if either the time stamps are not updated or the granularity of the time stamp is too large, then the changes become visible to the client after a long time interval. The nfs3_do_symlink_cache tunable caches the contents of symbolic links in NFSv3 mounted filesystems.
successful NFS I/O. If packets are not being dropped in the network, disabling this functionality results in increased throughput. However, if the server response is delayed or the network is overloaded, the number of timeouts can increase. HP recommends leaving this tunable enabled because it helps the system minimize NFS packet loss on congested networks. 2.1.
Modifying the value If biod processes are running on the client, enable this tunable to avoid unnecessary cache purging for sequential read, write, and lseek operations on an NFS-mounted file. If the cached data is purged unnecessarily, more over-the-wire (OTW) calls are needed. Enabling this tunable can improve the performance of the NFS client. Disable this tunable to purge the cache for a sequential read, write, and lseek operation, resulting in more OTW calls.
Tested values Default: 0 (Dynamic retransmission is disabled) Min: 0 Max: 1 (Dynamic retransmission is enabled) Restrictions on changing The nfs3_dynamic tunable is dynamic. System reboot is not required to activate changes made to this tunable. However, the dynamic retransmission feature is set per filesystem at mount time. You must unmount and re-mount each filesystem after changing this tunable. Only NFSv3 mount points are affected when you change the value of this tunable.
Modifying the value Certain NFS servers provide 64bit inode numbers for NFS clients. The NFS clients need to be capable enough to handle 64bit inode numbers. Enable this tunable to support 64bit inode numbers for NFSv3 mounted filesystems. The support for 64bit inode numbers is applicable to only NFSv3 mounted filesystems. This feature is not applicable to NFSv2 mounted filesystems, AutoFS, and CacheFS filesystems. This tunable is not available in the next release of HP-UX.
Note: If there are no NFS mount points in the system, any change made to the value of the tunable is effective immediately. If there are NFS mount points in the system, a system reboot is required for the change to be activated. Modifying the value Modifying the value of this tunable might reduce the time required to search for an entry in the access cache. For example, if the hash buckets contain numerous entries and you increase the number of buckets then the speed increases.
Restrictions on changing The nfs3_jukebox_delay tunable is dynamic. System reboot is not required to activate a change made to this tunable. Only NFSv3 mount points are affected when you change the value of this tunable. Modifying the value If it takes a considerable amount of time for files to migrate from your HSM storage devices, increase the value of this tunable. However, if you increase the value of the tunable, it can prevent the file from becoming immediately visible when it becomes available.
To decrease the size of NFSv3 TCP requests, decrease the value of the nfs3_max_transfer_size_cots tunable. For example, to decrease the size of I/O requests on all NFSv3 TCP filesystems to 8 KB, set the value of nfs3_max_transfer_size_cots to 8192. For NFS/UDP Filesystems: To increase the size of NFSv3 UDP requests, set the nfs3_max_transfer_size and nfs3_max_transfer_size_clts tunables to the same value. Otherwise, the transfer size defaults to the smaller value of these two tunables.
Tested values Default: 32768 Min: 4096 Max: 32768 Note: If the tunable is set to a value less than 4096 or greater than 32768, an informational warning is issued at runtime. These values are outside the tested limits. Restrictions on changing The nfs3_max_transfer_size_clts tunable is dynamic. System reboot is not required to activate changes made to this tunable. However, the transfer size for a filesystem is set when the filesystem is mounted.
Note: The nfs3_max_transfer_size_clts tunable affects every NFSv3 filesystem. To control the transfer sizes of specific NFS filesystems, use the rsize and wsize mount options. Refer to the mount_nfs(1M) manpage for more information. 2.1.22 nfs3_max_transfer_size_cots Description The nfs3_max_transfer_size_cots tunable specifies the maximum size of the data portion of NFSv3 READ, WRITE, READDIR, and READDIRPLUS requests.
Caution: If the NFS client is experiencing NFS READ failures and the system is reporting "NFS read failed for server : RPC: Can’t decode result" errors, this is an indication that the nfs3_max_transfer_size, nfs3_max_transfer_size_clts, or nfs3_max_transfer_size_cots tunable value was changed while NFS filesystems were mounted. You must unmount and re-mount the NFS filesystem to use the new value. Note: The nfs3_max_transfer_size_cots tunable affects every NFSv3 filesystem.
This tunable is not available in the next release of HP-UX. Starting with the HP-UX 11i v3 release, the hashed search algorithm is used by default and this tunable is not user-configurable.
2.2 NFS Server Tunables Table 2.2-1 lists the NFS server tunables. The last column specifies which NFS patch first introduced the tunable. Table 2.2-1 NFS Server Tunables Kctune Tunable Name Range Default Value Units Patch ID nfs_portmon 0 or 1 0 Boolean PHNE_33982 nfs_exportfs_rwlock* 0 or 1 0 Boolean PHNE_34550 nfs_srv_read_copy_avoid 0 or 1 0 Boolean PHNE_35118 nfs3_do_readdirplus 0 or 1 1 Boolean PHNE_33982 * This tunable is obsolete after the HP-UX 11i v2 release. 2.2.
Tested values Default: 0 (Hashed reader and writer spinlocks are disabled. exportfs lock is used) Min: 0 Max: 1 (Hashed reader and writer spinlocks are enabled) Restrictions on changing The nfs_exportfs_rwlock tunable is dynamic. System reboot is not required to activate a change made to this tunable. Note: If there are no exported filesystems in the system, any change made to the value of the tunable is effective immediately.
Restrictions on changing The nfs_srv_read_copy_avoid tunable is dynamic. System reboot is not required to activate a change made to this tunable. Modifying the value Enable this tunable to improve the read performance of NFS. 2.2.4 nfs3_do_readdirplus Description The nfs3_do_readdirplus tunable controls the NFSv3 READDIRPLUS functionality on the NFS server. In NFS-mounted filesystems, performance of some applications that read huge directories suffers if they use READDIRPLUS instead of READDIR.
2.3 KRPC Client Tunables Table 2.3-1 lists the KRPC client tunables. The last column specifies which NFS patch first introduced the tunable. Table 2.3-1 KRPC Client Tunables Kctune Tunable Name Range 31 rpc_clnt_idle_timeout 0 to 2 -1 rpc_clnt_max_conns 1 to 231 -1 Default Value Units Patch ID 300000 Milliseconds PHNE_33981 1 Connections PHNE_33981 2.3.
more than one TCP connection is used to communicate with the NFS server. This tunable is referred to as clnt_max_conns in the Solaris environment. Tested values Default: 1 Min: 1 Max: 10 Note: If the tunable is set to a value greater than 10 connections, an informational warning is issued at runtime. This value is outside the tested limits. Restrictions on changing The rpc_clnt_max_conns tunable is dynamic. System reboot is not required to activate a change made to this tunable.
2.4 KRPC Server Tunables Table 2.4-1 lists the KRPC server tunables. The last column specifies which NFS patch first introduced the tunable. Table 2.
2.4.2 rpc_svc_cotsmaxdupreqs Description The rpc_svc_cotsmaxdupreqs tunable controls the size of the duplicate request cache that detects RPC level transmissions on connection-oriented transports such as TCP. This cache avoids processing of retransmitted requests that might be non-idempotent. Tested values Default: 1024 Min: 1 Max: 2048 Note: If the tunable is set to a value greater than 2048, an informational warning is issued at runtime. This value is outside the tested limits.
Note: If the tunable is set to a value less than 1000 milliseconds or greater than 600000 milliseconds, an informational warning is issued at runtime. These values are outside the tested limits. Restrictions on changing The rpc_svc_idle_timeout tunable is dynamic. System reboot is not required to activate a change made to this tunable. Changes made to this tunable are applicable for TCP connections that are created after the value of this tunable is modified.
respond to NFS requests using a different IP address than the client’s request originally used. For example, on an HA/NFS server configured with an NFS package, the package will use a virtual IP address. When a client sends an NFS request to the package (i.e. virtual) IP address, the server is not required to send the reply back to the client using the same virtual IP address. The server may substitute the real IP address configured on the physical LAN interface in place of the virtual IP address.
2.4.6 rpc_svc_nfs_cpu_bind_method Description The rpc_svc_nfs_cpu_bind_method tunable specifies whether NFS/TCP service threads are bound to specific CPUs or can bind to any available CPU. When enabled, the server attempts to bind the NFS service threads to the CPU responsible for servicing the interrupt of the network card on which the NFS request arrived.
Note: For overall performance improvement, HP recommends you uniformly distribute interrupt lines from all the network interface cards across the available CPUs on the system. For details on how to configure interrupt lines, see intctl (1M).
Appendix A Recommended Tunable Values This section contains a list of tunable parameter settings that have been shown to provide improved performance and behavior on most HP-UX 11i v2 NFS systems. All of the tunables in Table A.1 have been obsolesced in HP-UX 11i v3. In order to configure an 11i v2 NFS system to behave similar to an 11i v3 system, set the kctune parameters listed in Table A.1 to their respective recommended value. Table A.
Appendix B Migrating from adb to kctune adb is a tool system administrators can use to tune the HP-UX kernel. However, using adb has several disadvantages when compared to kctune. First, any changes made to kernel tunables via adb do not persist through kernel rebuilds and kernel patch installations. In fact, changes made only to the running kernel memory via adb do not persist across a system reboot.
clnt_idle_timeout rpc_clnt_idle_timeout clnt_max_conns rpc_clnt_max_conns maxdupreqs rpc_svc_cltsmaxdupreqs cotsmaxdupreqs rpc_svc_cotsmaxdupreqs svc_idle_timeout rpc_svc_idle_timeout maxthreads rpc_svc_maxthreads* *This tunable is obsolete after the HP-UX 11i v2 release.
© Copyright 2008, 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. The only warranties for HP products and services are set forth in the express warranty statements accompanying such products and services. Nothing herein should be construed as constituting an additional warranty. HP shall not be liable for technical or editorial errors or omissions contained herein. Linux is a U.S. registered trademark of Linus Torvalds.