HP Insight Cluster Management Utility v7.2 User Guide

5.8.4 Scaling out an HP Insight CMU diskless solution with multiple NFS servers
By default, the HP Insight CMU diskless support configures the HP Insight CMU management node
as the NFS server that will serve the diskless image, regardless of the diskless implementation
method. HP recommends that a single NFS server can support up to ~200 diskless clients over a
1Gb ethernet management network, and an NFS server with a 10Gb ethernet management network
can support up to ~400 diskless clients. These are recommendations based on bootup tests. Users
should factor in their expected usage of the diskless solution and adjust these numbers accordingly.
In any case, to scale beyond these numbers means that you need more NFS servers to serve the
additional diskless clients and distribute the network traffic among multiple network switches.
HP Insight CMU diskless support includes support for configuring additional NFS servers and
assigning groups of compute nodes to these NFS servers.
To build a large-scale diskless cluster with HP Insight CMU:
1. Determine the number of nodes per NFS server and identify the NFS server nodes. HP
recommends:
• Not more than 200 nodes per NFS server over a 1 Gb network and not more than 400
over a 10 Gb netowrk.
• 4GB of (non-SATA) storage per compute node on each NFS server
These recommendations are sufficient for booting and serving the OS. Also, these
recommendations are for a cluster that includes a high-performance cluster-wide file system
and/or a local scratch disk for the user workload.
When choosing the NFS server nodes, factor in the network topology of the cluster. Make
sure that the compute nodes have uncongested access to the NFS server. Ideally, each NFS
server is on the same switch as all of the compute nodes it serves.
2. Install the NFS server nodes with the selected Linux distribution, and verify that the NFS server
package is installed and configured to start at bootup.
On Red Hat
# chkconfig nfs on
On SLES
# chkconfig nfsserver on
3. Ensure that enough NFS daemons and threads are configured to handle the anticipated volume
of NFS traffic.
On Red Hat
Set RPCNFSDCOUNT in the /etc/sysconfig/nfs file to the requested number of NFS
daemons. By default, RPCNFSDCOUNT=8.
On SLES
Set USE_KERNEL_NFSD_NUMBER in the /etc/sysconfig/nfs file, which defaults to 4.
HP recommends that the setting be at least half of the maximum number of compute nodes
served by each NFS server, and is typically a multiple of the total number of CPUs on the NFS
server. For more information and tips on tuning NFS, see NFS documentation.
4. HP recommends that you install one of these nodes, use HP Insight CMU to take a backup of
the node, and then clone this image to all of the NFS servers, including the node that was
initially installed. This approach ensures that all of the NFS servers are consistent with the
same /etc/ hosts file and with passwordless ssh configured for the root account. This is
required for the rest of the setup to succeed.
5. Scalable diskless support in HP Insight CMU is based on the presence of a single configuration
file: /opt/cmu/etc/cmu_diskless_nfs_servers. The existence and content of this file
82 Provisioning a cluster with HP Insight CMU