Matrix File Serving Solution Pack for Linux Administration Guide Matrix File Serving Solution Pack for Linux V3.1.
Copyright © 2005-2006 PolyServe, Inc. Use, reproduction and distribution of this document and the software it describes are subject to the terms of the software license agreement distributed with the product (“License Agreement”). Any use, reproduction, or distribution of this document or the described software not explicitly permitted pursuant to the License Agreement is strictly prohibited unless prior written permission from PolyServe has been received.
Contents 1 Introduction MxFS Features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . MxFS Concepts and Definitions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Supported NFS Versions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . MxFS Tested Configuration Limits . . . . . . . . . . . . . . . . . . . . . . . . . . . . RPC Program Usage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Contents iv Disable an Export Group. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Enable an Export Group . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Clear an Error Associated with an Export Group . . . . . . . . . . . . Delete an Export Group. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30 30 30 30 3 Configure Virtual NFS Services Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
1 Introduction Matrix Server and the Matrix File Serving Solution Pack for Linux (MxFS-Linux, or MxFS) provide scalability and high availability for the Network File System (NFS), which is commonly used on UNIX and Linux systems to share files remotely. MxFS Features MxFS provides the following features: • Scalable NFS client connectivity.
Chapter 1: Introduction 2 • Incremental NFS Server scalability. New physical NFS servers can be added to an existing file-serving cluster to serve the same filesystems for a set of new clients without disturbing the connectivity and operation of other NFS clients and servers that may be sharing those filesystems. • High availability for NFS clients. MxFS supports continuous NFS client operation across a hardware or software failure that inhibits NFS service from a given cluster server.
Chapter 1: Introduction 3 • Cluster-wide consistent user authentication. A user and NFS client may always be authenticated, by any NFS server in the cluster, as the same user and NFS client (assuming that an organizational authentication service such as LDAP has been deployed). • Cluster-wide, connection-oriented load balancing.
Chapter 1: Introduction 4 Export Records An export record is equivalent, indeed exactly equal to, the individual records contained in the /etc/exports file of a traditional NFS server. The format of the record and the options available are precisely the same in form and function.
Chapter 1: Introduction 5 ordered set of backup nodes to host the Virtual NFS Service in case of failover. Finally, the administrator selects one Export Group to be associated with the Virtual NFS Service. This Export Group defines which exports will be available via NFS for this Virtual NFS Service. It is possible to create multiple Virtual NFS Services and associate the same Export Group with each one.
Chapter 1: Introduction monitoring process as the Virtual NFS Services come and go from the cluster node. Program number 300277 has been assigned officially by the RPC numbering registrar. Copyright © 1999-2006 PolyServe, Inc. All rights reserved.
2 Configure Export Groups An Export Group describes a set of PSFS filesystems to be exported. It also specifies the Virtual NFS Services that will provide virtual IP addresses that clients use to access those filesystems. Overview Export Records and Groups The PSFS filesystems to be exported are described in export records, which are similar to the entries in an /etc/exports file.
Chapter 2: Configure Export Groups 8 Service; they do not need to know which physical node is providing the service. Clients can access any of the filesystems specified in the Export Group by using the DNS name or IP address of the Virtual NFS Service. When you create a Virtual NFS Service, you will need to assign it to a primary node and one or more backup nodes. A Virtual NFS Service can be associated with only one Export Group.
Chapter 2: Configure Export Groups 9 options, it is indeterminate which options the NFS service will assume. It is therefore recommended that any given filesystem subtree be exported by at most one Export Group. • Although an Export Group can specify that only certain nodes can export its filesystems, the contents of the Export Group are available via the /etc/exports file on every node in the cluster.
Chapter 2: Configure Export Groups 10 Considerations for Ensuring Data Safety NFS write performance and the risk of data loss during NFS write operations are affected by the use of the “sync” and “async” options.
Chapter 2: Configure Export Groups 11 Add an Export Group To create an Export Group, select File Serving > Add Export Group. You can also right-click on an existing Virtual NFS Service that is not currently assigned to an Export Group and select Add Export Group. Name: Type a name for this Export Group. The name must be unique among all of the Export Groups in the cluster and cannot include spaces.
Chapter 2: Configure Export Groups 12 Click Apply to save changes either since the last use of Apply or since the dialog was opened. When you click Apply, the dialog remains open. Clicking OK also saves changes, but closes the dialog. NFS Exports Tab This tab is used to configure the export records for the Export Group. An export record specifies a PSFS filesystem or directory to be exported, the NFS clients allowed to mount it, and the export options for the client.
Chapter 2: Configure Export Groups 13 Create Export Records To add export records directly to the NFS Exports tab, click on Insert. A row will then open up on the NFS Exports tab. You can type the record information directly on the row, or you can click the Edit button at the end of the row to display the Export Record Details dialog, which simplifies constructing the record.
Chapter 2: Configure Export Groups 14 Linux exports(5) man page. All options are supported, and comments are allowed. If you click the Edit button, the Export Record Details dialog appears. This dialog includes a Preview line at the bottom that shows the export record as you construct it. Following is an example of a completed export record. Copyright © 1999-2006 PolyServe, Inc. All rights reserved.
Chapter 2: Configure Export Groups 15 Exported Path. Specify the filesystem or directory to be exported. Client Names: Click Add to specify a client to be allowed to mount the exported filesystem. Then, on the Add NFS Client dialog, identify the client by either its Netmask, IP address, or FQDN. Optionally, you can enter an asterisk (*) to specify all clients. Options. Select any options that apply to this record. The default options are selected initially and do not appear on the Preview line.
Chapter 2: Configure Export Groups 16 Virtual NFS Services Tab On this tab, assign one or more Virtual NFS Services to the Export Group. (A Virtual NFS Service can be associated with only one Export Group; however, multiple Virtual NFS Services can be assigned to a single Export Group.) The Available column on the Virtual NFS Services tab lists the Virtual NFS Services that are not currently associated with an Export Group.
Chapter 2: Configure Export Groups 17 with the other servers as backups. See “Create a Spanning Set of Virtual NFS Services” on page 17 for more information. When you have completed your entries on the Add Export Group window, you can click Apply or OK to create the Export Group, or you can configure the advanced options as described under “Advanced Options for Export Groups” on page 22. (The advanced options are for the high-availability monitor associated with the Export Group.
Chapter 2: Configure Export Groups 18 You can enter the DNS name or IP address for each Virtual NFS Service directly on the dialog, or you can specify a range of virtual IP addresses to be used for the Virtual NFS Services. To specify a range, click the button “Automatically create range of VNFS addresses.” Then specify the range on the dialog that appears next. Copyright © 1999-2006 PolyServe, Inc. All rights reserved.
Chapter 2: Configure Export Groups 19 When you click OK, the IP addresses will be filled in on the “Create a Spanning Set of Virtual NFS Services” dialog. After specifying the DNS names or IP addresses for the Virtual NFS Services, you can set the policy, which determines the failback action that the Virtual NFS Services will take following a failover to a backup node. The options are: • AUTOFAILBACK.
Chapter 2: Configure Export Groups 20 page 22. (The advanced options are for the high-availability monitor associated with the Export Group.) Export Group High-Availability Monitor After an Export Group is created, MxFS starts a high-availability monitor on each node associated with the Export Group. This monitor is active on those nodes and watches the NFS processes running on them.
Chapter 2: Configure Export Groups 21 How the High-Availability Monitor Works When configuring the high-availability monitor associated with an Export Group, you should be aware of the actions taken by the monitor. During each probe cycle, the monitor performs a series of checks: 1. The monitor first checks basic NFS Server health by issuing a NULL RPC call to the NFS Server on the local node. If this call fails, the Export Group is considered to be DOWN. 2.
Chapter 2: Configure Export Groups 22 cannot find the export path. A deleted export path should be either recreated or removed from the Export Group at the administrator’s earliest convenience. Advanced Options for Export Groups The advanced options allow you to tailor the configuration of the high-availability monitor associated with an Export Group.
Chapter 2: Configure Export Groups 23 failovers of the Virtual NFS Service. If you see false failovers, increase the timeout in 30-second increments until the false failovers stop. Also note that the timeout should be increased for Export Groups containing more than a few exported paths. A greater timeout value ensures that the probe has adequate time to verify that each exported path in the Export Group is available and is mounted on the PSFS filesystem.
Chapter 2: Configure Export Groups 24 The default policies (AUTOFAILBACK for the Virtual NFS Service and AUTORECOVER for the high-availability monitor) cause Matrix Server to fail over the associated Virtual NFS Services to a backup network interface on another node when the monitor probe fails. When the NFS service is restored on the original node, Matrix Server fails back the Virtual NFS Services to the network interfaces on the original node.
Chapter 2: Configure Export Groups 25 Scripts The Scripts tab lets you configure custom Recovery, Start, and Stop scripts for the high-availability monitor. Monitors can optionally be configured with scripts that are run at various points during matrix operation. The script types are as follows: Recovery script. Runs after a monitor probe failure is detected, in an attempt to restore the NFS service. Start script. Runs as the NFS service is becoming active on a server. Stop script.
Chapter 2: Configure Export Groups 26 Start scripts must be robust enough to run when the service is already started, without considering this to be an error. Similarly, Stop scripts must be robust enough to run when the service is already stopped, without considering this to be an error. In both of these cases, the script should exit with a zero exit status.
Chapter 2: Configure Export Groups 27 You can configure the failover behavior with the Event Severity attribute. There are two settings: CONSIDER. This is the default value. Events are considered during failover decisions. IGNORE. Events are ignored and Start or Stop script failures will not cause failover. This is useful when the action performed by the Start and Stop scripts is not critical, but is important enough that you want to keep a record of it.
Chapter 2: Configure Export Groups 28 failover because Matrix Server does not have to wait for the failing node to finish running its Stop script.
Chapter 2: Configure Export Groups 29 To view status from the command line, use the following command: mx exportgroup status [--up|--down] [--enabled|--disabled] [--primary|--backup] [--active|--inactive] [ ...] Modify an Export Group To modify an existing Export Group, go to the Export Group Properties window, as described above. To change an export record, go to the NFS Exports tab.
Chapter 2: Configure Export Groups 30 Other Export Group Procedures Disable an Export Group Select the high-availability monitor associated with the Export Group on the Servers or Application tab, right-click, and select Disable. To disable the high-availability monitor from the command line, use this command. If no servers are specified, the action takes place on all servers. mx exportgroup disable ([ALL_SERVERS]|[ ...
3 Configure Virtual NFS Services A Virtual NFS Service exports the PSFS filesystems specified in its associated Export Group. Clients access the filesystems via the hostname/IP address of the Virtual NFS Service instead of using the hostname/IP address of the physical node. To create a Virtual NFS Service, you will need to specify the nodes on which the Virtual NFS Service should be configured. Optionally, you can also specify the Export Group that should be exported by the Virtual NFS Service.
Chapter 3: Configure Virtual NFS Services 32 located on a PSFS cluster filesystem, the backup node can immediately access the same filesystem data and continue with any I/O operations in progress. This type of configuration provides the best possible utilization of resources in the matrix, while also preserving high availability. Active-Passive Failover Configuration In an active-passive configuration, one or more nodes act as a backup for all of the Virtual NFS Services running on the other nodes.
Chapter 3: Configure Virtual NFS Services 33 the operating system appropriately to support the Virtual NFS Service. • Virtual NFS Services share a common name-space with other Matrix Server virtual hosts. You cannot create a virtual host and a Virtual NFS Service having the same network address. • After creating Virtual NFS Services, you will need to configure your applications to recognize them. Add a Virtual NFS Service To add a virtual NFS Service, select File Serving > Add Virtual NFS Service.
Chapter 3: Configure Virtual NFS Services 34 Export Group: Select the Export Group that will be exported by this Virtual NFS Service. If the Export Group has not yet been created, select NONE. Always active: If you check this box, upon server failure, the Virtual NFS Service will move to an active server even if the associated Export Group monitor is inactive or down.
Chapter 3: Configure Virtual NFS Services 35 The Virtual NFS Service appears on the Servers, Virtual Hosts, and Applications tabs on the Management Console. Following is the Virtual Hosts view. To create a Virtual NFS Service from the command line, use this command: mx vnfs add [--exportgroup |NONE] [--policy autofailback|nofailback] ...
Chapter 3: Configure Virtual NFS Services 36 NFS Service Re-Host window, use the arrows to move the Virtual NFS Service to another node. Use the up and down arrows to reorder the interfaces associated with the primary and backup nodes. To change the set of nodes used as a primary or backup for the Virtual NFS Service, move the appropriate interfaces to the Available or Configured columns.
Chapter 3: Configure Virtual NFS Services 37 From the Command Line Issue the following command, where is the Virtual NFS Service to be rehosted. You will need to specify all network interfaces on the which the Virtual NFS Service should be configured (the primary and all backups). mx vnfs move ... Other Virtual NFS Service Procedures Disable a Virtual NFS Service Select the Virtual NFS Service on Management Console, right-click, and select Disable.
4 Using the Applications Tab and Performance Dashboard The Applications tab on the PolyServe Management Console can be used to manage MxFS operations. The Management Console also includes a File Serving Performance Dashboard that contains NFS counters for Export Groups. The Applications Tab The Applications tab lists each Virtual NFS Service and Export Group and the servers on which the Virtual NFS Services are configured. On the Applications tab, the Status column reports the status of each Export Group.
Chapter 4: Using the Applications Tab and Performance Dashboard 39 available, or one or more Virtual NFS Services may be down and not running on any node. You can use the “drag and drop” feature to perform configuration operations as described in the PolyServe Matrix Server Administration Guide. You can also right click on an Virtual NFS Service or Export Group and make configuration changes.
Chapter 4: Using the Applications Tab and Performance Dashboard 40 • Delete the Virtual NFS Service. • Add a new Virtual NFS Service. • Add a new Export Group. • View all Export Groups. When you right-click on an Export Group, the following options are available: • View properties for this Export Group, including status. • Delete this Export Group. • Add a new Export Group. • Rehost the Export Group. • Enable or disable this Export Group on all servers. • View all Export Groups.
Chapter 4: Using the Applications Tab and Performance Dashboard 41 File Serving Performance Dashboard The File Serving Performance Dashboard provides performance information for Export Groups. The File Serving Performance Dashboard works in the same manner as the Matrix Performance Dashboard. To display the File Serving Performance Dashboard, select the Export Group on the Management Console, right-click, and select File Serving Dashboard. Copyright © 1999-2006 PolyServe, Inc. All rights reserved.
Chapter 4: Using the Applications Tab and Performance Dashboard 42 The File Serving Performance Dashboard displays most of the counters provided on the Matrix Performance Dashboard and also includes the following NFS counters. • Total NFS Transfer Rate (MB/s). The amount of data (in Megabytes) that has been moved across NFS. • Total NFS Operations/sec. The total number of RPC calls that NFS clients have made. • Total NFS connections. The approximate number of NFS clients that have been recently active.
5 Other Configuration Information This chapter provides information about the following: • NFS clients • Using the NLM protocol NFS Clients After MxFS is configured, your NFS clients can begin accessing the exported PSFS filesystems. Timeout Configuration It is recommended that NFS clients have a minimum timeout value of 120 seconds. NFS failovers typically take much less time, but in a worst-case scenario may approach 120 seconds.
Chapter 5: Other Configuration Information 44 The following command verifies that clients on the 99.10.210.100 network can now access the shared data: # ls -l /mnt/data1 total 1 drwxr-xr-x 2 root -rw-r--r-1 root -rw-r--r-1 root -rw-r--r-1 root root root root root 48 0 0 0 Mar Mar Mar Mar 24 24 26 24 00:32 00:32 12:29 00:32 dir1/ file1 file2 file3 Using the NLM Protocol NLM is the locking protocol used by NFS. By default, it is disabled when MxFS is installed.
Chapter 5: Other Configuration Information The mxnlmconfig command has this syntax: /opt/polyserve/bin/mxnlmconfig -q|-e|-d|-? The options are as follows: -q Show the current status of NLM locking in the cluster (either enabled or disabled). -e Enable NLM locking in the cluster. No reboot is necessary; the change is effective almost immediately. -d Disable NLM locking in the cluster. No reboot is necessary; the change is effective almost immediately. -? Display a syntax message.
Index A Applications tab Export Group, manage 38 Virtual NFS Service, configure 39 Virtual NFS Service, move 36 C configuration active-active 31 active-passive 32 examples 31 limits 5 E errors, status summary 28 event, clear 26 Export Group add 11 advanced options event severity 26 probe configuration 22 probe severity 23 script ordering 27 scripts, custom 25 Applications tab, manage from 38 caveats 8 clear errors 30 defined 7 delete 30 disable 30 enable 30 File Serving Performance Dashboard 41 high avail
Index 47 scripts 25 view on Management Console 20 mxnlmconfig command 44 R Recovery script 25 S N NFS clients filesystem access 7 timeout 43 NFS version, supported 5 NLM locking protocol 44 O objects Export Groups 3 export records 4 Virtual NFS Services 4 P probe, Export Group failover behavior 23 frequency 22 timeout 22 PSFS filesystems mount options 9 scripts custom 25 ordering of 27 Start script 25 status Export Group 28 summary report 28 Stop script 25 V Virtual NFS Service add 33 defined 7 del
