PolyServe® Installation Guide PolyServe Matrix Server 3.5.1 File Serving Utility™ 3.5.1 for Red Hat Enterprise Linux AS/ES 4.
Copyright © 2006-2008 PolyServe, Inc. Use, reproduction and distribution of this document and the software it describes are subject to the terms of the software license agreement distributed with the product (“License Agreement”). Any use, reproduction, or distribution of this document or the described software not explicitly permitted pursuant to the License Agreement is strictly prohibited unless prior written permission from PolyServe has been received.
Contents 1 Configuration Information Matrix Software Requirements. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Hardware Configuration Limits . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Supported HBA Drivers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Server Requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Network Configuration Requirements . . . . . . . . . . . . . . . . . . . . . . . . .
Contents 16. Install Hardware Snapshot Software (Optional) . . . . . . . . . . 17. Configure the Matrix . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Test the Fencing Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Install the Management Console on Other Hosts . . . . . . . . . . . . . . . Management Console for Linux Systems . . . . . . . . . . . . . . . . . . . Management Console for Windows Systems. . . . . . . . . . . . . . . . Administer init.d Scripts . .
Contents v D Build a RHEL4 Kernel from Non-PolyServe Sources Install the Kernel . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63 Compile the Kernel . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66 E Configure the Matrix from the Command Line Run mxconfig . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Verify the Fencing Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . .
1 Configuration Information PolyServe is continually expanding its supported hardware and operating system configurations. For the latest information, check the PolyServe Hardware and Software Compatibility Guide. Go to the following HP PolyServe web site, and, under “For more information,” click Compatibility Guide. http://www.polyserve.com Matrix Software Requirements The operating system and PolyServe software must be installed consistently across the servers in the matrix.
Chapter 1: Configuration Information 2 Hardware Configuration Limits The configuration limits for hardware used in a Matrix Server configuration are as follows. Hardware Configuration Limit Servers Two to 16 servers. Network Interface Cards Up to four network interfaces per server. FibreChannel Host Bus Adapters Four FC ports per server can be connected to the matrix SAN. Other FC ports can be connected to non-matrix SANs. FibreChannel Switches Two levels of cascading switches.
Chapter 1: Configuration Information 3 Server Requirements NOTE: To ensure that file times are consistent across the matrix, it is important that all matrix servers operate with synchronized timeof-day clocks. An NTP server is one commonly used mechanism for synchronizing system clocks. The minimum requirements for matrix servers are as follows: • A supported server architecture: – 32-bit Intel- or AMD-based servers (single processor or multiprocessor) running a supported operating system and kernel.
Chapter 1: Configuration Information 4 Network Configuration Requirements Networks used with Matrix Server should be configured as follows: • Each network interface card (NIC) must be on a separate physical network. For example, you could use either a separate Ethernet switch for each network or a managed switch that allows network traffic to be isolated. • Each NIC must be on a separate logical IP network.
Chapter 1: Configuration Information 5 switch2 and domain ID 7 must not exist on the fabric containing switch1. • To eliminate single points of failure, configure the SAN to use multipath I/O. See the PolyServe Administration Guide for some sample matrix configurations that include this feature. • Disk Power Management must be disabled on any Windows systems that have access to the SAN storage. NOTE: You can attach a matrix server to other SANs that are not under Matrix Server control.
2 Install Matrix Server and MxFSLinux This chapter describes how to perform a new installation of PolyServe Matrix Server. If you will be using MxFS-Linux, you will also need to install it as described in this chapter. Supported Operating Systems and Kernels Matrix Server 3.5.1 is supported on both RHEL4 and SLES9. RHEL4 Support The following versions of the operating system are supported. Operating System PolyServe Kernel RHEL 4.0 Update 3, 32-bit or 64-bit 2.6.9-34.EL RHEL 4.
Chapter 2: Install Matrix Server and MxFS-Linux 7 Linux. The PolyServe kernels are available on MatrixLink in the following binary and source formats: kernel-..rpm For uniprocessor systems kernel-largesmp-..rpm For multiprocessor systems having greater than 16 GB of RAM kernel-smp-..rpm For multiprocessor systems kernel-.src.rpm The kernel sources PolyServe also supplies devel packages for these kernels.
Chapter 2: Install Matrix Server and MxFS-Linux 8 • mxconsole_3.5.1...msi. The Management Console and mx utility in Microsoft Windows format. • pmxs--support-3.5.1-..rpm. Matrix Server support files for the supported kernels. The RHEL4 Support RPM is typically not needed during the installation. The SLES9 Support RPM will need to be installed if you will be building the kernel from source. • pmxs-quota-tools-3.13-..rpm.
Chapter 2: Install Matrix Server and MxFS-Linux Action Description Set up the SAN configuration. Set up the storage array, create LUNs for use as Matrix Server membership partitions, and configure the FC switches that will be under matrix control. Install the operating system. Install a supported operating system and modify the necessary system files. For SLES9, install a supported kernel. If using a RHEL4 source kernel, install the kernel. Install 3rd-party MPIO software (optional).
Chapter 2: Install Matrix Server and MxFS-Linux 10 If you will be using a client host outside the matrix to administer the matrix, you will also need to install the Management Console on that host. The Management Console is available on Linux and Windows. 1. Set Up the SAN Environment Configure the Storage Array Perform the initial configuration of the storage array as described in the array product documentation.
Chapter 2: Install Matrix Server and MxFS-Linux 11 NOTE: It is important to partition the newly created LUNs, which are represented to the operating system as sd devices, appropriately before Matrix Server is configured. (For example, use fdisk or a similar tool.) If you later need to repartition a disk containing a membership partition, you will need to stop Matrix Server before you change the layout. While the matrix is stopped, you will not be able to access other disks in the matrix.
Chapter 2: Install Matrix Server and MxFS-Linux 12 To configure FibreChannel switches, complete the following tasks: • Enable server access to the SAN. Each server that will be in the matrix must be able to see the disks in the SAN. You may need to enable server ports on the FC switches or to change the zoning configuration to give servers the necessary access to the SAN. • Modify the SNMP setup. Make the following changes: – Enable access to the SNMP agent from each server that will be in the matrix.
Chapter 2: Install Matrix Server and MxFS-Linux 13 RHEL4. Install the operating system and edit system files as described in Appendix A. If you will be using a PolyServe binary kernel, it should be installed when specified in step 5, below. For kernels built from source, Appendix A specifies when to install and compile the kernel. Appendix C contains information for PolyServe source kernels and Appendix D contains information for non-PolyServe source kernels. SLES9.
Chapter 2: Install Matrix Server and MxFS-Linux 14 5. Install a Binary Kernel (RHEL4 Only) Matrix Server provides several RHEL4 binary kernels that include the PolyServe patches. Select one of these kernels to install on the server. To install the kernel, run the following command: # rpm -ihv .rpm After installing the RPM, verify the bootloader. Check the file /boot/grub/ menu.1st and verify that the default is set to the PolyServe kernel.
Chapter 2: Install Matrix Server and MxFS-Linux 15 8. Install MxFS-Linux (File Serving Utility Only) Install the MxFS-Linux support RPM from the product CD or the location where you have downloaded the software. # rpm -i /mxfs--support-3.5.1-..rpm Install MxFS-Linux from the product CD or the location where you have downloaded the software. # rpm -i /mxfs-3.5.1-..
Chapter 2: Install Matrix Server and MxFS-Linux 16 included with Matrix Server or a version from another source), you can install that driver version in place of the Matrix Server default version. For more information, see the section “Host Bus Adapters (HBAs)” in Chapter 18 of the Matrix Server Administration Guide. Do not start Matrix Server after installing the driver.
Chapter 2: Install Matrix Server and MxFS-Linux 17 12. Verify the SAN Configuration This step verifies that the SAN devices are configured appropriately and can be viewed from the servers that will be in the matrix. You will need to perform this step on each server. If an HBA driver has not already been loaded, run the following command to load the driver: # /etc/init.
Chapter 2: Install Matrix Server and MxFS-Linux 18 14. Run the mxcheck Utility This utility should be run on each server. It verifies that the server’s configuration meets the requirements for running Matrix Server. Issue the following command: # /opt/polyserve/bin/mxcheck -l -p -t - install The utility performs checks such as the following: operating system version and configuration; available physical memory and disk space; HBA driver versions; FibreChannel switch versions; network addresses.
Chapter 2: Install Matrix Server and MxFS-Linux 19 16. Install Hardware Snapshot Software (Optional) Hardware snapshots are supported on Engenio storage arrays and Hewlett-Packard StorageWorks Enterprise Virtual Array (EVA) storage arrays. If you will be using this feature, the following software must be installed on the servers in the matrix.
Chapter 2: Install Matrix Server and MxFS-Linux 20 • An administrative password for configuring the matrix. • The fencing method (either FibreChannel switch-based fencing or Web Management-based fencing via server reset/shutdown). • The IP address or hostname of each FibreChannel switch that is included in the SAN. (This step is required for FibreChannel switchbased fencing and optional for Web Management-based fencing.) • The partitions or LUNs to be used for membership partitions.
Chapter 2: Install Matrix Server and MxFS-Linux 21 The Configure Matrix window then appears. You will need to specify information on the tabs in this order: General Settings, SAN & Fencing, Storage Settings, Matrix-Wide Configuration. General Settings Tab This tab asks for general information needed for matrix operations.
Chapter 2: Install Matrix Server and MxFS-Linux 22 1. License. Matrix Server can be used with either a temporary or a permanent license. The license is provided in a separate license file. (This file must be present on the server that you are using to connect to the Console.) To install the license, click the Change License File button. Type the location of the license file or use the Browse button to locate it. The license file is installed at /etc/opt/polyserve/licenses/license. 2. Secret Network Key.
Chapter 2: Install Matrix Server and MxFS-Linux 23 4. Cluster Administrative Traffic Protocol. Select either multicast or unicast mode. Multicast mode is recommended. When you have completed the fields on the General Settings tab, go to the SAN & Fencing tab.
Chapter 2: Install Matrix Server and MxFS-Linux 24 Select your storage type and, for FibreChannel, specify the fencing method that you want to use. (ISCSI storage must use web managementbased fencing.) FibreChannel Switch-Based Fencing Complete the following on the SAN & Fencing tab: 1. SAN Switches. Specify the hostnames or IP addresses of the FibreChannel switches that are directly connected to the nodes in the matrix. Click Add, and then specify the hostname or IP address of the first FC switch.
Chapter 2: Install Matrix Server and MxFS-Linux 25 2. SNMP Community String. The default SNMP community string for Matrix Server is private. If you want to use a custom community string, enter the appropriate value here. The SNMP community string must be set to the same value on Matrix Server and on the SAN switches configured above. When you have completed the information, go to the Storage Settings tab.
Chapter 2: Install Matrix Server and MxFS-Linux 26 1. Remote Management Controller Vendor. Select the vendor for your Remote Management Controllers. For an IBM Blade Center, also specify the Blade slot. 2. If you will be using IPMI as the fencing method, you should be aware that only one IPMI session can be active at a time. Matrix Server will fail to fence a server if another IPMI session is already active on that server at the time that the fencing attempt is made. 3. Remote Management Controller ID.
Chapter 2: Install Matrix Server and MxFS-Linux 27 this method if your Remote Management Controllers are from different vendors. This method must also be used for IBM BladeCenter servers. – Select “Cluster-wide Pattern” and then specify the common naming scheme that you are using for the Remote Management Controllers (either a hostname suffix or an IP address delta, as described below). Hostname Suffix.
Chapter 2: Install Matrix Server and MxFS-Linux 28 1. Shared configuration. Select the appropriate options. • Vendor and type selections apply to all servers. This option is enabled by default. Remove the checkmark if your Remote Management Controllers are from different vendors or if, in the case of IBM Remote Management Controllers, some are associated with IBM BladeCenter servers and others are not. • Login shared by all servers.
Chapter 2: Install Matrix Server and MxFS-Linux 29 2. Fencing Action. When a server needs to be restricted from the SAN, Matrix Server can either power-cycle the server or shut it down. Select the method that you want to use. If you have FibreChannel-connected storage and want to configure your FibreChannel switches into the matrix, go to the Switch Information tab.
Chapter 2: Install Matrix Server and MxFS-Linux 30 Repeat this procedure to specify the remaining FC switches, including cascading switches. 2. SNMP Community String. The default SNMP community string for Matrix Server is private. If you want to use a custom community string, enter the appropriate value here. The SNMP community string must be set to the same value on Matrix Server and on the SAN switches configured above. When you have completed the information, go to the Storage Settings tab.
Chapter 2: Install Matrix Server and MxFS-Linux 31 1. Membership Partitions. Matrix Server uses a set of membership partitions to control access to the SAN and to store the device naming database, which includes the global device names that Matrix Server assigns to the SAN disks placed under its control. You will need to select the LUNs or disk partitions that should be used as membership partitions.
Chapter 2: Install Matrix Server and MxFS-Linux 32 To create a membership partition, click Add. The Add Membership Partition window then lists all of the disks or LUNs that it can access. Select the disk or LUN where you want to place the first membership partition. All of the available partitions on that disk or LUN then appear in the bottom of the window. Select one of these partitions and click Add. (The minimum size for a membership partition is 100 MB.
Chapter 2: Install Matrix Server and MxFS-Linux 33 NOTE: Snapshots are supported on Engenio storage arrays and Hewlett-Packard StorageWorks Enterprise Virtual Array (EVA) storage arrays. The appropriate Engenio or HP software must be installed as described under step “16. Install Hardware Snapshot Software (Optional)” on page 19.
Chapter 2: Install Matrix Server and MxFS-Linux 34 3. Apply the configuration. When you have completed your entries on the Storage Configuration tab, click Apply (at the bottom of the Matrix Configuration window). You will then see a message stating the operation will erase all of the data on the membership partitions. Click Yes to continue. The configuration is then installed on the server that you are using to connect to the Management Console.
Chapter 2: Install Matrix Server and MxFS-Linux 35 1. Select the servers to be configured. To specify the first server, click Add Server and type the hostname or IP address of the server on the Input form. Then click OK to add the server to the Address column. Repeat this procedure to add the remaining servers to the Address column. 2. Export the configuration. Click Select All to select all of the servers in the Address column. Then click Export.
Chapter 2: Install Matrix Server and MxFS-Linux 36 If you are using Web Management-based fencing, you may be asked for additional information about each server. Based on the entries you made on the Fencing tab, the information can include any or all of the following: the vendor/type for the Remote Management Controller, the IP address/hostname of the Remote Management Controller, the IBM Blade slot, the username and password for the Remote Management Controller.
Chapter 2: Install Matrix Server and MxFS-Linux 37 for the Remote Management Controller, the server may not be configured to use that pattern. If an incorrect hostname or IP address was specified for the Remote Management Controller associated with the server, export the matrix configuration to that server and then supply the correct information.
Chapter 2: Install Matrix Server and MxFS-Linux 38 To start the Management Console, type the following command: $ mxconsole When the Connect window appears, you can log into the matrix.
Chapter 2: Install Matrix Server and MxFS-Linux 39 chapter “Configure MxFS-Linux”) for information about why file locking is disabled and how to enable it if necessary. Uninstall PolyServe Matrix Server NOTE: If you need to uninstall Matrix Server before upgrading to a later version of the product, use the directions in the PolyServe File Serving Utility Upgrade Guide. The software should be uninstalled from a location outside of the Matrix Server directory structure.
Chapter 2: Install Matrix Server and MxFS-Linux 40 To uninstall the Management Console on Windows, select Start > Settings > Control Panel > Add/Remove Programs and remove the application. Uninstall MxFS-Linux NOTE: If you need to uninstall MxFS-Linux before upgrading to a later version of the product, use the directions in the PolyServe Upgrade Guide.
Chapter 2: Install Matrix Server and MxFS-Linux 41 1. Adds a psfs policy line to the following file: /etc/selinux/targeted/src/policy/genfs_contexts 2. Rebuilds the binary policy file and installs it in: /etc/selinux/targeted/policy/policy.* 3. Reloads the kernel with the new policy information. If you modify SELinux and the psfs policy implemented by Matrix Server no longer works, you will need to make changes similar to the above to allow PSFS filesystems to be mounted.
A Install the RHEL4 Operating System Installation Steps Before installing Matrix Server, you will need to complete these steps: 1. Install Red Hat Enterprise Linux AS/ES 4, Update 3 or 4. 2. Build the kernel from source if desired: • For PolyServe source kernels, see “Build a RHEL4 Kernel from PolyServe Sources” on page 59. • For non-PolyServe source kernels, see “Build a RHEL4 Kernel from Non-PolyServe Sources” on page 63.
Appendix A: Install the RHEL4 Operating System 43 Installation Procedure 1. Install the Operating System The operating system must be installed on each server that will be in a matrix. When you perform the installation, do not place any operatingsystem files on the SAN disks. To ensure that files are not installed on these disks, you can either unplug the HBA card from the FC switch or unplug the SAN connection before starting the installation.
Appendix A: Install the RHEL4 Operating System • gcc • glibc-devel 44 • patch binutils, cpp and patch are installed by default if you select the default server installation during the initial installation procedure. You can install the remaining packages via a Package Management tool of your choice or directly from the RHEL4 distribution media. 2. Build the Kernel Build the kernel from source if desired: • For PolyServe source kernels, see “Build a RHEL4 Kernel from PolyServe Sources” on page 59.
Appendix A: Install the RHEL4 Operating System 45 2. Run the mkinitrd command to create the initrd image. See step “5. Run mkinitrd If Necessary” on page 47. NOTE: Although this procedure removes the HBA drivers from the initrd image, you will likely see the HBA drivers loaded after the first reboot of the kernel. The kernel HBA modules will be exempted from loading after the Matrix Server rpm is installed and the blacklist is updated.
Appendix A: Install the RHEL4 Operating System 46 4. Modify System Files You may need to modify the following files on each server: • /etc/hosts file. Red Hat Linux places both localhost and the server name on the 127.0.0.1 entry in the /etc/hosts file. For example: 127.0.0.1 localhost.localdomain localhost Matrix Server requires that the server name appear on a separate line with its real IP address, as in the following example.
Appendix A: Install the RHEL4 Operating System 47 – If you want Matrix Server to load the HBA driver, you may need to modify the /etc/modprobe.conf file. In the file, look for entries such as the following for scsi_hostadapter. alias scsi_hostadapter qla2x00 alias scsi_hostadapter1 aic7xxx alias scsi_hostadapter2 ... NOTE: Skip this step if the file has the following entry: alias scsi_hostadapter off Remove the scsi_hostadapter entry for QLogic or Emulex drivers, or comment them out with a “#” symbol.
Appendix A: Install the RHEL4 Operating System 6. Reboot and Verify the Kernel Configuration If you built the kernel from source, reboot the operating system and verify that the kernel is built and configured correctly.
B Install the SLES9 Operating System and Kernel Install the Operating System and Kernel Before installing Matrix Server, you will need to perform the following steps: 1. Install SuSE Linux Enterprise Server Version 9. 2. Either compile a new kernel from source or install a binary kernel. If you will be using MxFS-Linux or you want to use the kernel patch provided with Matrix Server, you will need to compile the kernel. 3. For iSCSI configurations, build and install the iscsi kernel modules. 4.
Appendix B: Install the SLES9 Operating System and Kernel 50 1. Install SuSE Linux Enterprise Server Version 9 SuSE Linux Enterprise Server Version 9 must be installed on each server in the matrix. Following are some considerations for the installation. • Do not place any operating-system files on the SAN disks. To ensure that files are not installed on these disks, you can either unplug the HBA card from the FC switch or unplug the SAN connection before starting the installation.
Appendix B: Install the SLES9 Operating System and Kernel 51 You will need to determine whether to compile a new kernel from source or to install a binary kernel. You will need to compile the kernel if you will be installing either (or both) of the following: • The MxFS-Linux product. • The PolyServe kernel patches. Matrix Server includes patches that fix problems with the SLES9 kernel. (For a description of the patches, see the section “PolyServe Kernel Patches” in the PolyServe Release Notes.
Appendix B: Install the SLES9 Operating System and Kernel 52 The installation creates a subdirectory in /usr/src/polyserve for each kernel provided in the RPM. Each kernel subdirectory contains the following: • A configs subdirectory containing the sample configuration files for that kernel. • A patches subdirectory containing any PolyServe kernel patches for that kernel. These patches are automatically applied when the RPM is installed.
Appendix B: Install the SLES9 Operating System and Kernel 53 The files for the 32-bit kernel are as follows: • ps_config.default. The standard kernel. • ps_config.smp. Kernel with multiprocessor support. • ps_config.bigsmp. Kernel with support for SMP and large memory. • ps_config.debug. Debug version of the kernel. The files for the 64-bit kernel are as follows: • ps_config.default. The standard kernel. • ps_config.smp. Kernel with multiprocessor support. • ps_config.debug. Debug version of the kernel.
Appendix B: Install the SLES9 Operating System and Kernel 54 Now run the following command to patch the .config file: # ./patch_config mxfs_patchlist /usr/src/linux/.config NOTE: Do not attempt to apply the 90_mxfs.patch file. This patch file was already applied when you installed the MxFS-Linux patches RPM earlier in this procedure. 4. Complete the kernel configuration and install the kernel.
Appendix B: Install the SLES9 Operating System and Kernel 55 make: Leaving directory '/usr/src/linux-2.6.5-7.252' Refer to the linux-iscsi RPM for the user-side control daemon and configuration. 4. HBA Drivers and Matrix Server When configuring the operating system, you will need to consider whether the HBA driver should be loaded either during the initial booting of the kernel or when Matrix Server is started.
Appendix B: Install the SLES9 Operating System and Kernel 56 to load its own HBA driver. (You may need to take steps to ensure that the appropriate HBA driver is loaded. See your vendor documentation for more information.) SAN Boot Disk With certain storage arrays, the boot disk can be on the SAN. In this case, the HBA driver must be loaded with the kernel so that the boot disk can be located. (You may need to take steps to ensure that the correct HBA driver is loaded.
Appendix B: Install the SLES9 Operating System and Kernel 57 hostnames include the hostnames of all servers in the matrix and the hostnames of all FibreChannel switches being used with the matrix. One easy way to ensure reliable hostname lookups is to put all critical hostnames and their IP addresses in /etc/hosts on each of the servers (see the hosts(5) man page.) After you have done this, ensure that the host lookup order in /etc/nsswitch.
Appendix B: Install the SLES9 Operating System and Kernel 58 1. Check the /boot directory for an initrd image for the kernel you just installed. Remove the image if it exists in the directory. 2. Run the following command from the /usr/src/linux directory: # mkinitrd For more information about this command, see the mkinitrd(8) man page. 7. Reboot and Verify the Kernel Configuration Reboot the operating system and then verify that the kernel is built and configured correctly.
C Build a RHEL4 Kernel from PolyServe Sources This appendix describes how to install and compile a RHEL4 kernel using a PolyServe source kernel. NOTE: If you will be using Oracle, the name of the kernel cannot be longer than 29 characters. This restriction is caused by an issue with the Oracle installer. Install the Kernel PolyServe provides 2.6.9-based source kernels on Matrix Link. These kernels include the PolyServe kernel patches.
Appendix C: Build a RHEL4 Kernel from PolyServe Sources 60 3. Run the following command on the spec file: # rpmbuild -bp --target= kernel-2.6.spec 4. Copy the kernel source tree to /usr/src: # cp -a /usr/src/redhat/BUILD/kernel-2.6.9/linux-2.6.9 /usr/src/ linux-2.6.9-34.EL 5. Create a symbolic link to the kernel source tree for the “linux” common name. # ln -s /usr/src/linux-2.6.9-34.EL /usr/src/linux 6.
Appendix C: Build a RHEL4 Kernel from PolyServe Sources 61 NOTE: If the servers have identical hardware configurations, you can create the kernel on one server and then copy it to the other servers. 1. Go to the /usr/src/linux- directory on the server and run the make mrproper command. # cd /usr/src/linux- # make mrproper 2. Patch the kernel as necessary for your site. 3.
Appendix C: Build a RHEL4 Kernel from PolyServe Sources 62 # rpm -i /mxfs-patches-3.5.1-..rpm 5. Complete the kernel configuration and install the kernel: # make oldconfig # make bzImage # make modules # make modules_install # make install 6. If you are using grub, the make install step should have updated the grub file. Verify that the correct kernel version is selected as the default. 7. If you are using LILO, you will need to take these steps: a.
D Build a RHEL4 Kernel from NonPolyServe Sources This appendix describes how to install and compile a RHEL4 kernel from a source other than PolyServe. CAUTION: The kernel must be a RHEL4 version supported by PolyServe. PolyServe products have been tested only with the patched source kernels provided with Matrix Server. If you apply nonPolyServe kernel patches and there are conflicts between those patches and the PolyServe kernel patches, the kernel may not function correctly.
Appendix D: Build a RHEL4 Kernel from Non-PolyServe Sources 64 3. Copy the extracted kernel sources to the /usr/src directory. Be sure to rename the destination directory with the correct version suffix (34.EL in our example). # cp -a /usr/src/redhat/BUILD/kernel-2.6.9/linux-2.6.9 /usr/src/ linux-2.6.9-34.EL 4. Create a symbolic link to the kernel source tree for the “linux” common name. # ln -s /usr/src/linux-2.6.9-34.EL /usr/src/linux 5.
Appendix D: Build a RHEL4 Kernel from Non-PolyServe Sources 65 version of the RPM that corresponds to your architecture (i586 for 32-bit or x86_64 for 64-bit). # rpm -i /pmxs-rhel4-support-3.5.1-.rpm The installation creates a subdirectory in /usr/src/polyserve for each kernel provided in the RPM. Each kernel subdirectory contains the following: • A configs subdirectory containing sample configuration files for that kernel.
Appendix D: Build a RHEL4 Kernel from Non-PolyServe Sources 66 Compile the Kernel Following is a suggested procedure that you can use as a guide for building the kernel. Modify this procedure as necessary for your local circumstances. NOTE: If the servers have identical hardware configurations, you can create the kernel on one server and then copy it to the other servers. 1. Go to the /usr/src/linux- directory on the server and run the make mrproper command.
Appendix D: Build a RHEL4 Kernel from Non-PolyServe Sources 67 3. If you will be installing MxFS-Linux, patch your kernel configuration file (.config). To begin, go to the following directory, where is the version of the kernel that you are about to compile. /usr/src/polyserve//tools Now run the following command to patch the .config file: # ./patch_config mxfs_patchlist /usr/src/linux/.config NOTE: Do not attempt to apply the 90_mxfs.patch file.
Appendix D: Build a RHEL4 Kernel from Non-PolyServe Sources The command should report the following: # /opt/polyserve/sbin/exportfssync -t Kernel supports NFS cluster-wide device aliases If the command reports that the kernel does not support NFS clusterwide device aliases, either the MxFS-Linux patch has not been applied, the kernel has not been compiled with the patches, or the recompiled kernel has not been booted.
E Configure the Matrix from the Command Line This appendix describes how to use the mxconfig utility to configure the matrix from the command line. The utility allows you to upgrade the license file, to select a matrix password and a Network Authentication Secret password, to select a fencing method, to specify the FibreChannel switches connected to the nodes in the matrix, to select the LUNs or disk partitions to be used as Matrix Server membership partitions, and to configure snapshots.
Appendix E: Configure the Matrix from the Command Line 70 Welcome to mxconfig Welcome to mxconfig. You may abort mxconfig at any time by pressing the key. < OK > On windows that require input, use the Tab key to move between OK and Cancel or between Yes and No. Press the Enter key to go to the next window. Press the Escape key to abort the mxconfig utility. If certain system files need to be modified on your server, mxconfig next displays messages about the changes needed to those files.
Appendix E: Configure the Matrix from the Command Line 71 Specify the Matrix Password The password you specify on the Enter cluster password window will be assigned to user admin. (You will need to log into the Management Console as user admin to configure the matrix.) The password does not display on the window as you type it. Enter cluster Password This password is used for authenticating the UI.
Appendix E: Configure the Matrix from the Command Line 72 Select the Cluster Administrative Traffic Protocol Specify either multicast or unicast mode. Multicast mode is recommended.
Appendix E: Configure the Matrix from the Command Line 73 Matrix Fence Module Selection Select a fencing module from the following list fcsan Fibrechannel switch port manipulation (recommended) webmgmt Web Management Based Fencing < OK > If you selected FibreChannel switch port manipulation, next go to “Configure FibreChannel Switches” on page 80.
Appendix E: Configure the Matrix from the Command Line 74 Remote Management Controller Vendor Select the vendor of the remote management controller for this server Dell HP IBM IPMI Dell ERA or DRAC III Hewlett Packard ILO IBM MM, RSA, or RSA II IPMI v1.5 (IPMI over LAN) < OK > You will next be asked whether all servers in the matrix are from the same vendor. If you specified IBM as the vendor, you will also be asked whether you are using a BladeCenter server.
Appendix E: Configure the Matrix from the Command Line 75 Remote Management Controller Configuration Select a method for Matrix Server to determine the remote management controller associated with each server Hostname-Suffix controller name = server name + common suffix IP-Delta controller IP address = server IP address + delta None Enter each controller hostname/address individually < OK > Enter the configuration information in accordance with the method that you selected. Hostname Suffix.
Appendix E: Configure the Matrix from the Command Line 76 IP Delta. Specify the delta to add to each server’s IP address to determine the IP addresses of the associated Remote Management Controllers. For example, if your servers are 1.255.200.12 and 1.255.200.15 and their Remote Management Controllers are 1.255.201.112 and 1.255.201.115, enter 0.0.1.100 as the delta.
Appendix E: Configure the Matrix from the Command Line 77 Blade Slot. If you are using an IBM BladeCenter server, specify the slot number for this server. IBM BladeCenter Enter the blade number of the server < OK > Fencing Action. When a server needs to be restricted from the SAN, Matrix Server can either power-cycle the server or shut it down. Specify the method that you want to use on the following window.
Appendix E: Configure the Matrix from the Command Line 78 Remote Management Controller Access Enter remote management controller username: < OK > Now specify the password associated with the user account. Remote Management Controller Access Enter remote management controller password: (Will not echo password) < OK > You are now asked whether the same username and password are used by all of the Remote Management Controllers in the matrix.
Appendix E: Configure the Matrix from the Command Line 79 SAN Configuration. You will next be asked whether you want to configure the SAN switches in Matrix Server. This step is optional for Web Management Based Fencing configurations; however, if the switches are configured the Management Console can display the switch ports used by the SAN. If the SAN switches have not previously been configured in Matrix Server, you will see the following window.
Appendix E: Configure the Matrix from the Command Line 80 Configure FibreChannel Switches On the SAN Configuration window, specify the hostnames or IP addresses of the FC switches that are directly connected to the nodes in the matrix. (If you are using Web Management-Based Fencing, you will see this window only if you chose to configure or reconfigure SAN switches.) SAN Configuration Enter hostnames of SAN switches to configure, separated by whitespace.
Appendix E: Configure the Matrix from the Command Line 81 If you selected Yes on the Specify SNMP Community String window, type the appropriate string on the Enter SNMP Community String window that appears next.
Appendix E: Configure the Matrix from the Command Line 82 Be sure that the partition layout is adequate for your future needs. When you start Matrix Server, the disks containing the membership partitions will be imported into the matrix. These disks cannot be removed from the matrix while Matrix Server is running. If you need to repartition a disk containing a membership partition, you will need to stop Matrix Server before you change the layout.
Appendix E: Configure the Matrix from the Command Line 83 Membership Partition Setup No partitions currently selected. Select one or three disk partitions to use as membership partitions. Select a disk to see a list of partitions on that disk.
Appendix E: Configure the Matrix from the Command Line 84 The partition you selected is displayed on the Membership Partition Setup window. Choose OK to complete the selection. Repeat this procedure to select two more membership partitions. We recommend that the partitions be on different disks. You can deselect and reselect partitions as necessary. NOTE: If you select a large partition, you will see a warning saying that the partition is much larger than needed.
Appendix E: Configure the Matrix from the Command Line Snapshot Configuration Select a snapshot configuration option Display Add Edit Remove Done Display snapshot configurations Add a new snapshot configuration Edit an existing snapshot configuration Remove an existing snapshot configuration Exit snapshot configuration < OK > To add a new snapshot configuration, select Add. Then specify the requested information on the screens that appear next.
Appendix E: Configure the Matrix from the Command Line 86 Export Configuration Enter hostnames of the servers you wish to copy this configuration to, separated by whitespace. < OK > mxconfig uses ssh as user root to copy the configuration to each server. Depending on your ssh configuration, you may be prompted for the root password one time on each host. If you are using Web Management-Based fencing, you may be asked for additional information about each server.
Appendix E: Configure the Matrix from the Command Line 87 When you run mxfence, Matrix Server uses the specified hostname or IP address to access the Remote Management Controller. The server is then either power-cycled or shut down in accordance with the method you selected when you configured the fencing module. If the correct server is not fenced, the hostname or IP address specified for the Remote Management Controller is probably incorrect.
Appendix E: Configure the Matrix from the Command Line 88 On the Authentication Parameters dialog, type admin for the user and specify the matrix password that you assigned earlier. When you click OK, the Management Console is displayed. The server where you started Matrix Server is currently the only server in the matrix.
Appendix E: Configure the Matrix from the Command Line 89 Add the Remaining Servers to the Matrix To add another server to the matrix, select Matrix > Server > New and enter the name or IP address of the server on the New Server window. Repeat this procedure to add the remaining servers to the matrix. The Management Console will show that the servers are down because Matrix Server is not yet running on them.
