Intel® RAID High Availability Storage User Guide Order Number: G85745-001
Disclaimer INFORMATION IN THIS DOCUMENT IS PROVIDED IN CONNECTION WITH INTEL® PRODUCTS. NO LICENSE, EXPRESS OR IMPLIED, BY ESTOPPEL OR OTHERWISE, TO ANY INTELLECTUAL PROPERTY RIGHTS IS GRANTED BY THIS DOCUMENT.
Important Safety Instructions Important Safety Instructions Read all caution and safety statements in this document before performing any of the instructions. ® See Intel Server Boards and Server Chassis Safety Information at http://support.intel.com/support/motherboards/server/sb/cs-010770.htm. Wichtige Sicherheitshinweise Lesen Sie zunächst sämtliche Warn- und Sicherheitshinweise in diesem Dokument, bevor Sie eine ® der Anweisungen ausführen.
WARNINGS Server power on/off: The push-button on/off power switch on the front panel of the server does not turn off the AC power. To remove AC power from the server, you must unplug the AC power cord from either the power supply or wall outlet. Hazardous conditions—power supply: Hazardous voltage, current, and energy levels are present inside the power supply enclosure. There are no user-serviceable parts inside it; servicing should only be done by technically qualified personnel.
Table of Contents 1. Introduction .................................................................................................................. 6 Concepts of High-Availability DAS ................................................................................ 6 Intel® RAID High Availability Storage Terminology ..................................................... 7 Intel® RAID High Availability Storage Solution Features ............................................... 7 2.
1. Introduction This document explains how to set up and configure the hardware and software for the Intel® RAID High Availability Storage solution. The Intel® RAID High Availability Storage solution provides fault tolerance capabilities as a key part of a high-availability data storage system.
virtual drives (VDs) are managed by a single initiator. In the case of Intel® RAID High Availability Storage, I/O transactions and RAID management operations are processed by a single Intel® RAID High Availability Storage controller, and the associated physical drives, drive groups, and VDs are only visible to that controller.
Write-back cache coherency SSD Cache 1.
2. Hardware and Software Setup This chapter explains how to set up the hardware and software aspects of the Intel® RAID High Availability Storage solution in a two-controller node with shared storage and fault-tolerant cabling for one or more JBODs. You can implement the Intel® RAID High Availability Storage solution with a two-server configuration connected to a JBOD, or with a Cluster-in-a-Box (CiB) configuration in which all the server hardware and disk drives are pre-connected inside a single enclosure.
Figure 2 Installing RAID Premium Feature Key on Intel® RAID Controller RS25DB080 4. Refer to documentation of the Intel® RAID Controller from http://www.intel.com/support/motherboards/server/ to finish installing the RAID controller into an Intel® Server Board, or qualified third party server board. Installing Intel® RAID High Availability Storage Hardware The first step to setting up the Intel® RAID High Availability Storage solution is to install and configure the hardware components.
board connections. As an option, the expander is configured with a third four-lane port for cascaded expanders, as shown in the following figure. Figure 3 Intel® RAID High Availability Storage Expander Configuration Note: Drive enclosures with dual ESM modules can support split modes or unified modes. For faulttolerant cabling configurations, you typically configure the enclosure in unified mode. (Check with your drive enclosure vendor to determine the appropriate settings).
Figure 4 Intel® RAID High Availability Storage Cluster-in-a-Box Configuration The cluster-in-a-box configuration for the Intel® RAID High Availability Storage solution requires a specially designed server and storage chassis that includes two Intel® RAID High Availability Storage controllers and multiple SAS disks. Because all components are inside the enclosure and are preconnected, the physical setup is minimal.
Figure 5 CiB Intel® RAID High Availability Storage Controller Configuration Setting Up a Two-Server Configuration with External JBOD Configuration The Intel® RAID High Availability Storage solution enables you to configure two separate, standard servers with Intel® RAID High Availability Storage controllers that provide access to disks in the same JBOD enclosure, or enclosures, for reliable, high-access redundancy, as shown in the following figure.
Figure 6 High-Availability Standard Server Configuration This configuration enables you to use standard, readily available server hardware and disk enclosures to set up a reliable Intel® RAID High Availability Storage configuration. The dual-server-JBOD configuration for the Intel® RAID High Availability Storage solution requires the following hardware for each of the two matched stand-alone server modules: A Intel® RAID High Availability Storage controller board (RS25NB008, RS25SB008, etc).
Figure 7 Two-Server Intel® RAID High Availability Storage Configuration Follow these steps to set up the hardware for a dual-server-JBOD configuration for Intel® RAID High Availability Storage clustering. 1. Install an Intel® RAID High Availability Storage controller board in each of the two server modules, following the instructions in the Quick Installation Guide. 2. If necessary, install network boards in the two server modules and install the cabling between them. 3.
5. Connect power cords to the server units and the JBOD enclosure and power the units. Cabling Configurations This section contains information about initially setting up a Intel® RAID High Availability Storage configuration with one or two JBODs. It also explains how to add a second JBOD to a single-JBOD configuration without interrupting service on the configuration. The following figure shows how to set up a two-controller-node configuration with a single JBOD enclosure.
Figure 9 Two-Controller-Node Configuration with Dual JBODs The top-down/ bottom-up cabling approach shown in this figure has the following benefits: The cross-connections between controllers and ESMs safeguard against the complete disconnection of a server from the drive enclosure. Continued access to drives is assured in the event of a complete drive enclosure failure or removal. Additional expansion drive enclosures can be hot-added without disrupting service, as shown in Figure 9.
Figure 10 Example of Incorrect Cabling of Two Disk Enclosures This configuration does not work for the following reasons (the numbers correspond to the labels in the figure): 1. The failure of an ESM in drive enclosure A causes disconnection between a server and both drive enclosures. 2. The lack of a proper connection from a controller port number to a common enclosure ESM results in inconsistent device reporting. 3.
Figure 11 Adding a Second Disk Enclosure - Redundant Configuration The steps for adding the second disk enclosure are as follows: 1. Connect a link from port 2 on ESM B of drive enclosure A to port 1 on ESM B of drive enclosure B. 2. Disconnect the link from port 1 on ESM A of drive enclosure A and reconnect it to port 1 on ESM A of drive enclosure B. 3. Disconnect the link from port 0 on ESM A of drive enclosure A and reconnect it to port 0 on ESM A of drive enclosure B. 4.
Installing Intel® RAID High Availability Storage Software The next step of setting up the Intel® RAID High Availability Storage solution is to install and configure the software components. The following Intel® RAID High Availability Storage software checklist lists the baseline software components needed for the two controller nodes.
Installing the Intel RAID Driver The Intel® RAID High Availability Storage controllers use a driver that is also used by the Intel RAID SAS products. Install the driver or update it to the version that supports the Intel® RAID High Availability Storage controller. Refer to the installation steps in the release notes in the Intel RAID SAS Device Driver package.
3. Creating the Intel® RAID High Availability Storage Configuration This chapter explains how to set up Intel® RAID High Availability Storage clustering on a Intel® RAID High Availability Storage cluster-in-a-box configuration or a two-server configuration after the hardware is fully configured and the operating system is installed. Validating the Failover Configuration Microsoft recommends that you validate the failover configuration before you set up failover clustering.
3. If the Introduction box is enabled (and appears), click Next. 4. In the Select Installation Type box, select Role Based or Feature Based. 5. In the Select Destination Server box, select the system and click Next. 6. In the Select Server Roles list, click Next to present the Features list. 7. Make sure that failover clustering is installed, including the tools. If necessary, run the Add Roles and Features wizard to install the features dynamically from this user interface. 8.
Creating Virtual Drives on the Controller Nodes The next step is creating VDs on the JBOD disks or on the disks in the cluster-in-a-box enclosure. The Intel® RAID High Availability Storage cluster configuration requires a minimum of one shared VD to be utilized as a quorum disk to enable Microsoft operating system support for clusters. Refer to the Intel® RAID Software User’s Guide for information about the available RAID levels and the advantages of each one.
Figure 13 Intel® RAID BIOS Console Main Page The first Configuration Wizard window appears. 4. Select Add Configuration and click Next. 5. On the next wizard screen, select Manual Configuration and click Next. The Drive Group Definition window appears. 6. In the Drives panel on the left, select the first drive, then hold down the Ctrl key and select more drives for the array, as shown in the following figure.
Figure 14 Selecting Drives 7. Click Add To Array, click ACCEPT, and click Next. 8. On the next screen, click Add to SPAN, then click Next. 9. On the next screen, click Update Size. 10. Select the Share Virtual Drive option on the bottom left of the window, as shown in the following figure.
Figure 15 Virtual Drive Definition This option enables a shared VD that both controller nodes can access. If you uncheck this box, the VD has a status of exclusive, and host access exists only for the controller node that created this VD. 11. On this same page, click Accept, then click Next. 12. On the next page, click Next. 13. Click Yes to accept the configuration. 14. Repeat the previous steps to create the other VDs.
Creating Shared VDs with CmdTool264.exe on Windows Server 2012 CmdTool2 is a command-line-driven utility used to create and manage VDs. CmdTool2 can run in any directory on the server. The following procedure assumes that a current copy of the 64-bit Windows version of CmdTool2 is located on the server in c:\intel\cli. The steps for creating a VD are slightly different depending on whether you run CmdTool2 in Windows PowerShell® or from a Windows command prompt.
Figure 17 Sample Configuration Information The command outputs many lines of information that scroll down in the PowerShell window. You need to use some of this information in the command line to create the shared VD. 4. Find the drive enclosure number Device ID for the system and also the Device IDs of available physical drives for the VD you will create. In the preceding figure, the enclosure device ID of 20 appears close to the top of the window.
The opening and closing square brackets define the list of drives for the VD. Each drive is listed in the form enclosure device ID: drive device ID. The WB parameter sets the controller to use the write cache. (Alternatively, the WT parameter sets the controller cache for write through.) The RA parameter sets the controller cache for read ahead. The direct parameter sets direct I/O. The –strpsz64 parameter sets the stripe size to 64 KB.
Figure 19 Searching for ‘cmd’ The system finds the command prompt. 3. Right-click the Command Prompt. A properties check indicator appears on the button, as shown in the following figure. Figure 20 Command prompt The following button appears at the bottom of the desktop to open the command prompt as Administrator. Figure 21 Run as Administrator Button Click the button to open a command prompt, as shown in the following figure.
Figure 22 Windows Command Prompt 4. Enter the following command at the prompt: CmdTool264 –cfgdsply –a0 The -a0 parameter presumes that there is only one Intel® RAID High Availability Storage controller in the system or that these steps reference the first Intel® RAID High Availability Storage controller in a system with multiple controllers. The following figure shows some sample configuration information that appears in response to the command.
The command outputs many lines of information that scroll down in the command prompt window. You need to use some of this information in the command line to create the shared VD. 5. Find the drive enclosure number Device ID for the system and also the Device IDs of available physical drives for the VD you will create. In the preceding figure, the enclosure device ID of 20 appears close to the top of the window. Use the scroll bar to find the device IDs of the physical drives for the VD.
Creating Shared VDs with RWC2 Follow these steps to create VDs for data storage with RWC2. When you create the VDs, you assign the Share Virtual Drive property to them so they are visible from both controller nodes. This example assumes you are creating a RAID 5 redundant VD. Modify the instructions as needed for other RAID levels. NOTE: Not all versions of RWC2 support Intel® RAID High Availability Storage.
6. Enter a name for the VD. 7. Select Always Write Back as the Write policy option, and select other VD settings as required. 8. Select the Share Virtual Drive option, as shown in the following figure. NOTE: If you do not select Share Virtual Drive, the VD is visible only from the server on which it is created. Figure 25 Share Virtual Drive Option 9. Click Create Virtual Drive to create the virtual drive with the settings you have specified.
Figure 26 Create Virtual Drive Summary 11. Click Finish to complete the VD creation process. 12. Click OK when the Create Virtual Drive - complete message appears. Unsupported Drives Drives that are used in the Intel® RAID High Availability Storage solution must have Intel® RAID High Availability Storage-compatible implementations of the SCSI-3 persistent reservation (PR) feature. Intel maintains a list of drives that meet this requirement.
Intel® RAID High Availability Storage SSD Cache Support The Intel® RAID High Availability Storage controller includes support for SSD Cache 1.0, a feature that uses SAS SSD devices for read caching of frequently accessed read data. When a VD is enabled for the SSD Cache feature, frequently read data regions of the VD are copied into the SSD when the SSD Cache algorithm determines the region is a good candidate.
2. In the Drive Group window, set the SSD Cache RAID level and select one or more unconfigured SSD drives. Use the Add button to place the selected drives into the drive group. RAID 0 is the recommended RAID level for the SSD Cache volume. The following figure shows the SSD Cache drive group. Figure 29 Creating SSD Cache Drive Group 1 3. Click Create Drive Group and then click Next. 4. In the Create SSD Cache SSD Caching Virtual Drive window, update the SSD Caching VD name and set the size as necessary.
Figure 30 Creating SSD Cache Drive Group 2 5. Click Create Virtual Drive and then click Next. 6. In the Create SSD Cache SSD Caching Summary window, review the configuration and then click Finish.
Figure 31 Reviewing the Configuration 7. In the Create SSD Cache Complete box, click OK. The CacheCache VD now appears on the Physical Tab of RWC2, as shown in the following figure. The SSD Cache volume association with the drive groups appears in this view.
Figure 32 New SSD Cache Drive Group ® Intel High Availability Storage User Guide 41
4. System Administration This chapter explains how to perform system administration tasks, such as planned failovers and updates of the Intel® RAID High Availability Storage controller firmware. High Availability Properties The following figure shows the high availability properties that RWC2 displays on the Controller Properties tab for a Intel® RAID High Availability Storage controller.
Incompatibility Details – If the peer controller is incompatible, this field lists the cause of the incompatibility. Understanding Failover Operations A failover operation in Intel® RAID High Availability Storage is the process by which VD management transitions from one server/controller node to the peer server/controller node.
The operating system on Server A only sees VDs with shared host access and exclusive host access to Server A. The operating system on Server B only sees VDs with shared host access and exclusive host access to Server B. Before failover, the operating system perspective of I/O transactions is as follows: Server A is handling I/O transactions that rely on A-DG0:VD1 and A-DG0:VD2. Server B is handling I/O transactions that rely on A-DG0:VD0 and B-DG0:VD3.
I/O transactions that rely on the exclusive A-DG0:VD1 on Server A fail because exclusive volumes do not move with a failover. NOTE: When Server A returns, the management and I/O paths of the pre-failover configurations are automatically restored. The following sections provide more detailed information about planned failover and unplanned failover.
Figure 36 Failover Cluster Manager 3. In the left panel, expand the tree to display the disks, as shown in the following figure. Figure 37 Expand Tree 4. Right-click the entry in the Assigned To column in the center panel of the window. A pop-up menu appears. 5. In the pop-up menu, select Move > Select Node, as shown in the following figure.
Figure 38 Expand Tree 6. Select the node for the planned failover. Understanding Unplanned Failover An unplanned failover might occur if the controller in one of the controller nodes fails, or if the cable from one controller node to the JBOD is accidentally disconnected. The Intel® RAID High Availability Storage solution is designed to automatically switch to the other controller node when such an event occurs, without any disruption of access to the data on the drive groups.
6. After you locate the file, click Ok. The RWC2 software displays the version of the existing firmware and the version of the new firmware file. 7. When you are prompted to indicate whether you want to upgrade the firmware, click Yes. The controller is updated with the new firmware code contained in the .rom file. 8. Reboot the controller node after the new firmware is flashed. The new firmware does not take effect until reboot. 9.
Figure 39 Windows Server 2008 R2 System Properties 3. Select the Hardware tab and click Device Manager. 4. Click Storage to expose the Intel® RAID High Availability Storage controller. 5. Right-click the Intel® RAID High Availability Storage controller and select Update Driver Software to start the Driver Update wizard, as shown in the following figure.
Figure 40 Updating the Driver Software 6. Follow the instructions in the wizard. Updating the Driver in Windows Server 2012 As a recommended best practice, always back up system data before updating the driver, and then perform a planned failover. These steps are recommended because a driver update requires a reboot of the system. 1. Run Server Manager and select Local Server on the left panel. 2. Click the Tasks selection list on the right-hand side of the window, as shown in the following figure.
Figure 41 Updating the Driver Software 3. Select Computer Management, then click Device Manager. 4. Click Storage to expose the Intel® RAID High Availability Storage controller, as shown in the following figure.
Figure 42 Updating the Driver Software 5. Right-click the Intel® RAID High Availability Storage controller and select Update Driver Software to start the Driver Update wizard. 6. Follow the instructions in the wizard. Performing Preventative Measures on Disk Drives and VDs The following drive and VD-level operations help to proactively detect disk drive and VD errors that could potentially cause the failure of a controller node.
Troubleshooting This chapter contains useful information for troubleshooting a Intel® RAID High Availability Storage system. Reference Checklist of Required Intel® RAID High Availability Storage Components The following summary list shows the components required for a Intel® RAID High Availability Storage configuration. For more detailed information, see Chapter 2: Hardware and Software Setup. 1.
Figure 43 Share Virtual Drive Property Intel® RAID High Availability Storage versions of RWC2 also require an Intel® RAID High Availability Storage-capable OS driver to present Intel® RAID High Availability Storage features. The in-box driver for Windows Server 2012, version 5.2.122.0 4/3/2012 does not present Intel® RAID High Availability Storage features in RWC2. NOTE: Intel tested the Intel® RAID High Availability Storage solution with the release to manufacturing (RTM) build of Windows Server 2012.
Confirming SAS Connections The high availability aspect of Intel® RAID High Availability Storage is based upon redundant SAS data paths between the clustered nodes and the disk drives. If all of the components in the SAS data path are configured and connected properly, each Intel® RAID High Availability Storage controller has two SAS addresses for every drive, when viewed from the Intel® RAID High Availability Storage controllers.
Figure 45 Redundant SAS Data Paths Are Missing The following figure shows the correct view with two drive SAS addresses.
Figure 46 Redundant SAS Data Paths Are Present Using CmdTool2 to Verify Dual-Ported SAS Addresses to Disk Drives The CmdTool2 configuration display command (-cfgdsply) returns many lines of information, including a summary for each physical disk. To confirm the controller discovery of both SAS addresses for a single drive, examine the CmdTool2 configuration text for the drive information following the Physical Disk line.
Device Id: 57 WWN: 5000C500178C4488 Sequence Number: 16 Media Error Count: 0 Other Error Count: 1961 Predictive Failure Count: 0 Last Predictive Failure Event Seq Number: 0 PD Type: SAS Raw Size: 68.366 GB [0x88bb998 Sectors] Non Coerced Size: 67.866 GB [0x87bb998 Sectors] Coerced Size: 67.
Figure 47 Redundant SAS Connections Displayed in RWC2 Understanding SSD Cache Behavior During a Failover A SSD Cache VD possesses properties that are similar to a VD with exclusive host access, and it is not presented to the host operating system. Therefore, the SSD Cache volume does not cache read I/Os for VDs that are managed by the peer controller node. Foreign import of a SSD Cache VD is not permitted.
Disk bus type does not support clustering. Disk partition style is MBR. Disk partition type is BASIC. operating system does not natively support internal RAID controllers for clustering. Confirming SAS Connections. controller parameter is detected. The peer controller is prevented from entering the HA domain. have settings that do not match the controller.
