Dell Shared PowerEdge RAID Controller 8 Cards For Dell PowerEdge VRTX Systems User’s Guide Regulatory Model: UCPM-800
Notes, cautions, and warnings NOTE: A NOTE indicates important information that helps you make better use of your product. CAUTION: A CAUTION indicates either potential damage to hardware or loss of data and tells you how to avoid the problem. WARNING: A WARNING indicates a potential for property damage, personal injury, or death. © 2017 - 2018 Dell Inc. or its subsidiaries. All rights reserved. Dell, EMC, and other trademarks are trademarks of Dell Inc. or its subsidiaries.
Contents 1 About the Shared PERC 8 card...................................................................................................................... 6 Configurations of the Shared PERC 8 card.................................................................................................................... 7 Specifications of a Shared PERC 8 card.........................................................................................................................
Installing a new Shared PERC 8 Internal card............................................................................................................... 21 Replacing a Shared PERC 8 Internal card......................................................................................................................21 Prerequisites for removing the Shared PERC 8 Internal card.............................................................................. 22 Removing the Shared PERC 8 Internal card.............
Configuration and management of SED capable hard drives...............................................................................49 Troubleshooting security key errors...............................................................................................................................50 Secured foreign import errors..................................................................................................................................
1 About the Shared PERC 8 card NOTE: Unless noted otherwise, all references to Shared PERC 8 throughout the document see both Internal and External Shared PERC 8 cards. The Dell Shared PowerEdge RAID controller (PERC) 8 card is a storage controller designed specifically for the Dell PowerEdge VRTX System. The Shared PERC 8 card supports Single Root Input Output Virtualization (SR-IOV), which allows multiple systems to share available hardware resources.
Figure 2. Features of Shared PERC 8 External card 1 external cable connectors 2 Shared PERC External card 3 heat sink 4 battery cable 5 battery Topics: • Configurations of the Shared PERC 8 card • Specifications of a Shared PERC 8 card • Supported operating systems • Best practices for Shared PERC 8 • Documentation matrix Configurations of the Shared PERC 8 card The Shared PERC 8 Internal and External cards are two separate SAS domains.
External card fails, access to the storage subsystem attached to that card is lost. This configuration option is not available for Shared PERC 8 Internal cards. Specifications of a Shared PERC 8 card The characteristics of a Shared PERC 8 card are as follows: Table 1. Shared PERC 8 card specifications Specification Description Supported drives Dell-qualified SAS hard drives, SAS solid-state drives (SSDs), and Self-Encrypting Disks (SEDs) drives.
Specification Description Hardware XOR engine Yes Virtual disk initialization; background, full, and fast Yes Consistency check Yes Disk roaming Yes FastPath Yes Virtual disk migration between two Shared PERC 8 cards Yes Virtual disk migration from PERC H310, H710, or H710P cards to a Shared PERC 8 card No SMART detection Yes Patrol Read Yes Automatic replace member Yes SAS (HDD/SSD) hard drive Yes Physical disk power management IDLE-C for compliant HDDs only.
• Microsoft Windows Server 2012 • Microsoft Windows Server 2008 R2 • Red Hat Enterprise Linux (RHEL) 6 Update 5 • RHEL 6 Update 6 • RHEL 6 Update 7 • RHEL 7.1 • SUSE Enterprise Linux (SLES) 11 SP4 • SLES 11 SP3 • SLES 12 • VMware ESXi 5.5 • VMware ESXi 6.0 and later NOTE: For information on supported software and hardware for the PowerEdge VRTX system, see PowerEdge VRTX Storage Subsystem Compatibility Matrix at Dell.com/support/manuals.
To... See the... Install, configure, and use the Chassis Management Controller (CMC) Dell Chassis Management Controller for Dell PowerEdge VRTX User’s Guide at Dell.com/poweredgemanuals See the error and event messages generated by the firmware or other agents that monitor system components Dell PowerEdge VRTX Chassis Management Controller Firmware Event Message Reference Guide at Dell.com/poweredgemanuals See the instructions for the current page on the CMC web interface The CMC Online Help.
2 Shared PERC 8 card features Some of the features of the Shared PERC 8 card are as follows: • Physical disk power management • Consistency checks • Virtual disk initialization • Disk roaming • FastPath • Virtual disk migration • Virtual disk cache policies • Fault tolerance • Patrol read • Multipath support • Supports controller level encryption Topics: • Physical disk power management • Consistency Checks • Virtual disk initialization • Physical disk roaming • FastPath • V
Virtual disk initialization You can initialize virtual disks as described in the following sections. Background Initialization of virtual disks Both Consistency Check (CC) and Background Initialization (BGI) correct parity errors. However, CC reports data inconsistencies through an event notification, whereas BGI is an automatic process. BGI does not run on RAID 0 virtual disks. You cannot disable BGI permanently. If you cancel BGI, it automatically restarts within five minutes.
NOTE: If you are moving an entire just a bunch of disks (JBOD) enclosure, make sure to turn off the JBOD before cabling to the new controller. Using physical disk roaming Perform the following steps to use disk roaming: 1 Turn off the power to the system, physical disks, enclosures, and system components. 2 Disconnect power cables from the system. 3 Move the physical disks to the required positions on the backplane or the enclosure. 4 Perform a safety check.
• From another system's Shared PERC 8 controller when the target system is either turned on or off. NOTE: When a controller detects a physical disk with an existing configuration, it flags the physical disk as a foreign disk and generates an alert indicating that a foreign disk was detected. Migrating virtual disks if the target system is turned off 1 If installed, remove the front bezel (optional) from the PowerEdge VRTX enclosure.
Write-back In Write-back caching, the controller sends a data transfer completion signal to the host when the controller cache has received all the data in a transaction. The controller then writes the cached data to the storage device in the background. The risk of using write-back cache is that the cached data can be lost if there is a power failure during or when the data is written to the storage device. This risk is mitigated by using a battery-backed non-volatile cache on the Shared PERC 8 card.
• No Read Ahead — Disables the Read-Ahead capability. • Adaptive Read Ahead — Read Ahead Mode is used when the two most recent disk accesses occurred in sequential sectors, otherwise the controller reverts to No Read Ahead mode. NOTE: The default read cache setting for virtual disks is Adaptive Read Ahead. Physical disk write cache policy Management utilities provide the option to modify disk cache policies for physical disks associated with a virtual disk.
Auto replace member The automatic Replace Member functionality allows a previously commissioned hot spare to be reverted to a usable hot spare. When a disk failure occurs within a virtual disk, an assigned hot spare (dedicated or global) is commissioned and begins rebuilding until the virtual disk is optimal.
Physical disk failure detection Failed physical disks are detected and rebuilds automatically start to new disks that are inserted into the same slot. Automatic rebuilds can also occur with hot spares. If you have configured hot spares, the controllers automatically try to use them to rebuild failed physical disks. Physical disk hot swapping NOTE: Shared PERC 8 supports physical disk hot-swapping, however this should only be implemented for unconfigured or failed drives.
Battery Transparent Learn Cycle A transparent learn cycle is a periodic operation that calculates the charge that is remaining in the battery to ensure there is sufficient energy. The operation runs automatically, and causes no impact to the system or Shared PERC 8 card performance. The Shared PERC 8 card automatically performs the Transparent Learn Cycle (TLC) on the battery to calibrate and gauge its charge capacity once every 90 days.
3 Deploying the Shared PERC 8 card Follow the documentation in this section to remove or install Shared PERC 8 Internal and External cards under different scenarios. Safety instructions WARNING: Whenever you need to lift the system, get others to assist you. To avoid injury, do not attempt to lift the system by yourself. WARNING: Opening or removing the system cover while the system is powered on may expose you to a risk of electric shock.
NOTE: Write-back caching in a fault tolerant configuration is not supported with firmware version 23.8.10-0061 and the earlier versions. NOTE: For write-back caching capability, ensure that all the storage solution components are updated with the latest firmware. For more information, see the storage subsystem compatibility matrix document at Dell.com/support/home. For information on removing and reinstalling system parts, see the VRTX Enclosure Owner's Manual of the system at Dell.com/ poweredgemanuals.
CAUTION: Shared PERC 8 Internal cards are not hot-swappable. Do not remove or install the Shared PERC 8 Internal cards from the system board while the PowerEdge VRTX system is turned on. 1 Push the release levers at the edge of the Shared PERC 8 Internal card to disengage the card from the card holder. 2 Orient the card so that the other end of the card disengages from the card holder on the system board. CAUTION: When removing or replacing the Shared PERC 8 Internal card, hold the card by its edges.
NOTE: Before closing the system, ensure that the release levers have snapped over the edge of the card. 4 Close the system. Related links Installing a new Shared PERC 8 Internal card Replacing a Shared PERC 8 Internal card Postinstallation tasks for the Shared PERC 8 Internal card 1 Turn on the PowerEdge VRTX system without hard drives and server modules inserted. 2 Log in to the CMC GUI. 3 Click Overview → Power → Control, and view the Power State to ensure that it is ON.
CAUTION: Shared PERC 8 External cards are hot add supported, but are not hot remove supported. Do not remove Shared PERC 8 External cards from the system board while the PowerEdge VRTX system is turned on. In the Single Shared PERC 8 External card configuration, the shared PERC External slot is labeled as Slot 5 or Slot 6 (either can be used) on the PowerEdge VRTX system board.
Removing the Shared PERC 8 External card CAUTION: Many repairs may only be done by a certified service technician. You should only perform troubleshooting and simple repairs as authorized in your product documentation, or as directed by the online or telephone service and support team. Damage due to servicing that is not authorized by Dell is not covered by your warranty. Read and follow the safety instructions that came with the system. CAUTION: Shared PERC 8 External cards are not hot-removable.
Figure 4. Removing and installing the Shared PERC 8 External card 1 Shared PERC 8 External card 2 card connector on the system board 3 expansion card latch 4 metal standoff 5 release tab 6 divider unit Related links Replacing a Shared PERC 8 External card Installing the Shared PERC 8 External card CAUTION: Many repairs may only be done by a certified service technician.
Related links Replacing a Shared PERC 8 External card Installing a new Shared PERC 8 External card Single or dual Shared PERC 8 External nonfault tolerance 1 Turn off the server modules. 2 Click Server Overview → Power → Control → Power Off Server for every server. 3 Click Chassis Overview → Power → Control → Power Off System to turn off the chassis. 4 Open the system and install a maximum of two Shared PERC 8 External cards in Slots 5 and 6.
Figure 6.
Figure 7. Cabling for dual Shared PERC 8 External non-fault tolerance mode NOTE: In dual Shared PERC 8 External nonfault tolerance mode, the two Shared PERC 8 External controllers are independent and cannot communicate with each other. Dual Shared PERC 8 External fault tolerance 1 Turn off the server modules. Click Server Overview → Power → Control → Power Off Server for every server. 2 Click Chassis Overview → Power → Control → Power Off System to turn off the chassis.
4 Click Chassis Overview → Power → Control → Power On System to turn on the chassis. 5 To enable Fault Tolerance on two Shared PERC 8 External cards, the firmware version must be the same for both cards. Click Chassis Overview → Storage → Update and upload the necessary firmware image for both Shared PERC 8 External cards. 6 Click Chassis Overview → Storage → Controllers → Troubleshooting → Actions → Enable Fault Tolerance for both controllers.
Figure 9. Cabling for Shared PERC 8 External card dual enclosure Fault Tolerance Postinstallation tasks for the Shared PERC 8 External card 1 Log in to the CMC GUI. 2 Click Chassis Overview → Storage → Controllers to view the firmware version of the installed Shared PERC 8 External card. NOTE: If the Shared PERC 8 External cards have different versions of their firmware, download and update the firmware to the latest version from Dell.com/drivers.
Prerequisites for transition scenarios CAUTION: Many repairs may only be done by a certified service technician. You should only perform troubleshooting and simple repairs as authorized in your product documentation, or as directed by the online or telephone service and support team. Damage due to servicing that is not authorized by Dell is not covered by your warranty. Read and follow the safety instructions that came with the system.
Two existing non-fault tolerant controllers to fault tolerant configuration Two existing non-fault tolerant controllers (connected to two enclosures each) to fault tolerant configuration. NOTE: In a dual controller fault tolerant configuration, a maximum of two enclosures are supported. This scenario results in a loss of two enclosures. CAUTION: Before performing the transition, back up all data and complete the steps outlined in prerequisites for transition scenarios section.
4 Firmware and driver installation The Dell Shared PowerEdge RAID Controller (PERC) 8 card requires software drivers to operate with the supported operating systems. This chapter contains the procedures for installing the drivers for the Shared PERC 8 card. NOTE: To check operating system compatibility, see Dell.com/ossupport. NOTE: The Shared PERC 8 does not support booting to the shared storage.
Downloading drivers from the Dell Systems Service and Diagnostic Tools media for Windows To download drivers from the Dell Systems Service and Diagnostic Tools media: 1 Insert the Dell Systems Service and Diagnostics Tools media in a system. The Welcome to Dell Service and Diagnostic Utilities screen is displayed. 2 Select your system model and operating system. For the list of supported operating systems, see Supported Operating Systems. 3 Click Continue.
NOTE: Dell provides the Dell Update Package (DUP) to update drivers on systems running Windows Server 2012/2012 R2 or Window Server 2008/2008 R2 operating systems. DUP is an executable application that updates drivers for specific devices. DUP supports command line interface and silent execution. For more information, see Dell.com/support. Installing or updating Linux drivers for Shared PERC 8 Use the procedures in this section to update the driver for supported Red Hat and SUSE Linux distributions.
Installing or updating VMware drivers for Shared PERC 8 For information on how to install drivers on VMware ESXi 5.1, 5.5, and 6.0, see kb.vmware.com/kb/2005205. For information on VMware ESXi drivers, see the VMware ESXi documentation at Dell.com/virtualizationsolutions. Firmware installation or update The VRTX storage subsystem includes of Shared PERC firmware, drivers, Chassis Infrastructure firmware and the CMC.
5 Multipath and clustering setup When configuring your VRTX solution with the Shared PERC 8 it may be necessary to tune your system for optimization according to your specific usage scenario and configuration. Each operating system exposes different settings to the user. Below is a list of some of the settings that may be helpful for your specific usage scenario.
MPIO Timer Value HKLM\system\CurrentControlSet\Services\MPIO\Parameters \PathRecoveryInterval 25 CAUTION: Always back up your registry files (system.dat and user.dat) before editing the registry. CAUTION: Using Registry Editor incorrectly can cause serious problems that may require you to reinstall Windows. Dell cannot guarantee that problems resulting from the incorrect use of Registry Editor can be solved. Use Registry Editor at your own risk.
1 If possible have more than one LUN for data storage to reduce LUN contention and reservation conflicts. 2 High numbers of VMs per LUN can also increase LUN contention. Due to varying workloads it is difficult to identify precise guidelines for VMs per LUN, but it is good practice not to exceed 64 virtual machines per LUN for proper optimization of your particular solution.
chkconfig boot.multipath on If the boot.multipath service does not start automatically on system boot, do the following to start them manually: /etc/init.d/boot.multipath start /etc/init.d/multipathd start 3 Multipath configuration can be verified with the information in /etc/multipath.conf file by utilizing following commands.
3 Multipath configuration can be verified with the information in /etc/multipath.conf file by utilizing following commands. multipath -v2 –d multipath -v3 –d Once the above commands are verified to be successful, the multipath devices are available at /dev/mapper/mpathX by default where X is multipath eligible devices. You can use multipath-ll to verify the status of the multipath configuration.
devices { device { vendor "DELL" product "Shared PERC8" hardware_handler "1 alua" path_grouping_policy failover prio alua path_checker tur rr_weight priorities failback immediate no_path_retry fail path_selector "round-robin 0" flush_on_last_del no user_friendly_names "yes" features "0" fast_io_fail_tmo 5 } } RHEL 7.
prio alua path_checker tur rr_weight priorities failback immediate no_path_retry fail path_selector "round-robin 0" flush_on_last_del no user_friendly_names "yes" alias_prefix "mpath-sperc" features "0" fast_io_fail_tmo 5 } } blacklist { wwid "" } For all versions up to SLES 11 SP4 defaults { verbosity 2 polling_interval 10 uid_attribute "ID_SERIAL" checker_timeout 90 } devices { device { vendor "DELL" product "Shared PERC8" hardware_handler "1 alua" path_grouping_policy failover prio alua pa
rr_weight priorities failback immediate no_path_retry fail path_selector "round-robin 0" flush_on_last_del no user_friendly_names "yes" alias_prefix "mpath-sperc" features "0" fast_io_fail_tmo 5 } } SLES 12 defaults { verbosity 2 polling_interval 10 uid_attribute "ID_SERIAL" checker_timeout 90 } devices { device { vendor "DELL" product "Shared PERC8" hardware_handler "1 alua" path_grouping_policy failover prio alua path_checker tur rr_weight priorities failback immediate no_path_retry fail path_selector "ro
user_friendly_names "yes" alias_prefix "mpath-sperc" features "0" fast_io_fail_tmo 5 } } blacklist { wwid "" } Clustering setup in Linux • When configuring STONITH sbd devices, ensure that the Message Timeout is set to 120 seconds and the Watchdog Timer is set to at least 60 seconds. This is to ensure that the SPERC controller functions correctly.
6 Management applications for Shared PERC 8 card The Dell Chassis Management Controller (CMC) allows you to: • Configure the shared RAID system • Create and manage multiple disk groups • Control and monitor multiple RAID systems • Enable LKM on Shared PERC • Provide online maintenance through a web interface or Command Line Interface (CLI) NOTE: Dell OpenManage Storage Management, BIOS Configuration Utility (Ctrl+L), or Unified Extensible Firmware Interface (UEFI) RAID Configuration Utility does no
7 Security key and RAID management NOTE: PERC H310 does not support any security implementations. Topics: • Security key implementation • Troubleshooting security key errors Security key implementation The Dell Shared PowerEdge RAID Controller 8 (SPERC8) internal and external cards support Self-Encrypting Disks (SED) for protection of data against loss or theft of SEDs. Protection is achieved by the use of encryption technology on the drives. There is one security key per controller.
Troubleshooting security key errors Secured foreign import errors A foreign configuration is a RAID configuration that already exists on a replacement physical disk that you install in a system. A secured foreign configuration is a RAID configuration that was created under a different security key.
8 Troubleshooting To get help with your Dell Shared PowerEdge RAID Controller Shared (PERC) 8 card, you can contact your Dell Technical Service representative or see Dell.com/contactdell. Topics: • General issues • Physical disk issues • Virtual disks issues • Driver issues • Controller issues • Hard drive indicator codes General issues Shared PERC 8 card not seen in Device Manager Issue: The device does not appear in the Device Manager.
Corrective Action: These error messages may be triggered under some noncritical cases. See the system status and the CMC logs to identify any persistent failures. NOTE: Event 153 or Event 11 seen in the Windows Event log do not require any corrective action. Shared PERC 8 cards fail to function Issue: Shared PERC 8 cards fail to function in systems with the Fault Tolerant Shared PERC 8 card configuration.
Physical disk issues Physical disk in failed state Issue: One of the physical disks in the disk array is in the failed state. Corrective Action: Update the Shared PERC 8 cards to the latest firmware available at Dell.com/support. Replace the physical disk with a working one which is compatible with the old physical disk and is of equal or greater capacity. Capable speed of a drive being shown as blank Issue: Capable Speed under Physical Disk → Properties is Blank for 12 Gbps SAS drives.
Rebuilding a Failed physical disk Issue: Rebuilding a physical disk after one of them is in a failed state. Corrective Action: If you have configured hot spares, the Shared PERC 8 card automatically tries to rebuild a physical disk that is in a failed state. The automatic rebuild is not available if there is no hot spare with enough capacity to rebuild the failed physical disk. You must make sure a physical disk with enough storage in the subsystem is set as a hot spare for rebuilding the physical disk.
4 Perform the Auto Replacement of a Virtual Disk operation. NOTE: The Auto Replacement of Virtual Disk operation allows you to copy data from a source physical disk of a virtual disk to a target physical disk that is not a part of the virtual disk. For more information about the Auto Replacement feature, seeAuto replace member. SMART error detected on a physical disk in a nonredundant virtual disk Issue: A SMART error is detected on a physical disk in a nonredundant virtual disk.
Corrective Action: No corrective action Virtual disks issues Degraded state of virtual disks Issue: A redundant virtual disk is in a degraded state when one or more physical disks have failed or are inaccessible. For example, if a RAID 1 virtual disk consists of two physical disks and one of them fails or becomes inaccessible, the virtual disk becomes degraded. Corrective Action: To recover a virtual disk from a degraded state, you must replace the failed physical disk and rebuild it.
Shared PERC 8 card has yellow exclamation mark in device manager Issue: The device is displayed in Device Manager but has a yellow exclamation mark (!). Corrective Action: Reinstall the driver. For more information on reinstalling drivers, see Driver Installation. If reinstalling the driver does not resolve the yellow exclamation mark, perform a shutdown of all server modules and power cycle the chassis.
Conditions Actions NOTE: If a Reset Configuration action is performed in CMC while pinned cache is present, the cache will be cleared and the data will not be recoverable. • Clear the pinned cache through CMC. NOTE: Clearing pinned cache means that the pinned data is lost. NOTE: Unresolved pinned cache can occur if a virtual drive goes offline for any reason while there is uncommitted cache data in memory. NOTE: When pinned cache is present, the controller failover feature will not occur.
Category Event Reported To the CMC Description of the Failure Controller in Chassis Slot X has entered safe mode with limited functionality due to Critical Error during boot All drives will be hidden Corrective Action configuration but both cards are not set to Fault Tolerant mode. The controller is cabled incorrectly and discovery detects a loop, or duplicate SAS address. • Allow cache to flush, this may include reinserting removed disks, or replacing cables. • Pinned Cache.
Table 8. Hard drive indicator codes Drive-status indicator pattern (RAID only) Condition Flashes green twice per second Identifying drive or preparing for removal. Off Drive ready for insertion or removal. NOTE: The drive status indicator remains off until all hard drives are initialized after the system is turned on. Drives are not ready for insertion or removal during this time.
9 Getting help Topics: • Contacting Dell • Locating Service Tag of your system Contacting Dell Dell provides several online and telephone based support and service options. If you do not have an active internet connection, you can find contact information about your purchase invoice, packing slip, bill, or Dell product catalog. Availability varies by country and product, and some services may not be available in your area.
