Sun StorEdge™ 3000 Family Best Practices Manual Sun StorEdge 3510 FC Array Sun Microsystems, Inc. www.sun.com Part No. 816-7325-12 October 2003, Revision A Submit comments about this document at: http://www.sun.
Copyright © 2003 Dot Hill Systems Corporation, 6305 El Camino Real, Carlsbad, California 92009, USA. All rights reserved. Sun Microsystems, Inc. and Dot Hill Systems Corporation may have intellectual property rights relating to technology embodied in this product or document. In particular, and without limitation, these intellectual property rights may include one or more of the U.S. patents listed at http://www.sun.com/patents and one or more additional patents or pending patent applications in the U.S.
Contents Overview 1 Introduction Overview 2 3 Fibre Channel Architecture Fibre Channel Protocols 3 Supported RAID Levels 4 Logical Drives 3 4 Cache Optimization 5 Array Management Tools 6 Saving and Restoring Configuration Information Direct-Attached Storage Storage Area Networking Scaling Capacity First Steps 7 7 8 9 10 General Configuration Considerations Non-Redundant DAS Configurations 11 12 Non-Redundant DAS Tips and Techniques Non-Redundant DAS Setup Details High-Availability
High-Availability DAS Tips and Techniques High-Availability DAS Setup Details Full-Fabric SAN Configurations 17 19 Full-Fabric SAN Tips and Techniques Full-Fabric SAN Setup Details 20 20 High-Performance SAN Configurations 23 High-Performance SAN Tips and Techniques High-Performance SAN Setup Details Summary iv 16 26 Sun StorEdge 3000 Family Best Practices Manual • October 2003 24 24
Best Practices for the Sun StorEdge 3510 FC Array This document highlights Fibre Channel (FC) best practices which apply to the Sun StorEdge 3510 FC array. Overview The Sun StorEdge 3510 array is a next generation Fibre Channel storage system designed to provide direct attached storage (DAS) to entry-level, mid-range, and enterprise servers, or to serve as the disk storage within a storage area network (SAN).
Introduction There are two popular methods for connecting storage to servers. ■ The original and still most widely used approach uses a direct connection between a server and its storage system. A storage system connected in this manner is commonly referred to as direct-attached storage (DAS). The DAS solution of connecting each server to its own dedicated storage system is straightforward, and the absence of storage switches can minimize costs in some instances.
Overview Fibre Channel Architecture The Sun StorEdge 3510 FC array RAID controller has six fibre channels that can support 1 or 2 Gb data transfer speeds. RAID controller channels 0, 1, 4, and 5 are normally designated for connection to hosts or Fibre Channel switches. RAID controller channels 2 and 3 are dedicated drive channels. In a dual RAID controller configuration, both RAID controllers have the same host channel designators, due to the architecture of the loops within the chassis.
In a point-to-point configuration, only one ID can be assigned to each host channel. If more than one ID is assigned, the point-to-point protocol rules are violated. Any host channel with more than one ID will not be able to log in to an FC switch in fabric mode. This “one-ID-per-channel” requirement is true in both single-controller and dual- controller configurations. Thus, in dual-controller configurations, either the primary or the secondary controller can have an ID assigned, but not both.
With at least one LD assigned to each controller, both controllers are active. This configuration is known as an active-active controller configuration and allows maximum use of a dual controller array's resources. Each LD can be partitioned in up to 128 separate partitions or used as a single partition. The partitions are presented to the host as LUNs.
Numerous controller parameters are also changed to optimize for sequential or random I/O. The change takes effect after the controller resets. Sequential or random optimization must be set prior to creating logical drives. There are two limitations that apply to the optimization modes. ■ One optimization mode must be applied to all logical drives in a RAID array. ■ Once the optimization mode is selected and logical drives are created, the optimization mode of those logical drives cannot be changed.
■ When your array is on a public network, it is susceptible to viruses, worms, and other malware attacks. ■ A variety of security software is available to detect and mitigate these attacks. Some port-scanning and other security software can have an adverse impact on your ability to access data. In extreme cases, some of this software can cause Sun StorEdge 3000 Family arrays to hang.
Standard DAS Configuration FIGURE 2 High Availability DAS Configuration Two DAS Configurations The actual number of servers that can be connected varies according to the number of Sun StorEdge 3510 FC array controllers. It also depends on the quantity of Fibre Channel connections used for each server and the total number of small form-factor pluggable (SFP) interface modules installed.
The SAN can also support multiple Sun StorEdge 3510 FC arrays. Increasing the number of StorEdge arrays makes more performance and capacity available within the storage network for sharing among the servers connected to the SAN. A SAN also provides great flexibility in how storage capacity can be allocated among servers and eliminates cabling changes when reallocation of storage becomes necessary.
Up to 12 disks Up to 24 disks Up to 36 disks Scalability FIGURE 3 Increasing Capacity First Steps There are two simple yet effective approaches for designing a Sun StorEdge 3510 FC array solution into your environment. Both methods allow for the rapid estimation of an appropriate DAS or SAN solution. Regardless of which method is used, the storage needs of each application and server involved must be identified to establish the total amount of storage capacity required.
environment. For environments with different server configurations, choose the solution that best matches the servers whose applications are mission-critical or most important. General Configuration Considerations The entry-level configuration for an FC array uses only one RAID controller. If this configuration is used, two single-controller arrays should use host-based mirroring to ensure high reliability, availability, and serviceability (RAS).
Non-Redundant DAS Configurations Note – Using single connections between Sun StorEdge 3510 SCSI arrays and servers creates single points of failure (SPOF) that can cause interruptions in the event a connection becomes unreliable or fails. This is not a recommended configuration unless host-based mirroring is utilized to protect against single points of failure.
TABLE 1 Configuration Overview for Non-Redundant DAS Single Server Configurations Dual Server Configurations Quad Server Configurations Number of Servers 1 2 4 RAID Enclosures 1 1 1 Expansion Units As needed As needed One or more Number of Controllers 1 1 1 Number of Disks 5 or more 12 or more 24 or more Cache Optimization Random or sequential Random or sequential Random or sequential RAID Levels Application-dependent Application-dependent Application-dependent Fibre connecti
Non-Redundant DAS Setup Details Server 1 FIGURE 5 TABLE 2 Server 3 Server 4 Server 2 Non-Redundant DAS Connections Setup Summary for Non-Redundant DAS Channel Number Primary ID Number Secondary ID Number 0 40 N/A 1 43 N/A 2 14 N/A 3 14 N/A 4 44 N/A 5 47 N/A The general procedure for creating this configuration follows. 1. Check the position of installed SFP modules. Move or add SFP modules as necessary to support the connections needed. 2. Connect expansion units if needed. 3.
5. Map Logical Drive 0 to controller channel 0. 6. Map Logical Drive 1 (if created) to controller channel 5. 7. Map Logical Drive 2 (if created) to controller channel 1. 8. Map Logical Drive 3 (if created) to controller channel 4. 9. Connect the first server to upper controller port 0. 10. Connect the second server (if needed) to controller port 5. 11. Connect the third server (if needed) to controller port 1. 12. Connect the fourth server (if needed) to controller port 4.
TABLE 3 Configuration Overview for High Availability DAS Single Server Configurations Dual Server Configurations Quad Server Configurations Number of Servers 1 2 4 RAID Enclosures 1 1 1 Expansion Units As needed As needed One or more Number of Controllers 2 2 2 Number of Disks 5 or more 12 or more 24 or more Cache Optimization Random or sequential Random or sequential Random or sequential RAID Levels Application-dependent Application-dependent Application-dependent Fibre conn
High-Availability DAS Setup Details Server 1 Server 2 Server 3 Server 4 FIGURE 7 TABLE 4 High-Availability DAS Connections Setup Summary for High-Availability DAS Channel Number Primary ID Number Secondary ID Number 0 40 41 1 43 42 2 14 15 3 14 15 4 44 45 5 47 46 The general procedure for creating this configuration follows. 1. Check the position of installed SFP modules. Move them as necessary to support the connections needed.
2. Connect expansion units if needed. 3. Configure cache optimization. 4. Ensure fibre connection is set to loop mode. 5. Configure target IDs. 6. Create one Logical Drive for each server and configure spare disks. 7. Map Logical Drive 0 to channels 0 and 5 of the primary controller. 8. Map Logical Drive 1 (if created) to channels 1 and 4 of the secondary controller. 9. Map Logical Drive 2 to channels 0 and 5 of the primary controller 10.
Full-Fabric SAN Configurations Typical Full-Fabric SAN Configuration FIGURE 8 TABLE 5 Configuration Overview for a Full-Fabric SAN Small Configuration Medium Configuration Large Configuration Number of Servers 2 to 4 2 to 14 2 to 62 RAID Enclosures 1 1 1 Expansion Units As needed As needed As needed Number of Controllers 2 2 2 Number of Disks 12 or more 12 or more 12 or more Cache Optimization Random or sequential Random or sequential Random or sequent RAID Levels Application
TABLE 5 Configuration Overview for a Full-Fabric SAN (Continued) Host Adapters per server Dual-port 2-Gbit FC Dual-port 2-Gbit FC Dual-port 2-Gbit FC Traffic Manager Required Required Required Storage Switches Two 8-port 2-Gbit FC fabric switches Two 16-port 2-Gbit FC fabric switches Two 32-port 2-Gbit FC fabric switches Full-Fabric SAN Tips and Techniques ■ In the fabric SAN configuration, the switches communicate with the Sun StorEdge 3510 FC array host ports using a fabric point-to-point (
Server 1 Server 2 Switch 3 Switch 4 Diagram showing a full-fabric SAN configuration FIGURE 9 TABLE 6 Full-fabric SAN connections Setup Summary for Full Fabric SAN Channel Number Primary ID Number Secondary ID Number 0 40 N/A 1 N/A 42 2 14 15 3 14 15 4 44 N/A 5 N/A 46 Best Practices for the Sun StorEdge 3510 FC Array 21
The general procedure for creating this configuration follows. 1. Check the position of installed SFP modules. Move them as necessary to support the connections needed. 2. Connect expansion units if needed. 3. Configure cache optimization 4. Ensure fibre connection option is set to point-to-point. 5. Ensure only one target ID per channel is configured. 6. Create at least two logical drives and configure spare disks. 7. Create one or more logical drive partitions for each server. 8.
High-Performance SAN Configurations Typical High-Performance SAN Configurations FIGURE 10 TABLE 7 Configuration Overview for High-Performance SAN Small Configuration Medium Configuration Large Configuration Number of Servers 2 to 4 2 to 14 2 to 62 RAID Enclosures 1 1 1 Expansion Units As needed As needed As needed Number of Controllers 2 2 2 Number of Disks 12 or more 12 or more 12 or more Cache Optimization Random or sequential Random or sequential Random or sequential RAID L
TABLE 7 Configuration Overview for High-Performance SAN (Continued) Small Configuration Medium Configuration Large Configuration Host Adapters per server Dual-port 2-Gbit FC Dual-port 2-Gbit FC Dual-port 2-Gbit FC Traffic Manager Required Required Required Storage Switches Two 8-port 2-Gbit FC fabric switches Two 16-port 2-Gbit FC fabric switches Two 32-port 2-Gbit FC fabric switches High-Performance SAN Tips and Techniques ■ In the high-performance SAN configuration, the switches communica
Server 1 Server 2 Switch 1 Switch 2 Diagram showing high-performance SAN connections FIGURE 11 TABLE 8 High-Performance SAN Connections Setup Summary for a High-Performance SAN Channel Number Primary ID Number Secondary ID Number 0 40 41 1 43 42 2 14 15 3 14 15 4 44 45 5 47 46 The general procedure for creating this configuration follows. 1. Check the position of installed SFP modules. Move them as necessary to support the connections needed.
2. Connect expansion units if needed. 3. Configure cache optimization. 4. Ensure fibre connection option set to loop mode. 5. Configure target IDs. 6. Create at least two logical drives and configure spare disks. 7. Create one or more logical drive partitions for each server. 8. Map Logical Drive 0 to channels 0, 1, 4 and 5 of the primary controller 9. Map Logical Drive 1 to channels 0, 1, 4 and 5 of the secondary controller. 10.
