SGI® Altix® 450 System User’s Guide Document Number 007-4857-002
CONTRIBUTORS Written by Mark Schwenden Illustrated by Chrystie Danzer Production by Mark Schwenden Engineering contributions by Rich Altmaier, Steve Bowen, Michael T. Brown, Dick Brownell, David Collins, Thomas Crapisi, Steve Dean, Hartmut Gottwald, Paul Wiley, Gary Meyer, Galen Flunker, Paul Kinyon, Mark Koneazny, William Kellerman, Paul Pedersen, Gregory Thorson, and Gary Spilde. COPYRIGHT © 2007 SGI. All rights reserved; provided portions may be copyright in third parties, as indicated elsewhere herein.
Record of Revision Version Description -001 August 2006, First release -002 March 2007, Engineering updates to cover RAID 1, DVD-R/W, 2D graphics and IA2.
Contents Contents 1. 007-4857-002 List of Figures . . . . . . . . . . . . . . . . . . . . . . . . xi List of Tables . . . . . . . . . . . . . . . . . . . . . . . xv About This Guide . . . . . . . . . . . . . . . . . . . . . . .xvii Audience. . . . . . . . . . . . . . . . . . . . . . .xvii Important Information . . . . . . . . . . . . . . . . . . . . . .xvii Chapter Descriptions . . . .
Contents Installing a Rack . . . . . . . . . . . . . . . . . . . . . . . . 6 Preinstallation Activities . . . . . . . . . . . . . . . . . . . . 6 Site Plan Verification . . . . . . . . . . . . . . . . . . . . 6 Tools Required . . . . . . . . . . . . . . . . . . . . 7 Power Receptacle Verification . . . . . . . . . . . . . . . . . . 8 Unloading and Moving System Equipment . . . . . . . . . . . .
Contents 2. Operation Procedures . . . . . . . . . . . . . . . . . . . . . . 47 Precautions . . . . . . . . . . . . . . . . . . . . . . . . . 47 ESD Precaution . . . . . . . . . . . . . . . . . . . . . . . 47 Safety Precautions . . . . . . . . . . . . . . . . . . . . . . 48 System Controller Network . . . . . . . . . . . . . . . . . . . . . 49 Console Connections . . . . . . . . . .
Contents System Components . 4. . . . . . . . . . . . . . . . . . . . . . 76 Bay (Unit) Numbering . . . . . . . . . . . . . . . . . . . . . 77 Rack Numbering. . . . . . . . . . . . . . . . . . . . . 77 Optional System Components . . . . . . . . . . . . . . . . . . . 78 System Control . . . . . . . . . . . . . . . . . . . . . . . . . 79 Levels of System Control . . . . . . . . . . . . .
Contents 6. Altix 42U and 20U Rack Features . . . . . . . . . . . . . . . . . . .102 Technical Specifications. . . . . . . . . . . . . . . . . . .104 Maintenance and Upgrade Procedures . . . . . . . . . . . . . . . . .107 Maintenance Precautions and Procedures . . . . . . . . . . . . . . . . .107 Preparing the System for Maintenance or Upgrade Returning the System to Operation . . . . . . . . . . . . . .108 . . . . .
Contents Physical Specifications . B. . . . . . . . . . . . . . . . . . . . . 154 Environmental Specifications . . . . . . . . . . . . . . . . . . . 156 Power Specifications . . . . . . . . . . . . . . . . . . . . 157 . I/O Port Specifications . . . . . . . . . . . . . . . . . . . . . 158 Ethernet Port. . . . . . . . . . . . . . . . . . . . . . . 159 Serial Ports . . . . . . . . . . .
List of Figures List of Figures 007-4857-002 Figure 1-1 30-amp Single-phase Power Receptacle for North American Sites . . . Figure 1-2 32-amp Single-phase Power Plug for International Sites . . 11 Figure 1-3 Dimensions of Tall Rack Shipping Crate . . . . . . . . . 13 Figure 1-4 Dimensions of Short Rack Shipping Crate . . . . . . . . . 14 Figure 1-5 Removing a Short Rack from the Shipping Crate . . . . . . .
List of Figures xii Figure 2-4 2D Graphics Card Location in IA2 Blade . . . . . . . . . 59 Figure 3-1 SGI Altix 450 System (Short Rack) . . . . . . . . . . . 68 Figure 3-2 SGI Altix 450 Server System (Tall Rack) . . . . . . . . . 69 Figure 3-3 Blade, IRU and Rack Components . . . . . . . . . 70 Figure 3-4 Functional Block Diagram of the Individual Rack Unit (IRU) . . . . 71 Figure 3-5 Altix 450 IRU System Components Example . . . . . . .
List of Figures 007-4857-002 Figure 6-23 Seating the Card in the Slot . . . . . . . .135 Figure 6-24 Removing the Card Access Panel From the IA Blade . . . . . .136 Figure 6-25 Replacing the IA Blade PCI Access Door . . . . . . . . .137 Figure 6-26 Location of Disk Drive Bays . . . . . . . . . . . . .138 Figure 6-27 Removing a Disk Drive . . . . . . . . . . . . . .139 Figure 6-28 Replacing a Disk Drive . . . . . . . . . . . . . .
List of Tables List of Tables 007-4857-002 Table 1-1 Installation Tools . . . . . . . . . . . . . . . . 7 Table 1-2 IRU Space Requirements. . . . . . . . . . . . . . 24 Table 5-1 Tall Altix Rack Technical Specifications . . . . . . . . .104 Table 5-2 Short Altix Rack Technical Specifications . . . . . . . . .104 Table 6-1 Customer-replaceable Components and Maintenance Procedures . . .109 Table 6-2 SGI PCIe Support Levels . . . . . . . .
About This Guide This guide provides an overview of the installation, architecture, general operation and descriptions of the major components that compose the SGI® Altix® 450 family of systems. It also provides the standard procedures for powering on and powering off the system, basic troubleshooting and maintenance information, and important safety and regulatory specifications. Audience This guide is written for owners, installers, system administrators, and users of SGI Altix 450 computer systems.
About This Guide • Using the ESI/ops panel (operating panel) on optional mass storage bricks. See the user’s guide for your optional SGI mass storage enclosure. Chapter Descriptions The following topics are covered in this guide: xviii • Chapter 1, “System Installation” guides an installer through the steps of unpacking and positioning the system rack(s). Rack mounting guidelines and basic cabling for the IRUs is also covered.
Related Publications Related Publications The following SGI documents are relevant to the Altix 450 series system: • SGI L1 and L2 Controller Software User’s Guide (P/N 007-3938-00x) This guide describes how to use the L1 and L2 controller commands at your system console to monitor and manage your SGI system.
About This Guide References in the documentation to these pages include the name of the command and the section number in which the command is found. For additional information about displaying man pages using the man command, see man(1). In addition, the apropos command locates man pages based on keywords. For example, to display a list of man pages that describe disks, type the following on a command line: apropos disk For information about setting up and using apropos, see apropos(1).
Conventions Conventions The following conventions are used throughout this document: Convention Meaning Command This fixed-space font denotes literal items such as commands, files, routines, path names, signals, messages, and programming language structures. variable The italic typeface denotes variable entries and words or concepts being defined. Italic typeface is also used for book titles. user input This bold fixed-space font denotes literal items that the user enters in interactive sessions.
About This Guide Reader Comments If you have comments about the technical accuracy, content, or organization of this document, contact SGI. Be sure to include the title and document number of the manual with your comments. (Online, the document number is located in the front matter of the manual. In printed manuals, the document number is located at the bottom of each page.) You can contact SGI in any of the following ways: • Send e-mail to the following address: techpubs@sgi.
Chapter 1 1. System Installation This chapter describes how to unpack and install your Altix 450 system.
1: System Installation System Installation Overview In this chapter, different installation instructions are provided for different types of system installations. Follow the instructions for your type of installation, as follows: Note: It is assumed that you will be connecting a system console to your system in each installation case.
Safety Precautions • • If you have ordered either a single- or dual-IRU system that you intend to install on a table top, follow the instructions in these sections: – “Safety Precautions” on page 3 – “Unpacking and Inspecting Modules” on page 22 – “Installing the IRUs in a Rack” on page 24 – “Cabling the System IRUs to a Power Source” on page 31 – “Connecting the System Console” on page 36 If you have your own rack, and have ordered system modules with the intention of rackmounting and cabling
1: System Installation Hazard Statements During the installation of your Altix 450 system, be alert for hazard advisory statements with icons, which signify the following: • Caution indicates a potentially hazardous situation that, if not avoided, can result in minor or moderate injury. A caution statement also alerts you to unsafe practices that can result in equipment damage and/or data corruption.
Safety Precautions Safety Measures Observe the following safety measures when you install the system: • Use caution when you remove the system from the shipping crate. Failure to handle the system carefully can result in personal injury or property damage. Caution: Ensure that the shipping crate is positioned close to its destination before you unpack the crate. ! Warning: Employ a minimum of two people to lift an IRU, to move the IRU(s) from one location to another, and to install the IRU(s) in a rack.
1: System Installation Installing a Rack This section describes how to install a rack that already has your system IRUs rackmounted and cabled together.
Installing a Rack Tools Required Table 1-1 lists the tools that you need to complete the installation. Table 1-1 007-4857-002 Installation Tools Tool Part Number Purpose 13-mm wrench 7260744 Adjust the leveling pads. 13-mm socket (3/8-in. drive) 7260726 Remove bracket bolts from tall rack shipping crate. 19-mm socket (3/8-in. drive) 9470618 Remove bolts from short rack shipping crate. Extension, 6-in. (3/8-in. drive) 7260655 Used with ratchet and sockets. Ratchet, reversible (3/8-in.
1: System Installation Power Receptacle Verification Ensure that a qualified technician installs the correct power receptacles. Your Altix 450 system uses one or two single-phase power receptacles. For North American sites, the single-phase receptacle is a 30-amp, 200- to 240-volt receptacle that has two phase sockets and one ground socket. For international sites, the single-phase receptacle is a 32-amp, 200-volt receptacle that has one phase socket, one neutral socket, and one ground socket.
Installing a Rack Pole X Power cord connector Ground pole Pole Y Socket Y Ground socket Socket X Receptacle Figure 1-1 007-4857-002 30-amp Single-phase Power Receptacle for North American Sites 9
1: System Installation For international sites, follow these steps to ensure that a single-phase power receptacle is properly wired: 1. Set the voltmeter to a high AC voltage range. 2. Check the voltage between socket 1 and socket 2 (see Figure 1-2). The meter should read between 200 and 240 VAC. 3. Check the voltage between socket 1 (line) and the ground socket. The meter should read between 200 and 240 VAC. Note: The ground pin is slightly larger in diameter than the other two pins. 4.
Installing a Rack Pin 2 (neutral) Pin 1 (line) Ground pin Socket 1 (line) Figure 1-2 007-4857-002 Power cord connector Ground socket Socket 2 (neutral) Receptacle 32-amp Single-phase Power Plug for International Sites 11
1: System Installation Unloading and Moving System Equipment Your Altix 450 system arrives at the site in a pallet-mounted cardboard shipping crate. For a short rack system, the documentation carton and the accessories carton are packed with the system. The documentation carton contains the system manuals as well as warranty and licensing information.
Installing a Rack If your site does not have a loading dock, arrange for a forklift to unload the system from the transportation vehicle. Ensure that two or three people are available to help unload the equipment. Move all crates slowly and carefully. Figure 1-3 shows the lift openings and dimensions of a tall rack shipping crate. This figure also shows where to position the pallet jack. Side view End view 81.5 (2070.1 mm) 52.0 (1320.8 mm) 42 (1066.
1: System Installation Figure 1-4 shows the lift openings and dimensions of a short rack shipping crate. This figure also shows where to position the pallet jack. Side view End view 42.75 (1068 mm) 29.25 (743 mm) 48.75 (1238 mm) Pallet jack position Figure 1-4 Dimensions of Short Rack Shipping Crate Inspecting the Shipping Crate After the system is unloaded from the truck, follow these steps before you unpack it: 1. Ensure that the crates and cartons arrive unopened. 2.
Installing a Rack Transporting the Shipping Crate Use a pallet jack with forks that are 48 in. (122 cm) long or longer to transport the shipping crate to the designated location. See Figure 1-3 and Figure 1-4 for the crate dimensions and location to position the pallet jack. If the crate does not fit through all access doors, you may need to partially disassemble the crate.
1: System Installation Removing a Short Rack from the Shipping Crate Warning: Be careful when you unpack and move the short rack system. Use a pallet jack to move the rack to the unpacking site. Ensure that the rack remains on a level surface and that the rack weight remains evenly distributed across the four casters. If you must lift the casters over an obstacle, such as a door threshold, use proper lifting techniques and employ a minimum of two people.
Installing a Rack 1 2 3 Ramp Cardboard cover Foam cushion 4a 4b 4c Rear mounting bracket 5 6 Figure 1-5 007-4857-002 Front mounting bracket Removing a Short Rack from the Shipping Crate 17
1: System Installation Unpacking and Installing the Tall Rack Warning: In its maximum configuration, a (42U) tall rack system weighs approximately 1,450 lb (658 kg). Use caution when you unpack and move this rack. A pallet jack is needed to move the rack prior to unpacking. Ensure that the rack remains on a level surface and that the rack weight remains evenly distributed across the four casters. To unpack a tall rack, you will need the following tools: • Extension, 6-inch (15.3 cm), 3/8-in.
Installing a Rack 1 2 3 Bungee cushion 5 7 COMPACT Foam plug COMPACT COMPACT COMPACT COMPACT 4 COMPACT COMPACT COMPACT COMPACT COMPACT 6 Figure 1-6 007-4857-002 Removing a Tall Rack from the Shipping Crate 19
COMPACT COMPACT COMPACT COMPACT 1: System Installation Figure 1-7 20 Reattaching the Tall Rack Doors 007-4857-002
Installing a Rack Positioning and Leveling Racks ! Caution: To avoid ESD damage to the electronic components, be sure to position the racks before you remove the ESD bags that cover the rack assemblies. To position and level your system racks, follow these steps: 1. Grasp the rear of each rack and roll the rack to its designated location. 2. Remove the ESD bags. 3. Adjust the leveling bolts of a rack, as shown in Figure 1-8, until the rack is level. Figure 1-8 Leveling Bolts 4.
1: System Installation Top view of short rack Top view of tall rack Seismic tie-down attachment points Casters Leveling pads Figure 1-9 Seismic Tie-down Attachment Points Unpacking and Inspecting Modules This section describes how to unpack and inspect individual IRUs. Follow these instructions if you plan to install individual modules in a rack or on a table top. Before unpacking your IRUs, inspect the packaging container for evidence of mishandling during transit.
Unpacking and Inspecting Modules Firmly connect the base module of the IRU to the fan module using these steps: 1. Align the rear connectors of the fan module with the rear of the IRU base module and carefully slide it in until it fully joins together. 2. Align the right side joining plate screw holes with the eight holes in the two adjacent modules. See Figure 1-10 for an example. 3. Insert and tighten the eight retention screws that hold the connection plate in place. COMPACT 4.
1: System Installation Installing the IRUs in a Rack This section describes how to install the IRUs that compose an Altix 450 system in a rack. This section also describes how to remove the IRUs from a rack. You can use the instructions to rackmount one or more IRUs in a 20U (short) or 42U (tall) Altix rack. ! Caution: Always use at least two people when moving, installing, or removing a rackmounted Altix 450 IRU.
Installing the IRUs in a Rack Determining Where to Attach Rails in a Rack The base IRU, and any additional IRUs, each require five units (5U) of vertical space within the rack (one unit is equivalent to 1.75 inches [44.5 mm]). To determine where you should install the unit’s shelf rails in the rack, you must count mounting holes. Each U contains three mounting holes; therefore, in the 5U of space that the module occupies, there are 15 mounting holes. The bottom hole of the 5U space is hole 1.
1: System Installation Rackmounting IRUs with Shelf Rails This section describes how to rackmount an IRU using shelf rails. The shelf rails, which are shipped with the module, support the module within the rack. To attach the shelf rails to the rack, follow these steps: 1. Locate the slot location in the rack in which you will install the IRU. See “Determining Where to Attach Rails in a Rack” on page 25 if you are uncertain. 2.
Installing the IRUs in a Rack Note: Even with the blades, power supplies and fan module disconnected, the IRU’s base unit is heavy. Always use two people when installing it into a rack. 7. Working from the front of the rack, set the rear edge of the IRU module on the shelf rails. Then slide the module into the rack until the module ears are snug against the front rack rails. See Figure 1-13. 8. Fasten the module to the front rails of the rack with the screws provided. See Figure 1-13. 9.
1: System Installation Removing a Rackmounted IRU Module To remove an IRU that is already mounted in a rack, follow these steps: 1. Power off the module. For instructions on how to power off the module, see “Powering the System On and Off” on page 38. 2. Disconnect all of the cables at the front of the module. 3. Removing the system blades and power supplies from the IRU is recommended to decrease the overall weight of the unit. Be sure to place any system blades in a static-free area.
NUMAlink Cabling the System IRUs NUMAlink Cabling the System IRUs Some configurations of an Altix 450 system will consist of either two or more IRUs. This section describes how to cable together these multiple units. If your system was shipped with all the cables pre-installed, go on to the next section. Note: If your system is made up of a single module, you can skip this section and proceed to “Cabling the System IRUs to a Power Source” on page 31 for instructions to connect your system to power.
COMPACT 1: System Installation inside COMPACT TM inside TM Figure 1-14 NUMAlink Cabling an IRU to an Additional IRU To connect two Altix 450 compute modules to each other with NUMAlink cables, follow these steps: 1. Power off both compute modules (see “Powering the System On and Off” on page 38). 2. Connect the NUMAlink cables to the NUMAlink connectors on both IRU modules. 3.
Cabling the System IRUs to a Power Source The console output should indicate that the processors and memory on both IRU modules have been discovered. If the secondary IRU module has not been discovered, use the reset L1 command to perform a reset of the system. Note: Use the network L1 command to set the network mode. (See the SGI L1 and L2 Controller Software User’s Guide (007-3938-00x) for more information on L1 commands.
1: System Installation inside TM Figure 1-15 Power-Supply Connectors on the IRU Module Figure 1-16 shows the connecting of a dual-IRU system located on either a tabletop or desktop, to a power source.
Cabling the System IRUs to a Power Source COMPACT Power distribution unit (PDU) inside COMPACT TM inside TM Power source Figure 1-16 007-4857-002 Connecting an Unracked IRU System to a Power Source 33
1: System Installation Connecting System Modules to a Rack Power Source If your Altix 450 system is shipped from SGI in a rack, there will be at least one power distribution unit (PDU) installed in the rack. Note that each single-phase PDU can support a maximum of two Altix 450 IRUs. Plug both power cords from each IRU into the PDU(s) installed in your short or tall rack. Figure 1-17 shows an example of cabling the system modules that reside in a rack to a PDU.
Cabling the System IRUs to a Power Source Figure 1-18 shows an example of the optional three-phase PDU available in tall racks only.
1: System Installation Connecting the System Console This section explains how to attach and establish a connection between a system console and an Altix 450 system. The system console enables you to perform the following activities: • Start a Linux console session to set up networking and other configuration parameters. • Monitor your system by reading the status and error message information that the L1 controller generates.
Connecting the System Console Console port Reset button TM de insi Ethernet port L1 controller panel Figure 1-19 007-4857-002 Location of the L1 Console Port 37
1: System Installation Operating Your System This section describes how to operate your Altix 450 system, as follows: • “Powering the System On and Off” on page 38 • “L1 Front Panel Controls” on page 44 • “Operating the L1 Controller” on page 45 Powering the System On and Off This section describes how to power on and power off your system (or an individual IRU) at a system console, or manually, as follows: • “Preparing to Power On” on page 38 • “Powering Off at the System Console” on page 42 •
Operating Your System Powering On With Optional System Console and L1/L2 Controller To power on and boot your system using the system console, follow these steps: 1. If not yet done so, connect a system console to the server system as described in “Connecting the System Console” on page 36. 2. If the monitors and other peripherals are equipped with voltage select switches, verify that they are set for the appropriate AC voltage and plug them in.
1: System Installation 4. Verify that all of the L1 controllers display “L1 running.” If any L1 controllers are not running, contact your SGI system support engineer (SSE). 5. At the system console, access the L2 controller by entering the following command: $> /stand/sysco/bin/12term 6. From the L2 prompt, display the system configuration by entering the following command: L2> cfg This command lists the IRUs that the L2 controller detects in the system and their system controller addresses.
Operating Your System Powering On Manually or With L1 Console If your system does not have a a local system console, you can manually power on and off the server by using the power button(s). To power on your system or an individual IRU manually, follow these steps: 1. If the monitors and other peripherals are equipped with voltage select switches, verify that they are set for the appropriate AC voltage and plug them in. Note that they are normally plugged into power sources outside the rack system. 2.
1: System Installation Powering Off at the System Console To power off your system using the system console, follow these steps: ! Caution: If you power off the system before you halt the operating system, you can lose data. 1. Shut down the operating system by entering the following command: # init 0 2. To access the L2 prompt, direct the keyboard input to the L2 controller by entering Ctrl+T. $> ctrl t 3. From the L2 prompt (L2>), power off an individual IRU by entering the following command.
Operating Your System Powering Off Manually To power off your system manually, follow these steps: ! Caution: If you power off the system before you halt the operating system, you can lose data. 1. Shut down the operating system by entering the following command: # init 0 2. Press the power buttons or power switches on each of the IRUs that you want to power off.
1: System Installation L1 Front Panel Controls The L1 front panel of the Altix 450 IRU provides the following control features, as shown in Figure 1-22: • Status LEDs. The front panel has the following LEDs: • Power button LED. This LED illuminates green when the internal components are on. • Service-required LED. This LED illuminates yellow to indicate that an item is not functioning properly (for example, a fan is off), but the IRU is operating. • Failure LED.
Operating Your System Note: If your system includes an optional storage module, see the user’s guide that comes with the product for operating instructions. Operating the L1 Controller The L1 controller operates in one of the two following modes: • L1 mode. The L1 prompt (001c01-L1>) is visible and all input is directed to the L1 command processor. • Console mode from L1. Output from the system is visible and all input is directed to the system.
1: System Installation escaping to L1 system controller 001c01-L1> l1 L1 command processor engaged, for console mode.
Chapter 2 2.
2: Operation Procedures Safety Precautions Warning: Before operating or servicing any part of this product, read the “Safety Information” on page 165. Danger: Keep fingers and conductive tools away from high-voltage areas. Failure to follow these precautions will result in serious injury or death. The high-voltage areas of the system are indicated with high-voltage warning labels. ! Caution: Power off the system only after the system software has been shut down in an orderly manner.
System Controller Network System Controller Network All Altix 450 enclosures contain an embedded microprocessor board and display assembly known as the system controller. This microprocessor runs an embedded version of the Linux operating system. The system controller runs off standby power and is running as long as the enclosure is connected to an active power source. There are two primary applications that run on the system controller.
2: Operation Procedures Console Connections A console is defined as a connection to the system that provides access to the system controller network. A console can be a personal computer (Ethernet connection) or a dumb terminal (serial connection).
Communicating with the System Controller Communicating with the System Controller Once a connection to the console is established, the user will be presented with either an L2 prompt (ethernet connection), known as L2 mode, or and L1 prompt (serial connection), known as L1 mode. From either of these prompts various system controller commands can be entered.
2: Operation Procedures Preparing to Power On To prepare to power on your system, follow these steps: 1. Check to ensure that the cabling between the rack’s power distribution unit (PDU) and the wall power-plug receptacle is secure. 2. For each individual IRU that you want to power on, make sure that the power cables are plugged into all the IRU power supplies correctly, as shown in Figure 2-1.
Powering the System On and Off Power distribution unit (PDU) On/off switches Power source Figure 2-2 007-4857-002 Single-Phase Rack PDU Circuit Breaker Switches 53
2: Operation Procedures Powering On and Off at the Console The power-on and off procedure at a console varies with your server setup, as follows: • If you have a console connected to a server with a serial interface, you can toggle between L1 and console mode. This enables you to power on your server with L1 commands and view the activity by changing to the console mode. For detailed instructions on using a system console using the L1 mode, see “Operating the L1” on page 84.
Powering the System On and Off oo1c01-L1> * power down (* indicates all) From the L1 prompt, display the system configuration information by entering the following command: 001c01-L1> config :0 001c01 L0C :2 002r01 L1H :4 002r05 L3H 001c01-L1> In L1 mode, you can obtain only limited information about the system configuration. An IRU has information about its internal blades, and also when other IRUs are NUMAlink attached to the IRU, information about those IRUs.
2: Operation Procedures Powering-On and Off From the L2 Commands issued at the L2 prompt are typically sent to all NUMAlink connected IRU enclosures in the system. To power on a system: system-101-L2> power up To power off a system: system-101-L2> power down Using Embedded Support Partner (ESP) Embedded Support Partner (ESP) automatically detects system conditions that indicate potential future problems and then notifies the appropriate personnel.
Monitoring Your Server Monitoring Your Server You can monitor your Altix 450 server from the following sources: • On the L1 controller’s display at the front of the IRU as shown in Figure 2-3, you can monitor system operational status. For example, you can monitor error messages that warn of power or temperature values that are out of tolerance. • You can connect and monitor the system via an Ethernet port adapter.
2: Operation Procedures 2D Graphics Video Interface If your system was ordered in January 2007 or later, it may come equipped with an optional 2D graphics board interface (supported with SLES 10 or later system software). This low-profile PCI interface card is installed in the base I/O blade in the Altix system (see Figure 2-4 on page 59). Note that this blade is commonly referred to as the IA2 blade. One 2D card is supported per system or partition; no audio function is supported.
PCI-X 0 Installing Optional Components PCI-X 1 2D graphics card ETH0 ETH1 DVDROM HDD1 P0 PF N0 N1 USB0 USB1 USB2 USB3 HDD0 F1 F2 F3 F4 SAS 4X A1 A2 A3 A4 Figure 2-4 2D Graphics Card Location in IA2 (base I/O) Blade Installing Optional Components Besides adding a system console, you can add or replace the following hardware items on your Altix 450 series server: • Peripheral component interface (PCI) cards into your system I/O blades. • Disk drives in your IA/IA2 blade (base I/O).
2: Operation Procedures Warning: You can add or replace only the items listed in this section. For your safety and for the protection of your server system, contact your SGI system support engineer (SSE) to install any hardware items not listed in this section. Warning: Before installing, operating, or servicing any part of this product, read the “Safety Information” on page 165. I/O Blade Overview System I/O is primarily accomplished through standard and optional blades in the SGI Altix 450 IRUs.
Installing Optional Components Adding or Removing Disk Drives The IA/IA2 blade (base I/O blade) within the IRU supports the system boot functions and contains one or two low-profile disk drives. See “Installing or Replacing a Disk Drive in the IA Blade” on page 138 for detailed instructions on installing or removing disk drives. For information on removing or replacing disk drives in optional mass storage enclosures, see the hardware document supplied with the storage option.
Select a device: 1. 2. 4. 8. 10. 13. 16. 18. 20. 21. 22. 23. 24. 39. 45. 60. 61. 97. 98. 99.
Installing Optional Components 1. 2. 4. 8. 10. 13. 16. 18. 20. 21. 22. 23. 24. 39. 45. 60. 61. 97. 98. 99.
2: Operation Procedures B___T___L Type 1. 0 0 0 Disk 2.
Installing Optional Components fs0:\> reset . . . EFI Boot Manager ver 1.10 [14.62] Partition 0: CBlades RBlades IOBlades 2 0 1 Nodes CPUs Mem(GB) Enabled Disabled 3 0 4 0 6 0 Please select a boot option EFI Shell >>> netboot Boot option maintenance menu Loading: netboot Running LoadFile() CLIENT MAC ADDR: 08 00 69 14 E3 0C CLIENT IP: 137.38.82.91 MASK: 255.255.255.0 GATEWAY IP: 137.38.82.252 137.38.82.254 DHCP IP: 137.38.228.4 TSize.Running LoadFile() Starting: netboot ELILO . . . .
2: Operation Procedures 1 volume is active, 2 physical disks are active Volume 0 is Bus 0 Target 0, Type IM (Integrated Mirroring) Volume State: optimal, enabled Volume Settings: write caching disabled, auto configure Volume draws from Hot Spare Pools: 0 Volume Size 139898 MB, Stripe Size 0 KB, 2 Members Primary is PhysDisk 1 (Bus 0 Target 56) Secondary is PhysDisk 0 (Bus 0 Target 1) For information on the two RAID 1 disks, use the lsscsi command as shown in the following example: # lsscsi [1:0:0:0] [1:0:
Chapter 3 3. System Overview This chapter provides an overview of the physical and architectural aspects of your SGI Altix 450 system. The major components of the Altix 450 series systems are described and illustrated. The Altix 450 series is a family of multiprocessor distributed shared memory (DSM) computer systems that initially scale from 2 to 76 Intel 64-bit processor cores as a cache-coherent single system image (SSI).
3: System Overview Figure 3-1 SGI Altix 450 System (Short Rack) System Models The basic enclosure within the Altix 450 system is the 5U high (8.68 inch or 22 cm) “individual rack unit” (IRU). The IRU enclosure houses a maximum of four single-wide blades and one double-wide blade. Each IRU comes with two built-in high-speed routers. The routers connect to the installed blades via a backplane. Each router has two ports that are brought out to external NUMAlink 4 connectors.
System Models Figure 3-2 SGI Altix 450 Server System (Tall Rack) You can also add additional PCI expansion blades or RAID and non-RAID disk storage to your server system. Figure 3-3 shows an individual blade, IRU and Rack.
3: System Overview Rack (Contains 4 IRUs) Blade Individual Rack Unit (IRU) (Contains 5 blades) insideTM Blade slot 4 Blade slot 3 Blade slot 2 Blade slot 1 Blade slot 0 insideTM insideTM insideTM insideTM Figure 3-3 Blade, IRU and Rack Components System Architecture The Altix 450 computer system is based on a distributed shared memory (DSM) architecture. The system uses a global-address-space, cache-coherent multiprocessor that scales up to 76 Intel 64-bit processor cores in a single rack.
System Architecture interface. The SHub ASIC memory interface enables any processor in the system to access the memory of all processors in the system. Another component of the NUMAlink 4 architecture is the router ASIC. The router ASIC is a custom designed 8-port crossbar ASIC. Using the router ASICs with a highly specialized backplane and NUMAlink 4 cables provides a high-bandwidth, extremely low-latency interconnect between all processor, I/O, and other option blades within the system.
3: System Overview System Features The main features of the Altix 450 series server systems are introduced in the following sections: • “Modularity and Scalability” on page 72 • “Distributed Shared Memory (DSM)” on page 72 • “Distributed Shared I/O” on page 73 • “Reliability, Availability, and Serviceability (RAS)” on page 74 Modularity and Scalability The Altix 450 series systems are modular systems.
System Features Memory latency is the amount of time required for a processor to retrieve data from memory. Memory latency is lowest when a processor accesses local memory. Distributed Shared I/O Like DSM, I/O devices are distributed among the blade nodes within the IRUs (each base I/O blade node has two NUMAlink ports) and are accessible by all compute nodes within the SSI through the NUMAlink interconnect fabric.
3: System Overview Non-uniform Memory Access (NUMA) In DSM systems, memory is physically located at various distances from the processors. As a result, memory access times (latencies) are different or “non-uniform.” For example, it takes less time for a processor blade to reference its locally installed memory than to reference remote memory.
System Features • • 007-4857-002 Power-on and boot: – Automatic testing occurs after you power on the system. (These power-on self-tests or POSTs are also referred to as power-on diagnostics or PODs). – Processors and memory are automatically de-allocated when a self-test failure occurs. – Boot times are minimized. Further RAS features: – Optional RAID 1 in base I/O (IA2 blade), check with your SGI sales or service representative for availability.
3: System Overview System Components The Altix 450 series system features the following major components: • 20U rack. The “short” rack is a custom rack used with the Altix 450 system. It holds up to 4 IRUs or a combination of IRUs and option modules (such as mass storage). • 42U rack. The “tall” rack is a custom rack used for both the compute and I/O rack in the Altix 450 system. Up to 8 IRUs can be installed in each rack. There is also a 2U space reserved at the top for an option module.
System Components PCI-X 1 PCI-X 0 L1 controller panel Port H ETH0 Port G ETH1 DVDROM HDD1 P0 PF N0 N1 USB3 HDD0 USB0 USB1 USB2 insideTM Port G F1 F2 F3 F4 Port H SAS 4X A1 A2 A3 A4 Power supplies Router System control board board Compute/memory blades Router board Figure 3-5 IA blade Altix 450 IRU System Components Example Bay (Unit) Numbering Bays in the racks are numbered using standard units. A standard unit (SU) or unit (U) is equal to 1.75 inches (4.445 cm).
3: System Overview Optional System Components Availability of optional components for the SGI 450 systems may vary based on new product introductions or end-of-life components. Some options are listed in this manual, others may be introduced after this document goes to production status. Check with your SGI sales or support representative for the most current information on available product options not discussed in this manual.
Chapter 4 4.
4: System Control Remote workstation monitor Local Area Network (LAN) Altix 450 Local Area Network (LAN) Cat-5 Ethernet Figure 4-1 SGI Altix 450 System Control Network (Example) Levels of System Control The system control network configuration of your server will depend on the size of the system and control options selected. Typically, an Ethernet connection to the system controller network is used. This Ethernet connection is usually made from the IRU housing the system disk(s).
System Controller Interaction The system controller is designed into all IRUs. An Ethernet connection directly from the IRU to a local private or public Ethernet allows the system to be administered directly from a local or remote console. Note that there is no interconnected system controller function in the optional storage modules.
4: System Control L1 Controller Functions The following list summarizes the control and monitoring functions that the L1 controller performs: • Controls voltage margining within the IRUs • Controls and monitors IRU fan speeds • Reads system identification (ID) PROMs • Monitors voltage levels and reports failures • Monitors and controls warning LEDs on the IRU • Monitors the On/Off power switch • Monitors the reset switch and the nonmaskable interrupt (NMI) switch • Reports the population of
Ethernet Switch The front panel display contains the following items: • 2 x 12 character liquid crystal display (LCD). The display identifies the IRU, shows system status, warns of required service, and identifies a failed component. • Power (On/Off) button (insert paper clip to actuate) and power on LED. • Service required LED. • Failure LED. • Reset switch (insert paper clip to actuate). • Non-maskable interrupt (NMI) switch (insert paper clip to actuate).
4: System Control Console Hardware Requirements The console type and how these console types are connected to the Altix 450 servers is determined by what console option is chosen. If you have an Altix 450 server with a dumb terminal, you can connect the terminal via a serial cable to the (DB-9) console port connector on the system control board of the IRU.
Operating the L1 application that is always running on the system controller. When the enclosure is connected to a LAN via the L2 host connector, the system controller spawns an application that provides L2 functionality. The L1 operates in one of these two modes, which are discussed in the sections that follow: L1 Mode The L1 prompt is visible and all input is directed to the L1 command processor.
4: System Control • “Viewing Information, Warnings, and Error Messages” on page 94 Viewing System Configuration (from an IRU’s Perspective) An L1 has limited knowledge of the system topology, depending on the system’s configuration. Typically, an L1 has information only about L1s that are directly NUMAlink connected. In large configurations with more than one L1, the L1 may have knowledge of only a portion of the L1s in the system.
Operating the L1 Commands can be targeted to other L1s by preceding the command with a rack and slot: 001c01-L1> 1.11 version The command above issues a version command to the IRU in rack 001, U position 11. Some commands can be targeted to a specific blade within an IRU. Precede the command with the blade designator: 001c01-L1> b1 power down The command above issues a power down command to the blade in blade slot 1 of the IRU in rack 001, U position 01.
4: System Control L1 Console Selection If the system contains more than one IRU and a serial connection is utilized for the console, the serial cable must be connected to the IRU that is located in the lowest rack and slot position. The select command shows the current console mode settings: 001c01-L1> select console input: 001c01 console0 console output: not filtered The following are common subchannels associated with console communications: • Subchannel 0A specifies Blade 0, CPU A.
L2 Operation 001c01-L1> entering console mode 001c01 console, to escape to L1 no response from 001c01 junk bus console UART:UART_TIMEOUT When this time-out condition occurs, either the IRU is hung or the subchannel is incorrect. An IRU is identified by its rack, type, and slot (001c01).
4: System Control To see if the L2 is running (it will be if the LAN cable is plugged in and the cable is connected to an active LAN). At the L1 prompt type: 001c01-L1> l2 L2 Controller is running. 001c01-L1> If the L2 is not running type: 001c01-L1>! init 4 This switches the system controller to run level 4 and forces the L2 to be started whether or not the LAN is plugged in. There is a space between the “!” and “init” Verify the L2 is running again as above.
L2 Operation Once the L2 is running, you can telnet to the L2, or use an optional SGIconsole. After the connection to the L2 controller is established, the following prompt appears, indicating that the L2 is ready to accept commands: olympic-101-L2> Common operations are discussed in the subsections that follow. Viewing System Configuration You can use the L2’s config command to view the current system configuration from an IRU level: olympic-101-L2> config L2 127.0.0.1: - 001 (LOCAL) L1 127.0.0.
4: System Control b is the enclosure type. ss is the slot location of the enclosure. p is the partition of the enclosure (not present if the system is not partitioned). In the example shown above, 001c01 is an IRU in rack 001 and slot position 01. Setting Command Targeting If a command is not understood by the L2 system controller, in general it is passed to the L1 system controllers. The destination determines which L1s receive the command.
L2 Operation To target individual blades in an IRU use the following syntax: olympic-101-L2> 1.11 b1 power down Executing the above command will power down the blade in blade slot 1 of the IRU in rack 001 U position 11. Default Destination When the L2 starts, the default destination is set to all racks and all slots.
4: System Control L2> r 2-4,7 This is a one-time destination. Command Interpretation Some L2 commands are the same as the L1 commands. In many cases, this is intentional because the L2 provides sequencing that is necessary for a command to function correctly. When L1 and L2 commands are similar, you can ensure that an L1 command is entered for the IRUs current destination by preceding the command with the l1 command: L2> r 2-4,7 l1 This is a one-time destination.
L2 Operation To power on or power off a specific IRU, specify a current destination: L2> r 2 s 5 power up L2> To power on or power off all IRUs in a partition, enter the following: L2> partition To reset the system, enter the following: L2> reset L2> This command restarts the system by resetting all registers to their default settings and rebooting the system controllers.
4: System Control Ctrl+T escaping to L2 system controller L2> l2 L2 command processor engaged, for console mode. L2> Console Selection When in console mode, the L2 communicates with the IRU set with the select command to be the system console or global master. All input from the console is directed to that IRU. You can set and view the system console with the select command.
L2 Operation • Subchannel 3C specifies Blade 3, CPU C. • Subchannel console0 Blade 0 console subchannel. • Subchannel console1 Blade 1 console subchannel. The select command output: “console input: 001c01 console0” shows that the L2 will send console input to IRU 001c01 blade 0 and the console subchannel will be used. To change the IRU that will be the system console, use the select . command, where is the rack and is the slot where the IRU is located: L2> select 1.
4: System Control Generally, the L2 will track the correct IRU rack, slot, blade, and subchannel for system console input and no user setting should be required. If the default value was manually overridden, it can be reset by entering the following command: 001c01-L1> select reset L1 Mode From L2 In L1 mode, the prompt from a single L1 is visible, and all input is directed to that L1 command processor.
Upgrading L1 Firmware Upgrading L1 Firmware The L1 firmware is currently distributed as part of the snxsc_firmware package. To determine which version of the package is installed on your system console, enter the following command: $> rpm -q snxsc_firmware If the package is installed, the full package name (including the revision) is returned: snxsc_firmware-1.18.3-1 The L1 firmware binary and the utilities used to update it are stored in /usr/cpu/firmware/sysco.
4: System Control You can update individual IRUs by replacing all with a rack and slot number: $> /usr/cpu/firmware/sysco/flashsc --12 10.1.1.1 /usr/cpu/firmware/sysco/l1.bin 1.16 This updates only the IRU in rack 1, slot 16.
Chapter 5 5. Rack Information This chapter describes the physical characteristics of the tall (42U) and short (20U) Altix racks in the following sections: • “Overview” on page 101 • “Altix 42U and 20U Rack Features” on page 102 • “Technical Specifications” on page 104 Overview At the time this document was published only the tall (42U) and short (20U) Altix racks shown in Figure 5-1 and Figure 5-2 were tested and approved for use with Altix 450 systems.
5: Rack Information Altix 42U and 20U Rack Features The Altix racks (shown in Figure 5-1 and Figure 5-2) have the following features and components: • Front and rear door. The front door is opened by grasping the wide end of the triangle-shaped door piece and pulling outward. Note that the tall rack uses a key lock for security purposes; the short (20U) rack does not. The rear door of the tall (42U) rack has a push-button key lock to prevent unauthorized access to the system.
Altix 42U and 20U Rack Features Figure 5-1 007-4857-002 Tall (42U) Altix Rack and Front Lock 103
5: Rack Information Figure 5-2 Short (20U) Altix Rack (Rear View) Technical Specifications Table 5-1 lists the technical specifications of the Altix 450 tall rack. Table 5-1 Tall Altix Rack Technical Specifications Characteristic Specification Height 79.5 in. (201.9 cm) Width 25.8 in. (65.5 cm) Depth 43.45 in. (110.4 cm) Weight (full) 1500 lb (682 kg) Table 5-2 lists the technical specifications of the Altix 450 short rack.
Technical Specifications Table 5-2 007-4857-002 Short Altix Rack Technical Specifications (continued) Characteristic Specification Depth 40.9 in. (103.
Chapter 6 6.
6: Maintenance and Upgrade Procedures Preparing the System for Maintenance or Upgrade To prepare the system for maintenance, follow these steps: 1. If you are logged on to the system, log out. Follow standard procedures for gracefully halting the operating system. 2. At your system console, switch from console mode to system control mode by entering the following command: $> Ctrl+T The system will respond with the system control prompt: L2> 3.
Installing or Removing Internal Parts Installing or Removing Internal Parts ! Caution: The components inside the system are extremely sensitive to static electricity. Always wear a wrist strap when you work with parts inside your system. To use the wrist strap, follow these steps: 1. Unroll the first two folds of the band. 2. Wrap the exposed adhesive side firmly around your wrist, unroll the rest of the band, and then peel the liner from the copper foil at the opposite end. 3.
6: Maintenance and Upgrade Procedures Replacing IRU Components While the individual rack units (IRUs) are not considered end-user replaceable, a number of their internal components can be removed and replaced.
Replacing IRU Components A B TM inside TM inside C Loosen screw inside Figure 6-1 TM Removing the L1 Controller Panel Replace the system control panel using the following steps: 1. Position the controller panel assembly slightly above the retention screw. 2. Carefully slid it back into place, see Figure 6-2.
6: Maintenance and Upgrade Procedures 3. Be careful not to snag the ribbon cable while replacing the panel. 4. Use the T10 torx driver to tighten the retention screw holding the panel to the IRU. 5. Restart the IRU or system as applicable.
Replacing IRU Components Removing or Adding IRU Power Supplies Under most circumstances a single power supply in an IRU can be replaced without shutting down the IRU or the complete system. In the case of a fully configured (loaded) IRU, this may not be possible. In a fully loaded system, power supply failure could result in shutdown of the IRU due to the power loss.
6: Maintenance and Upgrade Procedures 4. Install the new power supply with the retention latch at the top of the supply. 5. Slide the power supply into the chassis until the retention latch engages. 6. Reconnect the power cord to the power supply. 7. Reattach the L1 front panel using the information in “Remove and Replace the System Control Display Panel” on page 110.
Replacing IRU Components 3. A line will be added to the L1 system controller’s log file indicating the fan warning. 4. If ESP is used, a warning message will be sent to it also, (see “SGI Electronic Support” on page 149). The system controller monitors the temperature within each IRU. If the temperature increases due to the failed fan, the remaining fans will run at a higher RPM to compensate for the failed fan. The system will continue running until a scheduled maintenance occurs.
6: Maintenance and Upgrade Procedures A B Loosen screw Screw C Figure 6-6 Removing a Fan From the IRU 3. Slide a new blower assembly completely into the open slot, see Figure 6-7. 4. Tighten the blower assembly screw to secure the new fan.
Replacing IRU Components A B Tighten screw Figure 6-7 007-4857-002 Replacing an IRU Fan 117
6: Maintenance and Upgrade Procedures Removing or Replacing a System Blade From the IRU The blades installed in your SGI Altix 450 can generally be removed from or installed in an IRU without powering down the entire system. However, the operating system should be halted and the power to the IRU should be shut down in most cases. Note that the IA blade holding your system disk(s) should never be removed without backing up and gracefully powering down the unit.
Replacing IRU Components B COMPACT A insid eTM insid eTM COMPACT C insid eTM Figure 6-8 007-4857-002 Removing a Compute/Memory Blade From the IRU 119
6: Maintenance and Upgrade Procedures B insid eTM COMPACT COMPACT A insid eTM C insid eTM Figure 6-9 120 Replacing a Compute/Memory Blade in the IRU 007-4857-002
Replacing IRU Components Memory DIMM Group Placement and Guidelines Memory is contained on cards that are referred to as DIMMs (dual inline memory modules). Each individual memory or compute/memory blade can contain four, eight, or twelve DIMMs installed in DIMM slots located on the blade's node board. These twelve DIMM slots are organized into three groups of four DIMMs each, as shown in Figure 6-10. DIMMs are installed one per DIMM slot, and must be installed in groups of four.
6: Maintenance and Upgrade Procedures Left bus Y DIMM 2 (DIMM location 5) Left bus Y DIMM 1 (DIMM location 4) Left bus Y DIMM 0 (DIMM location 3) Left bus X DIMM 2 (DIMM location 2) Left bus X DIMM 1 (DIMM location 1) Left bus X DIMM 0 (DIMM location 0) Memory Group 2 NUMAlink 4 SHUB 2 Group 0 Front-side bus (FSB) SAW OSC Group 1 NUMAlink 4 Memory Right bus X DIMM 0 (DIMM location 0) Right bus X DIMM 1 (DIMM location 1) Right bus X DIMM 2 (DIMM location 2) Right bus Y DIMM 0 (DIMM location 3) Right
Replacing IRU Components DIMM slots Figure 6-11 DIMM Memory Slot Location Example Remove and Replace the DVD Drive Use the following steps to remove the DVD drive from an IA blade: 1. Power off the IRU, use instructions in “Powering the System On and Off” on page 51. 2. Extract the IA blade you are going to install the new DVD drive in. See the information in “Replacing IRU Components” on page 110. 3.
6: Maintenance and Upgrade Procedures 8. Slide the new DVD drive assembly into the IA blade housing. 9. Connect the DVD power and data cable connectors to the replacement drive. 10. Screw in the DVD drive assembly retention screw. 11. Replace the DVD access panel and tighten the retention screw. 12. Return the IA blade to its IRU slot. 13. Power-up the IRU and/or system.
Replacing IRU Components A B DVD retention screw CO MP AC T CO MP AC T D C Power connector Data connector CO MP AC T Figure 6-12 007-4857-002 DVD Removal from IA Blade 125
6: Maintenance and Upgrade Procedures A B Power connector Data connector CO MP AC T C D CO MP AC T CO MP AC T Figure 6-13 126 DVD Drive Replacement in the IA Blade 007-4857-002
Overview of PCI Express Operation Overview of PCI Express Operation This section provides a brief overview of the new PCI Express (PCIe) technology that will be available as an option with your system. PCI Express has both compatibility and differences with older PCI/PCI-X technology. Check with your SGI sales or service representative for more detail on PCI Express options available with the SGI Altix 450.
6: Maintenance and Upgrade Procedures 1”. SGI Altix 450 technology uses an x16 connector (64 wires) or “by 16” in PCI Express card slots. This technology will support PCIe boards that use connectors up to x16 in size. Table 6-2 shows this concept.
Adding or Replacing PCI/PCI-X Cards ! Caution: To protect the PCI cards from ESD damage, SGI recommends that you use a grounding wrist strap while installing a PCI card. Installing PCI Cards in the Three-Slot Blade Use the following instructions and illustrations to install or replace a PCI/PCI-X card in the three-slot blade: 1. Follow the instructions in “Replacing IRU Components” on page 110 and extract the three-slot blade from the IRU. 2.
6: Maintenance and Upgrade Procedures Figure 6-16 130 Extracting the Carrier Metal Filler Plate 007-4857-002
Adding or Replacing PCI/PCI-X Cards Note: If you are adding or replacing a half-height PCI card, skip ahead to step 6. 5. If the full-height card that you want to install does not fit in the carrier, follow these steps: a. Loosen the three screws on the guide bar with a Phillips screwdriver (if your carrier uses T8 Torx screws, you will need a Torx screw driver). b. Adjust the carrier guide bar (also known as the alignment rail) up or down as necessary (see Figure 6-17).
6: Maintenance and Upgrade Procedures c. Mount the card in the carrier, (see Figure 6-18) and ensure the following occurs: – The card connectors protrude through the bulkhead where the metal filler plate was located. – The edge connectors fit between the bottom guides of the carrier. – The top of the card fits under the clips on the guide bar. d. Tighten the three screws on the guide bar. Skip step 6 and proceed to step 7.
Adding or Replacing PCI/PCI-X Cards 6. To install a half-height PCI card in a carrier, follow these steps: a. Remove the two screws from the carrier guide bar, as shown in Figure 6-19. Note: Take care not to lose the threaded inserts in the carrier guide bar. b. Loosen the third screw of the carrier guide bar. c. Place the half-height PCI card in the carrier so that the connector edge of the card rests against the lower guide tab of the carrier (see Figure 6-19).
6: Maintenance and Upgrade Procedures e. Place the PCI card bracket so that the screw holes of the bracket align with the empty screw holes of the carrier guide bar, as shown in Figure 6-21. f. Secure the bracket to the carrier guide bar with two T8 Torx screws. g. Tighten the screw that you loosened in step 6b.
Adding or Replacing PCI/PCI-X Cards 7. Insert the carrier-mounted PCI card into the vacant slot, using the slot guide, as shown in Figure 6-22. Figure 6-22 Installing a Card in a Slot 8. Push in horizontally on the carrier seating bar, as shown in Figure 6-23, to seat the card securely in the slot. Figure 6-23 Seating the Card in the Slot 9. Install the blade into the IRU. 10. Power on the IRU or restart the system. 11. Run the ls -pci PCI hardware inventory command to verify the installation.
6: Maintenance and Upgrade Procedures Add or Remove a PCI Card in Base IA Blades To add or replace a PCI card in the base IA blade (base I/O), follow these steps: 1. Power off the IRU by following the power-off instructions in the “Powering the System On and Off” on page 51 if you are removing the IA blade. 2. Extract the blade you are going to install the new PCI card in. See the information in “Replacing IRU Components” on page 110 if you have not already removed the blade.
Adding or Replacing PCI/PCI-X Cards 2 CO MP AC T CO MP AC T 1 Figure 6-25 Replacing the IA Blade PCI Access Door 8. Replace the PCI card access door by installing it into the slotted opening and sliding it forward, as shown in Figure 6-25. 9. Reinstall the top PCI card access door screw. 10. Reinstall the front PCI card access door screw. 11. Reinstall the IA blade in the IRU chassis. 12. Return the system to service. 13. Boot your operating system software.
6: Maintenance and Upgrade Procedures Installing or Replacing a Disk Drive in the IA Blade PCI-X 0 PCI card slot 0 PCI-X 1 The IA/IA2 blade (base I/O) in an IRU has two system disk drive bays as shown in Figure 6-26. This section describes how to install or remove the drives.
Installing or Replacing a Disk Drive in the IA Blade Removing a System Disk Drive Before you remove a disk drive, you must power off the IRU or system according to the instructions in the “Powering the System On and Off” on page 51. To remove a disk drive, follow these steps: 1. Open the front of the rack and access the two disk drive bays, as shown in panel 1 of Figure 6-27. 2.
6: Maintenance and Upgrade Procedures Installing a System Disk Drive Before you install a disk drive, you must power off the IRU that houses the IA blade (base I/O) where the disk(s) are installed, see the instructions in the “Powering the System On and Off” on page 51. Note that you should always back up all files before replacement of the disk. Be sure to replace any Altix 450 hard disk with an SGI approved drive.
Figure 6-28 007-4857-002 COMPACT COMPACT Installing or Replacing a Disk Drive in the IA Blade Replacing a Disk Drive 141
Chapter 7 7.
7: Troubleshooting and Diagnostics Troubleshooting Chart Table 7-1 lists recommended actions for problems that can occur. To solve problems that are not listed in this table, use the SGI Electronic Support system or contact your SGI system support engineer (SSE). For more information about the SGI Electronic Support system, see the “SGI Electronic Support” on page 149. Table 7-1 Troubleshooting Chart Problem Description Recommended Action The system will not power on.
L1 Controller Error Messages L1 Controller Error Messages Table 7-2 lists error messages that the L1 controller generates and displays on the L1 display. This display is located on the front of the IRU. Note: In Table 7-2, a voltage warning occurs when a supplied level of voltage is below or above the nominal (normal) voltage by 10 percent. A voltage fault occurs when a supplied level is below or above the nominal voltage by 20 percent.
7: Troubleshooting and Diagnostics Table 7-2 L1 Controller Messages (continued) L1 System Controller Message Message Meaning and Action Needed Temperature messages: low alt. ATTN: advisory temperature reached @ xxC xxF The ambient temperature at the IRU’s air inlet has exceeded 30° C. ATTN: critical temperature reached @ xxC xxF The ambient temperature at the IRU’s air inlet has exceeded 35 °C.
LED Status Indicators LED Status Indicators There are a number of LEDs on the front of the IRUs that can help you detect, identify and potentially correct functional interruptions in the system. The following subsections describe these LEDs and ways to use them to understand potential problem areas. IRU Power Supply LEDs Each power supply installed in an IRU has a single bi-color (green/amber) status LED.
7: Troubleshooting and Diagnostics Compute/Memory Blade LEDs Each compute/memory blade installed in an IRU has a total of eight LED indicators arranged in two rows of four and behind the perforated of the blade: 148 • One green LED shows power-on complete status for the blade. • One red LED shows power failure or bad voltage status within the blade. • Two green NUMAlink indicators show NI0 and NI1 connection status between the blade and the router board within the IRU.
SGI Electronic Support Power fail NI0 link NI1 link Power on Heartbeat LEDs Logical processor A Logical processor D Logical processor B Figure 7-1 Logical processor C Compute Blade Status LED Locations SGI Electronic Support SGI Electronic Support provides system support and problem-solving services that function automatically, which helps resolve problems before they can affect system availability or develop into actual failures.
7: Troubleshooting and Diagnostics 1 Customer's system Implement solution 2 6 e-mail 3 Supportfolio Online 5 Page or e-mail alert View the case solutions SGI customer and SGI support engineer SGI global customer support center 4 SGI Knowledgebase Figure 7-2 150 Full Support Sequence 007-4857-002
SGI Electronic Support The sequence of events can be described as follows: 1. Embedded Support Partner (ESP) monitors your system 24 hours a day. 2. When a specified system event is detected, ESP notifies SGI via e-mail (plain text or encrypted). 3. Applications that are running at SGI analyze the information, determine whether a support case should be opened, and open a case if necessary. You and SGI support engineers are contacted (via pager or e-mail) with the case ID and problem description. 4.
The following three components compose the integrated SGI Electronic Support system: SGI Embedded Support Partner (ESP) is a set of tools and utilities that are embedded in the SGI Linux ProPack release. ESP can monitor a single system or group of systems for system events, software and hardware failures, availability, performance, and configuration changes, and then perform actions based on those events.
Appendix A A. Technical Specifications and Pinouts This appendix contains technical specification information about your system, as follows: • “System-level Specifications” on page 153 • “Physical Specifications” on page 154 • “Environmental Specifications” on page 156 • “Power Specifications” on page 157 • “I/O Port Specifications” on page 158 System-level Specifications Table A-1 summarizes the Altix 450 system configuration ranges.
A: Technical Specifications and Pinouts Physical Specifications Table A-2 shows the physical specifications of the Altix 450 system in a 42U rack. Table A-2 Altix 450 Tall Rack System Physical Specifications Feature Specification Dimensions for a single tall (42U) rack Height: 79.5 in. (201.9 cm) system, including doors and side Width: 25.8 in. (65.5 cm) panels Depth: 43.45 in. (110. 4 cm) Shipping dimensions Height: 81.5 in. (207 cm) Width: 42 in. (106.7 cm) Depth: 52 in. (132.
Physical Specifications Table A-3 Altix 450 Short Rack System Physical Specifications Feature Specification Dimensions for a single short (20U) Height: 41.8 in. (106.2 cm) rack system, including doors and side Width: 25.8 in. (65.5 cm) panels Depth: 40.9 in. (103.9 cm) Shipping dimensions Height: 47.5 in. (120.7 cm) Width: 31.25 in. (79.4 cm) Depth: 49.5 in. (125.7 cm) Weight of full rack (maximum) 750 lb (340 kg) Shipping weight (maximum) 862 lb (391 kg) Access requirements Front 48 in. (121.
A: Technical Specifications and Pinouts Environmental Specifications Table A-4 lists the environmental specifications of the 42U rack system. Table A-5 provides the environmental information for the 20U system. Table A-4 Environmental Specifications (42U Rack) Feature Specification Temperature tolerance (operating) +5 °C (41 °F) to +35 °C (95 °F) (up to 1500 m / 5000 ft.) +5 °C (41 °F) to +30 °C (86 °F) (1500 m to 3000 m /5000 ft. to 10,000 ft.
Power Specifications Table A-5 Environmental Specifications (20U Rack) (continued) Feature Specification Heat dissipation Altix 450 29.86 kBTU/hr maximum (based on 8.75 kW) Cooling requirement Ambient air cooling only Air flow: intake (front), exhaust (rear) Approximately 750 CFM (normal operation) Maximum altitude 10,000 ft. (3,049 m) operating 40,000 ft.
A: Technical Specifications and Pinouts I/O Port Specifications This section contains specifications and port pinout information for the base I/O ports of your system, as follows: 158 • “Ethernet Port” on page 159 • “Serial Ports” on page 160 • “External SAS/SATA Port Connector” on page 162 • “USB Type A Connector” on page 163 007-4857-002
I/O Port Specifications Ethernet Port The system auto-selects the Ethernet port speed and type (duplex vs. half-duplex) when the server is booted, based on what it is connected to. Figure A-1 shows the Ethernet port. Pin 4 Pin 3 Pin 5 Pin 6 Pin 7 Pin 2 Pin 1 Pin 8 Figure A-1 Ethernet Port Table A-7 shows the cable pinout assignments for the Ethernet port operating in 10/100-Base-T mode and also operating in 1000Base-T mode.
A: Technical Specifications and Pinouts Serial Ports The IRU modules have 9-pin serial interface connectors. These ports are for console interface and are capable of transferring data at rates as high as 230 kbps. Other features of the ports include the following: • Programmable data, parity, and stop bits • Programmable baud rate and modem control Figure A-2 shows a serial port example.
I/O Port Specifications Table A-8 shows pinout assignments for the 9-pin male DB-9 connector.
A: Technical Specifications and Pinouts External SAS/SATA Port Connector Figure A-3 shows the connector pin locations for the external single-channel Serial Attached SCSI (SAS) and Serial Advanced Technology Attachment (SATA) connector. This combined technology connector is used to support optional external storage. Table A-9 lists the pin assignments for the 14-pin SAS/SATA connector.
I/O Port Specifications USB Type A Connector Figure A-4 shows the USB type A connector provided on the base I/O that supports general USB applications and optional keyboard and mouse configurations. Table A-10 lists the pin assignments for the USB type A connector.
Appendix B B. Safety Information and Regulatory Specifications This appendix provides safety information and regulatory specifications for your system in the following sections: • “Safety Information” on page 165 • “Regulatory Specifications” on page 167 Safety Information Read and follow these instructions carefully: 1. Follow all warnings and instructions marked on the product and noted in the documentation included with this product. 2. Unplug this product before cleaning.
B: Safety Information and Regulatory Specifications 9. Do not attempt to service this product yourself except as noted in this guide. Opening or removing covers of blade and switch internal components may expose you to dangerous voltage points or other risks. Refer all servicing of items not listed in this guide to qualified service personnel. 10.
Regulatory Specifications Regulatory Specifications The following topics are covered in this section: • “CMN Number” on page 167 • “CE Notice and Manufacturer’s Declaration of Conformity” on page 167 • “Electromagnetic Emissions” on page 168 • “Shielded Cables” on page 170 • “Electrostatic Discharge” on page 170 • “Laser Compliance Statements” on page 171 • “Lithium Battery Statements” on page 172 This SGI system conforms to several national and international specifications and European Direc
B: Safety Information and Regulatory Specifications Electromagnetic Emissions This section provides the contents of electromagnetic emissions notices from various countries. FCC Notice (USA Only) This equipment complies with Part 15 of the FCC Rules. Operation is subject to the following two conditions: • This device may not cause harmful interference. • This device must accept any interference received, including interference that may cause undesired operation.
Regulatory Specifications Industry Canada Notice (Canada Only) This Class A digital apparatus meets all requirements of the Canadian Interference-Causing Equipment Regulations. Cet appareil numérique német pas de perturbations radioélectriques dépassant les normes applicables aux appareils numériques de Classe A préscrites dans le Règlement sur les interferences radioélectriques établi par le Ministère des Communications du Canada.
B: Safety Information and Regulatory Specifications Shielded Cables This SGI system is FCC-compliant under test conditions that include the use of shielded cables between the system and its peripherals. Your system and any peripherals you purchase from SGI have shielded cables. Shielded cables reduce the possibility of interference with radio, television, and other devices. If you use any cables that are not from SGI, ensure that they are shielded. Telephone cables do not need to be shielded.
Regulatory Specifications Laser Compliance Statements The DVD drive in this computer is a Class 1 laser product. The DVD drive’s classification label is located on the drive. Warning: Avoid exposure to the invisible laser radiation beam when the device is open. Warning: Attention: Radiation du faisceau laser invisible en cas d’ouverture. Evitter toute exposition aux rayons. Warning: Vorsicht: Unsichtbare Laserstrahlung, Wenn Abdeckung geöffnet, nicht dem Strahl aussetzen.
B: Safety Information and Regulatory Specifications Lithium Battery Statements Warning: If a lithium battery is a soldered part, only qualified SGI service personnel should replace this lithium battery. For other types, replace the battery only with the same type or an equivalent type recommended by the battery manufacturer, or the battery could explode. Discard used batteries according to the manufacturer’s instructions. Warning: Advarsel!: Lithiumbatteri - Eksplosionsfare ved fejlagtig håndtering.
Index A Altix server illustration, 69 Altix servers monitoring locations, 57 console mode from L1 mode, 87 from L2 emulator mode, 95 console selection, 96 controller L1, 81 customer service, xxi B battery statements, 172 block diagram IRU, 71 C cache coherency, 73 circuit breaker switches, 39 CMN number, 167 command targeting L1 mode, 86 L2 mode, 92 connector pin assignment SAS/SATA, 162 USB type A, 163 console L1 mode power on, 54 powering on bricks, 54 console communications, 96 subchannels, 96 007-4
Index fan RPM level, 114 front panel display L1 controller, 82 front panel LEDs, 44 I information, warning, and error messages viewing in L2 mode, 89, 94 install disk drive, 140 IRU block diagram, 71 IRU slot numbering, 77 K Knowledgebase, 152 L L1 console mode, 45 operation mode, 45 L1 console selection, 88 L1 controller, 57 description, 81 error messages, 145 front panel display, 82 front panel display items, 83 functions, 82 L1 firmware upgrade, 99 L1 mode, 45 command targeting, 86 from L1 software,
Index racks, 77 S PCI and PCI-X cards installation procedure PCI and PCI-X card installation, 128 PCIe slots, 60 physical specifications Altix Tall Rack Physical Specifications, 154 pinouts Ethernet connector, 159 serial connector, 160 power receptacle, 8-11 power specifications, 157 powering off L2 mode, 94 powering on at system console, 54 in L1 mode, 54 L2 emulator mode, 94 preparation, 52 product support, xxi SAS/SATA external connector pin assignments, 162 server monitored at L1 controller, 57 moni
Index U unpack a tall rack, 18 unpack and move the short rack, 16 upgrading L1 firmware, 99 USB type A connector pin assignments, 163 V viewing system configuration L1 mode, 86 voltage select switches for peripherals, 39 W warning messages viewing in L2 mode, 89, 94 weight of the short rack, 16 VGA video card, 57 176 007-4857-002
