SGI® UV 2000 System User Guide Document Number 007-5832-001
COPYRIGHT © 2012 SGI. All rights reserved; provided portions may be copyright in third parties, as indicated elsewhere herein. No permission is granted to copy, distribute, or create derivative works from the contents of this electronic documentation in any manner, in whole or in part, without the prior written permission of SGI.
Record of Revision Version Description 001 June 2012 First Release 007-5832-001 iii
Contents 1. 007-5832-001 List of Figures . . . . . . . . . . . . . . . . . . . . . . . . ix List of Tables . . . . . . . . . . . . . . . . . . . . . . . . xi Audience. . . . . . . . . . . . . . . . . . . . . . . . . xiii Important Information . . . . . . . . . . . . . . . . . . . . . . . xiii Chapter Descriptions . . . . . . . . . . . . . . . . . . . . . . xiv Related Publications . .
Contents System Control Overview . . . . . . . . . . . . . . . . . . . . . 12 Communicating with the System . . . . . . . . . . . . . . . . . . 13 The SGI Management Center Graphical User Interface . . . . . . . . . . . . 13 Powering-On and Off From the SGI Management Center Interface The Command Line Interface . . . . . . . . . 13 . . . . . . . . . . . 14 Powering On and Off from the Command Line Interface . . . . . . . . .
Contents System Features . . . . . . . . . . . . . . . . . . . . . . 39 Modularity and Scalability . . . . . . . . . . . . . . . . . . . . 39 Distributed Shared Memory (DSM) . . . . . . . . . . . . . . . . . 39 Distributed Shared I/O . 5. 6. . . . . . . . . . . . . . . . . . . 41 Chassis Management Controller (CMC) . . . . . . . . . . . . . . . . 41 ccNUMA Architecture . . . . . . . . . . . . . .
Contents LED Status Indicators . . . . . . . . . . . . . . . . . . . . . . 69 IRU Power Supply LEDs . . . . . . . . . . . . . . . . . . . . 69 Compute/Memory Blade LEDs . . . . . . . . . . . . . . . . . . . 70 . . . . . . . . . . . . . . . . . . 71 Technical Specifications and Pinouts . . . . . . . . . . . . . . . . . . 75 System-level Specifications . . . . . . . . . . . . . . . . . . . .
List of Figures 007-5832-001 Figure 1-1 IRU Power Supply Cable Location Example . . . . . . . . 3 Figure 1-2 Single-Phase 2-Outlet PDU Example . . . . . . . . . . 4 Figure 1-3 Single-Phase 8-Outlet PDU . . . . . . . . . . . . . 5 Figure 1-4 Three-Phase PDU Examples . . . . . . . . . . . . . 6 Figure 1-5 System Management Node Rear Video Connections . . . . . . 8 Figure 1-6 UV CMC Connection Faceplate Example . . . . . .
List of Figures x Figure 7-1 UV Compute Blade Status LED Locations Example . . . . . . . 70 Figure 7-2 Full Support Sequence Example . . . . . . . . . . . . 71 Figure A-1 VGA Port Pinouts . . . . . . . . . . . . . . . . 79 Figure A-2 Ethernet Port . . . . . . . . . . . . . . . . 81 Figure A-3 Serial Port Connector . . . . . . . . . . . . . . . 82 Figure A-4 Pin Number Locations for USB Type A Connector . . . . . . .
List of Tables 007-5832-001 Table 4-1 Tall Rack Technical Specifications . . . . . . . . . . . 52 Table 5-1 NUMAlink ORC Technical Specifications . . . . . . . . . 56 Table 6-1 SGI UV 2000 PCIe Support Levels . . . . . . . . . 60 Table 6-2 PCIe Expansion Slot Bandwidth Support Levels . . . . . . . 61 Table 7-1 Troubleshooting Chart . . . . . . . . . . . . . . 68 Table 7-2 Power Supply LED States . . . . . . . . . . . . .
About This Guide This guide provides an overview of the architecture, general operation and descriptions of the major components that compose the SGI UV 2000 family of servers. It also provides the standard procedures for powering on and powering off the system, basic troubleshooting and maintenance information, and important safety and regulatory specifications. Audience This guide is written for owners, system administrators, and users of SGI UV 2000 computer systems.
About This Guide Chapter Descriptions The following topics are covered in this guide: xiv • Chapter 1, “Operation Procedures,” provides instructions for powering on and powering off your system. • Chapter 2, “System Control,” describes the function of the overall system control network interface and provides basic instructions for operating the controllers.
Related Publications Related Publications The following SGI documents are relevant to the UV 2000 series system: • SGI UV CMC Software User Guide (P/N 007-5636-00x) This guide describes how to use the system console controller commands to monitor and manage your SGI UV 2000 system via line commands. Coverage of control includes descriptions of the interface and usage of the commands. These commands are primarily used when a system management node is not present in the system.
About This Guide • SGI UV Systems Linux Configuration and Operations Guide (P/N 007-5629-00x) This guide is a reference document for people who manage the operation of SGI UV 2000 systems. It explains how to perform general system configuration and operation under Linux for SGI UV. For a list of manuals supporting SGI Linux releases and SGI online resources, see the SGI Performance Suite documentation.
Conventions For additional information about displaying man pages using the man command, see man(1). In addition, the apropos command locates man pages based on keywords. For example, to display a list of man pages that describe disks, type the following on a command line: apropos disk For information about setting up and using apropos, see apropos(1).
About This Guide Product Support SGI provides a comprehensive product support and maintenance program for its products, as follows: • If you are in North America, contact the Technical Assistance Center at +1 800 800 4SGI or contact your authorized service provider. • If you are outside North America, contact the SGI subsidiary or authorized distributor in your country. International customers can visit http://www.sgi.
Chapter 1 1.
1: Operation Procedures Safety Precautions Warning: Before operating or servicing any part of this product, read the “Safety Information” on page 85. Danger: Keep fingers and conductive tools away from high-voltage areas. Failure to follow these precautions will result in serious injury or death. The high-voltage areas of the system are indicated with high-voltage warning labels. ! Caution: Power off the system only after the system software has been shut down in an orderly manner.
Power Connections Overview long as there is power coming into the unit. Turn off the PDU breaker switch on each of the PDUs that supply voltage to the IRU’s power supplies if you want to remove all power from the unit. Important: In a system configuration using 2-outlet single-phase PDUs, each power supply in an IRU should be connected to a different PDU within the rack. This will ensure the maximum amperage output of a single PDU is not exceeded if a power supply fails.
1: Operation Procedures Figure 1-2 Single-Phase 2-Outlet PDU Example Figure 1-3 on page 5 shows an example of an eight-plug single-phase PDU that can be used in the SGI UV 2000 rack system. This unit is used to support auxiliary equipment in the rack.
Power Connections Overview Power distribution unit (PDU) Power source Figure 1-3 007-5832-001 Single-Phase 8-Outlet PDU 5
1: Operation Procedures Figure 1-4 shows examples of the three-phase PDUs that can be used in the SGI UV 2000 system.
System Connections Overview System Connections Overview You can monitor and interact with your SGI UV 2000 server from the following sources: • Using the SGI 1U rackmount console option you can connect directly to the system management node (SMN) for basic monitoring and administration of the system. See “1U Console Option” in Chapter 2 for more information; SLES 11 or later is required.
1: Operation Procedures A micro-USB serial connection can be used to communicate directly with the CMC. This connection is typically used for service purposes or for system controller and system console access in small systems where an in-rack system console is not used or available.
System Connections Overview Serial Console Hardware Requirements The console type and how these console types are connected to the SGI UV 2000 servers is determined by what console option is chosen. If you have an SGI UV 2000 server and wish to use a serially-connected “dumb terminal”, you can connect the terminal via a micro-USB serial cable to the console port connector on the CMC.
1: Operation Procedures Note: Both options require the use of the CMC's serial port, refer to Figure 1-6 on page 8. For DHCP, you must determine the IP address that the CMC has been assigned; for a static IP, you must also configure the CMC to use the desired static IP address. To use the serial port connection, you must attach and properly configure a micro-USB cable to the CMC's "CONSOLE" port. Configure the serial port as described in “Serial Console Hardware Requirements” on page 9.
System Connections Overview BOOTPROTO=static IPADDR= NETMASK= GATEWAY= HOSTNAME= Note that the "GATEWAY" and "HOSTNAME" lines are optional. After modifying the file, save and write it using the vi command ":w!", and then exit vi using ":q". Then reboot the CMC (using the "reboot" command); after it reboots, it will be configured with the specified IP address.
1: Operation Procedures System Control Overview All SGI UV 2000 system individual rack units (IRUs) use an embedded chassis management controller (CMC). The CMC communicates with both the blade-level board management controllers (BMCs) and the system management node (SMN), which runs the SGI Management Center software. In concert with the SGI Management Center software, they are generically known as the system control network.
System Control Overview Communicating with the System The two primary ways to communicate with and administer the SGI UV 2000 system are through SGI Management Center interface or the UV command line interface (CLI). The SGI Management Center Graphical User Interface The SGI Management Center interface is a server monitoring and management system. The SGI Management Center GUI provides status metrics on operational aspects for each node in a system.
1: Operation Procedures 5. On the command line, enter mgrclient and press Enter. The SGI Management Center Login dialog box is displayed. 6. Enter a user name (root by default) and password (root by default) and click OK. The SGI Management Center interface is displayed. 7. The power on (green button) and power off (red button) are located in the middle of the SGI Management Center GUI’s Tool Bar - icons which provide quick access to common tasks and features.
System Control Overview about its internal blades, and also (if other IRUs are attached via NUMAlink to the IRU) information about those IRUs. Command Options for Power On The following example command options can be used with either the SMN or CMC CLI: usage: power [-vcow] on|up [system-SSN]...
1: Operation Procedures The system will take time to fully power up (depending on size and options).
System Control Overview Note: The command line interface for the CMC is virtually the same as that for the SMN, with the exception that the CMC does not have the ability to target a system when multiple systems are supported from one SMN. Booting Directly From a CMC If a system management node (SMN) is not available, it is possible to power on and administer your system directly from the CMC. When available, the optional SMN should always be the primary interface to the system.
1: Operation Procedures The system will take time to fully power up (depending on size and options). Larger systems take longer to fully power on. Information on booting Linux from the shell prompt is included at the end of the subsection (“Monitoring Power On” on page 19). Optional Power On Using the SMN to Connect to the CMC Typically, the default password for the CMC set out of the SGI factory is root.
System Control Overview Monitoring Power On Open a separate window on your PC or workstation and establish another connection to the SMN or CMC and use the uvcon command to open a system console and monitor the system boot process. Use the following steps: CMC:r1i1c> uvcon uvcon: attempting connection to localhost... uvcon: connection to SMN/CMC (localhost) established. uvcon: requesting baseio console access at r001i01b00...
1: Operation Procedures Power off a UV System To power down the UV system, use the power off command, as follows: CMC:r1i1c> power off ==== r001i01c (PRI) ==== You can also use the power status command, to check the power status of your system CMC:r1i1c> power status ==== r001i01c (PRI) ==== on: 0, off: 32, unknown: 0, disabled: 0 The following command options can be used with the power off|down command: usage: power [-vo] off|down [system-SSN]...
System Control Overview usage: power [-v10ud] [status] [system-SSN]...show power status -v, --verbose verbose output -1, --on show only blades with on status -0, --off show only blades with off status -u, --unknown show only blades with unknown status -d, --disabled show only blades with disabled status usage: power [-ov] nmi|debug [system-SSN]...issue NMI -o, --override override partition check -v, --verbose verbose output usage: power [-v] margin [high|low|norm|] [system-SSN]...
1: Operation Procedures Using Embedded Support Partner (ESP) Embedded Support Partner (ESP) automatically detects system conditions that indicate potential future problems and then notifies the appropriate personnel. This enables you and SGI system support engineers (SSEs) to proactively support systems and resolve issues before they develop into actual failures. ESP enables users to monitor one or more systems at a site from a local or remote connection.
Optional Components Optional Components Besides adding a network-connected system console or basic VGA monitor, you can add or replace the following hardware items on your SGI UV 2000 series server: • Peripheral component interface (PCIe) cards into the optional PCIe expansion chassis. • PCIe cards into the blade-mounted PCIe riser card. • Disk drives in your dual disk drive riser card equipped compute blade.
1: Operation Procedures 24 Figure 1-7 PCIe Option Blade Example with Full-Height and Low-Profile Slots Figure 1-8 PCIe Option Blade Example with Two Low-Profile Slots 007-5832-001
Chapter 2 2. System Control This chapter describes the general interaction and functions of the overall SGI UV 2000 system control.
2: System Control System Management Node (SMN) Overview An Ethernet connection directly from the SMN (Figure 2-1 on page 27) to a local private or public Ethernet allows the system to be administered directly from a local or remote console via the SGI Management Center interface installed on the SMN. Note that there is no direct inter-connected system controller function in the optional expansion PCIe modules. The system controller network is designed into all IRUs.
Levels of System Control System LEDs Slim DVD drive option System reset Main power Disk drive bays Power Supply Module BMC Port Mouse Keyboard USB Port 1 COM Port1 USB Port 0 Figure 2-1 Full-height (full-depth) x16 PCIe slot LAN ports 1-4 Full-height (half-depth) x16 PCIe slot VGA Port System Management Node Front and Rear Panels CMC Overview The CMC system for the SGI UV 2000 servers manages power control and sequencing, provides environmental control and monitoring, initiates system resets,
2: System Control blade installed. The CMC provides data collected from the compute nodes within the IRU to the system management node upon request. CMCs can communicate with the blade BMCs and other IRU CMCs when they are linked together under a single system image (SSI); also called a partition. Each CMC shares its information with the SMN as well as other CMCs within the SSI.
System Controller Interaction BMC Overview Each compute blade in an IRU has a baseboard management controller (BMC). The BMC is a built-in specialized microcontroller hardware component that monitors and reports on the functional “health” status of the blade. The BMC provides a key functional element in the overall Intelligent Platform Management Interface (IPMI) architecture.
2: System Control IRU Controllers All IRUs have a chassis management controller (CMC) board installed. The following subsection describes the basic features of the controllers: Note: For additional information on controller commands, see the SGI UV CMC Software User Guide (P/N 007-5636-00x). Chassis Management Controller Functions The following list summarizes the control and monitoring functions that the CMC performs.
IRU Controllers 1 2 3 4 Figure 2-3 Optional 1U Rackmount Console Flat Panel Rackmount Console Option Features The 1U flat panel console option has the following listed features: 1. Slide Release - Move this tab sideways to slide the console out. It locks the drawer closed when the console is not in use and prevents it from accidentally sliding open. 2. Handle - Used to push and pull the module in and out of the rack. 3.
2: System Control 4. Power LED - Illuminates blue when the unit is receiving power. The 1U console attaches to the system management node server using PS/2 and HD15M connectors or to an optional KVM switch (not provided by SGI). See Figure 2-4 for the SMN video connection points. The 1U console is basically a “dumb” VGA terminal, it cannot be used as a workstation or loaded with any system administration program. The 27-pound (12.27-kg) console automatically goes into sleep mode when the cover is closed.
Chapter 3 3. System Overview This chapter provides an overview of the physical and architectural aspects of your SGI UV 2000 series system. The major components of the SGI UV 2000 series systems are described and illustrated. The SGI UV 2000 series is a family of multiprocessor distributed shared memory (DSM) computer systems that can scale from 16 to 2,048 Intel processor cores as a cache-coherent single system image (SSI).
3: System Overview SG IU V2 00 Figure 3-1 34 0 SGI UV 2000 Single-Rack System Example 007-5832-001
System Models System Models The basic enclosure within the SGI UV 2000 system is the 10U high “individual rack unit” (IRU). The IRU enclosure contains up to eight compute blades connected to each other via a backplane. Each IRU has ports that are brought out to external NUMAlink 6 connectors. The 42U rack for this server houses all IRU enclosures, option modules, and other components; up to 64 processor sockets (512 processor cores) in a single rack.
3: System Overview UV Rack IRU IRU (System Management Node) 1U (I/O) Expansion Slot IRU Individual Rack Unit (IRU) IRU Figure 3-2 36 SGI UV 2000 IRU and Rack 007-5832-001
System Architecture System Architecture The SGI UV 2000 computer system is based on a distributed shared memory (DSM) architecture. The system uses a global-address-space, cache-coherent multiprocessor that scales up to 512 processor cores in a single rack. Because it is modular, the DSM combines the advantages of lower entry cost with the ability to scale processor count, memory, and I/O independently in each rack. Note that a maximum of 2,048 cores are supported on a single-system image (SSI).
3: System Overview IRU Backplane Connections -2 NI3 1-2 NI NI1-0 NI1-0 1 2 0NI -2 2 I N NI3-0 2 2NI -2 0 NI NI1-3 NI1-1 NI1-3 NI0 NI2 -3 -3 NI1-0 -2 NI3 -2 NI1 NI1-0 6 NI3-0 NI3-0 NI1-1 NI0 NI2 -3 -3 -2 NI2 -2 NI0 NI1-3 2 4 -2 NI3 -2 NI1 NI1-1 NI3-1 NI3-3 NI0 -3 NI2 -3 NI3-1 NI3-3 NI1-0 NI3-0 2 2NI -2 0 I N 7 NI3-3 NI3-1 NI1-1 NI1-3 NI3-1 NI3-3 NI1-3 NI1-1 NI1-0 NI3-0 -2 NI2 -2 NI0 NI1-2 NI3-2 NI0 NI2 -3 -3 NI NI2 0-3 -3 NI1-0 NI3-0 3 NI3-3 NI3-1 NI1-1 NI1-3 NI3-1 NI3-3
System Features System Features The main features of the SGI UV 2000 series server systems are discussed in the following sections: • “Modularity and Scalability” on page 39 • “Distributed Shared Memory (DSM)” on page 39 • “Chassis Management Controller (CMC)” on page 41 • “Distributed Shared I/O” on page 41 • “Reliability, Availability, and Serviceability (RAS)” on page 42 Modularity and Scalability The SGI UV 2000 series systems are modular systems.
3: System Overview BACKPLANE CONNECTOR DDR3 A B DDR3 A B DDR3 A B DDR3 Chan-0 Chan-1 Chan-2 A B DDR3 VPPL Chan-3 VTT PCIe Gen 3 X16 TOP NODE PCA (4) QFSP iPass QPI NL6 4 Channels PSOC SGPIO Local Bus 0 &1 QPI DDR3 Vcore/ VSA P R O C QPI 1 C O N N E C SGPIO T PSOC O R NL6 12 Channels CPU/DDR3 Power BNI Bus 0 & 1 HARP PCA HARP CONNECTOR Clocks QPI Spartan 6 FPGA PSOC SGPIO Only used for BaseIO RSPI Parallel Data/ Addr Bus Clocks PCIe X4 PCIe X4 DDR3 Only used for BaseIO SPI B
System Features Distributed Shared I/O Like DSM, I/O devices are distributed among the blade nodes within the IRUs. Each BaseIO riser card equipped blade node is accessible by all compute nodes within the SSI (partition) through the NUMAlink interconnect fabric. Chassis Management Controller (CMC) Each IRU has a chassis management controller (CMC) located directly below the upper set of cooling fans in the rear of the IRU.
3: System Overview When a processor modifies a block of data, the processors that have the same block of data in their caches must be notified of the modification. The SGI UV 2000 server series uses an invalidation method to maintain cache coherence. The invalidation method purges all unmodified copies of the block of data, and the processor that wants to modify the block receives exclusive ownership of the block.
System Features • • 007-5832-001 – Each blade/node installed has status LEDs that indicate the blade’s operational condition; LEDs are readable at the front of the IRU. – Systems support the optional Embedded Support Partner (ESP), a tool that monitors the system; when a condition occurs that may cause a failure, ESP notifies the appropriate SGI personnel. – Systems support remote console and maintenance activities. Power-on and boot: – Automatic testing occurs after you power on the system.
3: System Overview System Components The SGI UV 2000 series system features the following major components: • 42U rack. This is a custom rack used for both the compute and I/O rack in the SGI UV 2000 system. Up to four IRUs can be installed in each rack. There is also space reserved for a system management node and other optional 19-inch rackmounted components. • Individual Rack Unit (IRU).
System Components Compute blade 3 Compute blade 7 Compute blade 2 Compute blade 6 Compute blade 1 Compute blade 5 Compute blade 0 Compute blade 4 PS0 Figure 3-5 007-5832-001 PS1 PS2 SGI UV 2000 IRU System Components Example 45
SAS4-7 PCIE 3: System Overview VGA LAN1 SERIAL LAN0 2 SAS0-3 1 0 USB Figure 3-6 BMC BaseIO Riser Enabled Blade Front Panel Example Bay (Unit) Numbering Bays in the racks are numbered using standard units. A standard unit (SU) or unit (U) is equal to 1.75 inches (4.445 cm). Because IRUs occupy multiple standard units, IRU locations within a rack are identified by the bottom unit (U) in which the IRU resides. For example, in a 42U rack, an IRU positioned in U01 through U10 is identified as U01.
Chapter 4 4. Rack Information This chapter describes the physical characteristics of the tall (42U) SGI UV 2000 racks in the following sections: • “Overview” on page 47 • “SGI UV 2000 Series Rack (42U)” on page 48 • “SGI UV 2000 System Rack Technical Specifications” on page 52 Overview At the time this document was published only the tall (42U) SGI UV 2000 rack (shown in Figure 4-2) was available from the SGI factory for use with the SGI UV 2000 systems.
4: Rack Information SGI UV 2000 Series Rack (42U) The tall rack (shown in Figure 4-1 on page 49) has the following features and components: • Front and rear door. The front door is opened by grasping the outer end of the rectangular-shaped door piece and pulling outward. It uses a key lock for security purposes that should open all the front doors in a multi-rack system (see Figure 4-2 on page 50). Note: The front door and rear door locks are keyed differently.
SGI UV 2000 Series Rack (42U) Each three-phase power distribution unit has 9 outlet connections.
4: Rack Information SG IU V2 00 Figure 4-2 50 0 Front Lock on Tall (42U) Rack 007-5832-001
SGI UV 2000 Series Rack (42U) 00 V IU 20 SG Figure 4-3 007-5832-001 Optional Water-Chilled Cooling Units on Rear of SGI 42U Rack 51
4: Rack Information SGI UV 2000 System Rack Technical Specifications Table 4-1 lists the technical specifications of the SGI UV 2000 series tall rack. Table 4-1 Tall Rack Technical Specifications Characteristic Specification Height 79.5 in. (201.9 cm) Width 31.3 in. (79.5 cm) Depth 45.8 in. (116.3 cm) Single-rack shipping weight (approximate) 2,381 lbs. (1,082 kg) air cooled 2,581 (1,173 kg) water assist cooling Single-rack system weight 2,300 lbs.
Chapter 5 5. Optional Octal Router Chassis Information This chapter describes the optional NUMAlink router technology available in SGI UV 2000 systems consisting of two or more racks. This router technology is available in an enclosure “package” known as the Octal Router Chassis (ORC). This optional ORC chassis can be mounted on the top of the SGI UV 2000 rack. NUMAlink 6 advanced router technology reduces UV 2000 system data transfer latency and increases bisection bandwidth performance.
5: Optional Octal Router Chassis Information SGI UV 2000 Series NUMAlink Octal Router Chassis The NUMAlink 6 ORC router is a 7U-high fully self contained chassis that holds up to eight 16-port NL6 router blade assemblies. Figure 5-1 shows an example rear view of the ORC with no power or NUMAlink cables connected.
SGI UV 2000 Series NUMAlink Octal Router Chassis 7 6 5 4 3 2 1 0 Figure 5-2 SGI UV 2000 Optional ORC Chassis Example (Front View) Note: The NUMAlink unit’s CMC is connected to the CMC in each IRU installed in the rack.
5: Optional Octal Router Chassis Information SGI UV 2000 External NUMAlink System Technical Specifications Table 5-1 lists the basic technical specifications of the SGI UV 2000 series external NUMAlink ORC chassis. Table 5-1 56 NUMAlink ORC Technical Specifications Characteristic Specification Height 7U or 12.25 in. (31.1 cm) Width 13.83 in. (35.13 cm) Depth 14.66 in (37.24 cm) Top-mount NUMAlink router weight (approximate) 53 lbs. (24.
Chapter 6 6.
6: Add or Replace Procedures Preparing the System for Maintenance or Upgrade To prepare the system for maintenance, follow these steps: 1. If you are logged on to the system, log out. Follow standard procedures for gracefully halting the operating system. 2. Go to the section “Powering-On and Off From the SGI Management Center Interface” in Chapter 1 if you are not familiar with power down procedures. 3.
Overview of PCI Express (PCIe) Operation Overview of PCI Express (PCIe) Operation This section provides a brief overview of the PCI Express (PCIe) technology available as an option with your system. PCI Express has both compatibility and differences with older PCI/PCI-X technology. Check with your SGI sales or service representative for more detail on specific PCI Express board options available with the SGI UV 2000.
6: Add or Replace Procedures SGI UV 2000 PCIe technology is available up to a x16 connector (64 wires) or “by 16” in PCI Express card slots. This technology will support PCIe boards that use connectors up to x16 in size. Table 6-1 shows this concept. For information on which slots in the PCIe expansion chassis support what lane levels, see Table 6-2 on page 61.
Adding or Replacing PCIe Cards in the Expansion Enclosure Installing Cards in the 1U PCIe Expansion Chassis The PCIe expansion chassis functions in a similar manner to a computer chassis that supports PCIe slots. Always follow the manufacturer’s instructions or restrictions for installing their card. Important: Replacement (swapping) of a PCIe card in the 1U chassis may be done while the system is powered on.
6: Add or Replace Procedures 6. Fit the PCIe card into the slot connector with the connector(s) extending out the front of the bracket, then secure the board with the screw that previously held the metal slot cover. 7. Push the PCIe board “carriage” back into the enclosure until it is seated and twist the retaining thumb screws clockwise (right) until fully secure. Important: After installation, be sure to power on the PCIe expansion enclosure before re-booting your system.
Adding or Replacing PCIe Cards in the Expansion Enclosure Figure 6-3 007-5832-001 Card Slot Locations 63
6: Add or Replace Procedures Removing and Replacing an IRU Enclosure Power Supply To remove and replace power supplies in an SGI UV 2000 IRU, you do not need any tools. Under most circumstances a single power supply in an IRU can be replaced without shutting down the enclosure or the complete system. In the case of a fully configured (loaded) enclosure, this may not be possible. Caution: The body of the power supply may be hot; allow time for cooling and handle with care.
Adding or Replacing PCIe Cards in the Expansion Enclosure 3. Press the retention latch of the power supply toward the power connector to release the supply from the enclosure, see Figure 6-4 on page 64. 4. Using the power supply handle, pull the power supply straight out until it is partly out of the chassis. Use one hand to support the bottom of the supply as you fully extract it from the enclosure. 5. Align the rear of the replacement power supply with the enclosure opening. 6.
Chapter 7 7.
7: Troubleshooting and Diagnostics Troubleshooting Chart Table 7-1 lists recommended actions for problems that can occur. To solve problems that are not listed in this table, use the SGI Electronic Support system or contact your SGI system support representative. For more information about the SGI Electronic Support system, see the “SGI Electronic Support” on page 71. For an international list of SGI support centers, see: http://www.sgi.com/support/supportcenters.
LED Status Indicators LED Status Indicators There are a number of LEDs on the front of the IRUs that can help you detect, identify and potentially correct functional interruptions in the system. The following subsections describe these LEDs and ways to use them to understand potential problem areas. IRU Power Supply LEDs Each power supply installed in an IRU has a bi-color status LED.
7: Troubleshooting and Diagnostics Compute/Memory Blade LEDs Each compute/memory blade installed in an IRU has a total of seven LED indicators visable behind the perforated sheetmetal of the blade. At the bottom end (or left side) of the blade (from left to right): • System power good green LED • BMC heartbeat green LED • Blue unit identifier (UID) LED • BMC Ethernet 1 green LED • BMC Ethernet 0 green LED • Green 3.
SGI Electronic Support SGI Electronic Support SGI Electronic Support provides system support and problem-solving services that function automatically, which helps resolve problems before they can affect system availability or develop into actual failures. SGI Electronic Support integrates several services so they work together to monitor your system, notify you if a problem exists, and search for solutions to problems.
7: Troubleshooting and Diagnostics The sequence of events can be described as follows: 1. Embedded Support Partner (ESP) software monitors your system 24 hours a day. 2. When a specified system event is detected, ESP notifies SGI via e-mail (plain text or encrypted). 3. Applications that are running at SGI analyze the information, determine whether a support case should be opened, and open a case if necessary.
SGI Electronic Support The following three components compose the integrated SGI Electronic Support system: SGI Embedded Support Partner (ESP) is a set of tools and utilities that are embedded in the SGI Linux ProPack release. ESP can monitor a single system or group of systems for system events, software and hardware failures, availability, performance, and configuration changes, and then perform actions based on those events.
Appendix A A. Technical Specifications and Pinouts This appendix contains technical specification information about your system, as follows: • “System-level Specifications” on page 75 • “Physical Specifications” on page 76 • “Environmental Specifications” on page 77 • “Power Specifications” on page 78 • “I/O Port Specifications” on page 79 System-level Specifications Table A-1 summarizes the SGI UV 2000 system configuration ranges.
A: Technical Specifications and Pinouts Physical Specifications Table A-2 shows the physical specifications of the SGI UV 2000 system. Table A-2 SGI UV 2000 Physical Specifications Feature Specification Dimensions for a single 24-inch Height: 79.5 in. (201.9 cm) wide tall rack, including doors and Width: 31.3 in. (79.5 cm) side panels Depth: 43.45 in. (110. 4 cm) Shipping dimensions Height: 81.25 in. (206.4 cm) Width: 42 in. (106.7 cm) Depth: 52 in. (132.
Environmental Specifications Environmental Specifications Table A-3 lists the environmental specifications of the system. Table A-3 Environmental Specifications Feature Specification Temperature tolerance (operating) +5 C (41 F) to +35 C (95 F) (up to 1500 m / 5000 ft.) +5 C (41 F) to +30 C (86 F) (1500 m to 3000 m /5000 ft. to 10,000 ft.
A: Technical Specifications and Pinouts Power Specifications Table A-4 shows the power specifications for the system. Table A-4 Power Specifications Feature Specification Single-phase power requirements Voltage 200-240V (180-254 VAC min/max) Frequency 50-60 Hz Power 33.88 kW per rack Hold-up time 16 ms Three-phase power requirements Voltage US/Japan 208V (180-254 VAC) International 400V (312-440 VAC) 78 Frequency 50-60 Hz Power 33.
I/O Port Specifications I/O Port Specifications This section contains specifications and port pinout information for the base I/O ports of your system, as follows: • “BaseIO VGA Port Information” on page 79 • “Ethernet Port” on page 81 • “Serial Ports” on page 82 • “USB Type A Connector” on page 84 BaseIO VGA Port Information The 15-pin VGA port on the baseIO blade (see Figure A-1) has the following features: • Server-class 2D hardware acceleration support with integrated 24-bit RAMDAC • Display
A: Technical Specifications and Pinouts Table A-5 80 VGA Pin Functions Pin Number Function 1 Red 2 Green 3 Blue 4 N/C 5 Ground 6 Ground 7 Ground 8 Ground 9 Ground 10 Ground 11 N/C 12 DDCDAT 13 HSYNC 14 VSYNC 15 DDCCLK 007-5832-001
I/O Port Specifications Ethernet Port The system auto-selects the Ethernet port speed and type (duplex vs. half-duplex) when the server is booted, based on what it is connected to. Figure A-2 shows the Ethernet port. Pin 4 Pin 5 Pin 6 Pin 3 Pin 7 Pin 2 Pin 1 Pin 8 Figure A-2 Ethernet Port Table A-6 shows the cable pinout assignments for the Ethernet port operating in 10/100-Base-T mode and also operating in 1000Base-T mode.
A: Technical Specifications and Pinouts Serial Ports The IRU and the optional router modules have 9-pin serial interface connectors. These ports are for console interface and are capable of transferring data at rates as high as 230 kbps. Other features of the ports include the following: • Programmable data, parity, and stop bits • Programmable baud rate and modem control Figure A-3 shows a serial port.
I/O Port Specifications Table A-7 shows pinout assignments for the 9-pin male DB-9 connector.
A: Technical Specifications and Pinouts USB Type A Connector Figure A-4 shows the USB type A connector provided on the baseIO riser blade that supports general USB applications and optional keyboard and mouse configurations. Table A-8 lists the pin assignments for the USB type A connector.
Appendix B B. Safety Information and Regulatory Specifications This appendix provides safety information and regulatory specifications for your system in the following sections: • “Safety Information” on page 85 • “Regulatory Specifications” on page 87 Safety Information Read and follow these instructions carefully: 1. Follow all warnings and instructions marked on the product and noted in the documentation included with this product. 2. Unplug this product before cleaning.
B: Safety Information and Regulatory Specifications 9. Do not attempt to service this product yourself except as noted in this guide. Opening or removing covers of node and switch internal components may expose you to dangerous voltage points or other risks. Refer all servicing to qualified service personnel. 10. Unplug this product from the wall outlet and refer servicing to qualified service personnel under the following conditions: • When the power cord or plug is damaged or frayed.
Regulatory Specifications Regulatory Specifications The following topics are covered in this section: • “CMN Number” on page 87 • “CE Notice and Manufacturer’s Declaration of Conformity” on page 87 • “Electromagnetic Emissions” on page 88 • “Shielded Cables” on page 90 • “Electrostatic Discharge” on page 90 • “Laser Compliance Statements” on page 91 • “Lithium Battery Statement” on page 92 This SGI system conforms to several national and international specifications and European Directives li
B: Safety Information and Regulatory Specifications Electromagnetic Emissions This section provides the contents of electromagnetic emissions notices from various countries. FCC Notice (USA Only) This equipment complies with Part 15 of the FCC Rules. Operation is subject to the following two conditions: • This device may not cause harmful interference. • This device must accept any interference received, including interference that may cause undesired operation.
Regulatory Specifications Industry Canada Notice (Canada Only) This Class A digital apparatus meets all requirements of the Canadian Interference-Causing Equipment Regulations. Cet appareil numérique német pas de perturbations radioélectriques dépassant les normes applicables aux appareils numériques de Classe A préscrites dans le Règlement sur les interferences radioélectriques établi par le Ministère des Communications du Canada.
B: Safety Information and Regulatory Specifications Shielded Cables This SGI system is FCC-compliant under test conditions that include the use of shielded cables between the system and its peripherals. Your system and any peripherals you purchase from SGI have shielded cables. Shielded cables reduce the possibility of interference with radio, television, and other devices. If you use any cables that are not from SGI, ensure that they are shielded. Telephone cables do not need to be shielded.
Regulatory Specifications Laser Compliance Statements The DVD-ROM drive in this computer is a Class 1 laser product. The DVD-ROM drive’s classification label is located on the drive. Warning: Avoid exposure to the invisible laser radiation beam when the device is open. Warning: Attention: Radiation du faisceau laser invisible en cas d’ouverture. Evitter toute exposition aux rayons. Warning: Vorsicht: Unsichtbare Laserstrahlung, Wenn Abdeckung geöffnet, nicht dem Strahl aussetzen.
B: Safety Information and Regulatory Specifications Lithium Battery Statement Warning: There is danger of explosion if a lithium battery is incorrectly replaced in this product. Replace a battery only with the same or equivalent type recommended by the manufacturer. Dispose of used batteries according to the manufacturer’s instructions.
