Reference COMPUTE PROCESSING MODULE ATCA-4616 ATCA-4618 ATCA-4648 March 2012 007-03446-0000
Revision history Version -0000 Date March 2012 Description First edition. © 2012 by RadiSys Corporation. All rights reserved. Radisys is a registered trademark of RadiSys Corporation. AdvancedTCA, ATCA, and PICMG are registered trademarks of PCI Industrial Computer Manufacturers Group. All other trademarks, registered trademarks, service marks, and trade names are the property of their respective owners.
Table of Contents Preface ................................................................................................................................................ 7 About this manual........................................................................................................................................7 Where to get more product information .......................................................................................................7 About related Radisys products...........
Table of Contents Intel C600 series Platform Controller Hub (PCH) .................................................................................29 IPMI controller.......................................................................................................................................33 CPU Complex (CC) FPGA....................................................................................................................34 IPMI FPGA............................................................
Table of Contents Chapter 5: Troubleshooting and Repair ......................................................................................... 69 Introduction................................................................................................................................................69 Field Replaceable Units (FRUs) ................................................................................................................69 FRU information areas used................................
Table of Contents Appendix B: IPMI Commands and Managed Sensors .................................................................. 86 IPMI command interfaces..........................................................................................................................86 IPMI commands.........................................................................................................................................86 OEM command descriptions.....................................................
Preface About this manual This manual describes the ATCA‐46xx, a compute processing module (CPM), which is fully compliant with AdvancedTCA® (ATCA®). The CPM is designed to be incorporated into High Availability (HA) systems such as the Radisys platforms SYS‐6006 and SYS‐6010. Use this manual as a hardware reference for the operation and maintenance of the ATCA‐46xx CPM. The manual also provides information on the electrical, the mechanical, and the environmental aspects of the ATCA‐46xx CPM.
Preface • • • Software reference information. The Software Guide for Management Processors and General Computing Processors describes software concepts and serves as a reference for procedural and usage information. When referenced in this manual, the simplified name of Software Guide will be used. Shelf Manager information. The Shelf Management Software Reference describes the architecture and the operation of the Shelf Manager.
Preface IEEE Std 802.3‐2002 Telecommunications and information exchange between systems — Local and metropolitan area networks — Specific requirements, Part 3: Carrier sense multiple access with collision detection (CSMA/CD) access method and physical layer specifications, IEEE Computer Society, March 8, 2002. Intelligent Platform Management Interface Specification v1.5, Revision 2.0, Intel Corporation; Hewlett Packard Company, NEC Corporation, and Dell Computer Corporation. Linux PAM Modules web site.
Preface Electrostatic discharge WARNING! This product contains static‐sensitive components and should be handled with care. Failure to employ adequate anti‐static measures can cause irreparable damage to components. Electrostatic discharge (ESD) damage can result in partial or complete device failure, performance degradation, or reduced operating life. To avoid ESD damage, the following precautions are strongly recommended. • Keep each carrier in its ESD shielding bag until you are ready to install it.
Chapter 1 Product Overview Introduction The ATCA‐46xx Compute Processing Module (CPM) is a high‐end, general‐purpose computing module that provides multi‐core processing power and multiple data‐storage options within a single Advanced Telecommunications Computing Architecture (ATCA) slot. The remaining sections in this chapter present the major features, specifications compliance, and product options that apply to the ATCA‐46xx CPM.
1 Product Overview Supported external interfaces The ATCA‐46xx CPM supports both internal and external interfaces. Internal interfaces include the buses and communication protocols that are fully contained within the CPM blade or are included within the front panel, backplane, or RTM external interfaces.
1 Product Overview Specification compliance The ATCA‐46xx CPM complies with the following specifications: • PCI Industrial Computers Manufacturers Group (PICMG) 3.0 R2.0 ECN0002 Advanced Telecommunications Computing Architecture (ATCA) specification • PICMG 3.1 R2.
1 Product Overview Product options The ATCA‐46xx CPM has the following product options: • A4616‐CPU‐Base ‐ The basic low‐power CPM with 10G Fabric interface, dual Intel Xeon six core processors, and no installed memory. • A4618‐CPU‐Base ‐ The basic high‐power CPM with 10G Fabric interface, dual Intel Xeon eight core processors, and no installed memory. • A4648‐CPU‐Base ‐ The basic high‐power CPM with 40G Fabric interface, dual Intel Xeon eight core processors and no installed memory.
Chapter 2 Hardware Description Introduction The ATCA‐46xx Compute Processing Module (CPM) uses a number of hardware components to implement the functions required of an ATCA node. The major hardware components in the ATCA‐46xx CPM are as follow: • Dual E5‐2400 family processors mounted in FCLGA1356 sockets • Intel C600 series Platform Controller Hub (PCH) chip that supports the major I/O functionality on the CPM.
2 Hardware Description Functional block diagram Figure 1 is a functional block diagram that indicates major hardware components of the CPM. Figure 1. ATCA-46xx Functional Block Diagram DDR3 VLP RDIMMs DDR3 VLP RDIMMs CPU0 CPU1 CH. A Xeon® E5-2400 Family CPU PCIe Gen 3 Xeon® E5-2400 Family CPU QPI Mini Display Port CH. B CH. A PCIe Gen 3 DMI2 x16 (Gen 3) MXM 3.0 Digital Video DDR3 CH. B CH. C DDR3 CH.
2 Hardware Description ATCA-46xx front panel components The following sections use text, figures, and lists to identify the physical features of the CPM. Figure 2 shows the CPM front panel and calls out the major features. Figure 2.
2 Hardware Description Front panel connectors There are connectors for four separate interfaces on the front panel. Plugging into the connectors is straightforward, but the underlying interfaces need some explanation. The serial (COM) RJ45 connector is an RS232 serial interface with the pinout listed in Table 39 on page 108. The USB0 and USB1 connectors support USB 2.0 transactions. A USB cable connected to a single device up to 5 meters away can be used with the front panel USB connectors.
2 Hardware Description Front panel LEDs The front panel LEDs can be separated into two major groups; the edge LEDs along the left (bottom) edge of the front panel and the Base/Fabric channel status LEDs. Table 2 describes the status and activity LEDs along the left (bottom) edge of the front panel. Table 2.
2 Hardware Description Table 3 lists the meanings assigned to the light color/activity of each Base/Fabric LED. Table 3.
2 Hardware Description Rear panel connectors The CPM supports E‐Key control by describing its backplane interfaces to the Shelf Manager. Alignment keys The CPM implements the K1 and K2 alignment blocks at the top of Zone 2 and Zone 3, as required by the PICMG 3.0 specification. The Zone 2 alignment block (K1) is assigned a keying value of 11. The Zone 3 alignment block (K2) is set to allow insertion of ATCA‐46xx‐compatible RTMs.
2 Hardware Description Zone 3 connectors The CPM includes the standard Zone 3 backplane interface to provide connectivity to an optional RTM, such as the ATCA‐5400. This interface consists of two connectors: J30 for common and maintenance signals, and J31 for SerDes (serialization/deserialization connectivity). For details, refer to Table 47 on page 113 and Table 48 on page 113. The electrical connections between the CPM and the associated RTM include: • Switched +3.
2 Hardware Description ATCA-46xx board components During normal operation the CPM board components are covered by the CPM cooling shroud sheet metal (side panel). This sheet metal must be removed to see and gain access to removable board components, jumper blocks, or headers. Figure 4 shows the CPM board layout and calls out the major components and other features. Figure 4.
2 Hardware Description Heatsinks The CPU heatsinks are called out in Figure 4. In addition to the CPU heatsinks, the Intel C600 series PCH, and the Mellanox CX3 10/40 GbE controller each have individual heatsinks. The CPM incorporates a large heatsink covering each processor plus the additional heatsinks for other onboard high power devices to support a maximum CPM subsystem power dissipation of up to 50W.
2 Hardware Description Intel® Xeon® E5-2400 family processor The CPM uses two Intel® Xeon E5‐2400 family multi‐core, 64‐bit processors built using a 32‐ nm process. The E5‐2400 processor includes a 3‐channel memory controller, QuickPath interconnect, and integrated I/O for PCI Express support. The processor cores share an up to 20MB cache and include support for the Execute Disable Bit, Speed‐Step, Virtualization, Streaming SIMD Extensions, Hyper‐Threading, Turbo Boost, and TXT.
2 Hardware Description Integrated Memory Controller (IMC) The integrated memory controller incorporated into each E5‐2400 family processor supports three channels of DDR3, each channel with 64 data bits and 8 ECC bits. The CPM supports up to six registered VLP RDIMMs per socket (two per channel) for a system total of twelve RDIMMs. Table 6 lists the supported RDIMM memory types of the CPM and provides the required specifications for the modules. Table 6.
2 Hardware Description QuickPath Interconnect (QPI) The Quickpath interconnect (QPI) provides a point‐to‐point contact between the E5‐2400 family processors. The QPI interface is 20 lanes wide under full operation and is the communication path between the CPUs. Data of any width is converted to packets and then sent serially over the QPI link. The E5‐2400 supports QPI speeds of 6.4 GT/s to 8.0 GT/s depending on the installed processor.
2 Hardware Description DIMM memory This memory is directly addressed by the internal memory controller of each E5‐2400 family processor. Due to the board height limitations posed by the ATCA PICMG specifications, only Very Low Profile (VLP) Dual In‐line Memory Module (DIMM) modules are supported with the CPM. The Registered DIMM (RDIMM) memory used on the CPM is buffered by integral registers and has built‐in Error Correcting Code (ECC) bits to support more reliable operation.
2 Hardware Description Intel C600 series Platform Controller Hub (PCH) The Intel C600 series Platform Controller Hub (PCH) provides a connection point between various I/O components and the E5‐2400 family Xeon processors used on the CPM.
2 Hardware Description Serial ATA (SATA) The PCH includes two SATA host controllers that provide six SATA 3.0 ports. These ports support data transfer rates of 3.0 Gb/s. The CPM connects two SATA ports (0 and 1) for use with the optional 1.8 inch micro SATA SSD drives that can be installed in an optional Radisys DSSD MXM module. Low Pin Count (LPC) Bridge The Low Pin Count (LPC) bridge of the PCH provides read/write cycles for memory, I/O, DMA, and Buss Master devices.
2 Hardware Description SMBus/I2C bus The PCH provides an SMBus host controller (SMBus 2.0 compliant) as well as an SMBus secondary interface. The host controller provides a mechanism for the CPU to initiate communications with SMBus peripherals (master/slave interface). The PCH also can operate in a mode that supports communication with I2C compatible devices. The slave interface allows an external master to read from or write to the PCH.
2 Hardware Description Figure 5 shows the SMBus/I2C bus mapping and lists important device addresses on the CPM. Figure 5.
2 Hardware Description Management Engine (ME) The CPM Management Engine (ME) uses Node Manager 2.0 firmware to perform power monitoring & alert, power limiting policies, thermal monitoring & alert, and power reduction tasks during boot. As described in Serial Peripheral Interface (SPI) on page 30 a redundant firmware image is stored in the ME Flash in case the ME firmware needs to be recovered.
2 Hardware Description CPU Complex (CC) FPGA The CC FPGA module performs a number of significant monitoring and interface functions on the CPM. Many of these functions are indicated in Figure 1 on page 16. The following sections provide more details on the CC FPGA functions. Power management and monitoring Most of the CPM payload power supplies are monitored by two ADM1066 power sequencers, but the CC FPGA controls the power sequencing for the CPM.
2 Hardware Description CC FPGA RTM-link The RTM‐Link relays the state of various control and status signals from a compatible programmable device on an RTM.
2 Hardware Description IPMI FPGA The Intelligent Platform Management Interface (IPMI) Field Programmable Gate Array (FPGA) provides the specialized interfaces and glue logic needed between the H8 IPMC and rest of the CPM. The IPMC and the IPMI FPGA provide watchdog timers to help prevent the CPM from entering an unrecoverable state.
2 Hardware Description The CX3 has iSCSI and PXE boot support enabled in the SPI flash firmware. iSCSI and PXE cannot be enabled at the same time; the desired function must be selected in the BIOS setup menu. Clock synthesizer subsystem CPM uses two major components, a CK420BQ and a DB1900Z to generate the clock signals used by the CPUs, by the PCH, and by other peripheral components. The CK420BQ clock generator provides host, chipset, PCI and LPC peripheral clocks on the CPM. The device uses a 25.
2 Hardware Description The platform reset signal originates in the PCH. The potential sources or triggers of a platform reset include: • The front panel Reset button • A reset assertion generated over the board debug header during a troubleshooting session • An IPMC reset command generated due to one of the following events: • Watchdog timer expiration • An MR reset is asserted (by either the PCH or the IPMC) but they are disabled in the CC FPGA. In this case it reverts to a platform reset.
2 Hardware Description Table 9 lists the potential sources/triggers for the MR‐Reset. Though similar, these sources are slightly different from those for a platform or powergood reset. Table 9. Memory-Retained Reset Sources Platform Reset Source Front panel Reset button (FP_RESET*) RTM reset push button Description Pushing the button triggers an RCIN* assertion to the PCH, thus beginning the INIT* sequence. The RTM MMC sends OEM command to IPMC.
2 Hardware Description IPMC watchdog timer (Watchdog 2) The IPMC FPGA includes a hardware watchdog timer, Watchdog 2. This watchdog is enabled by default and will start running as soon as +3_3V_SUS power is present and the FPGA has loaded its internal flash image into its internal SRAM. The default timeout on power‐up is 10 seconds. After the IPMC boot‐loader is finished, the IPMC reprograms the timeout for 6 seconds and will continue to strobe every 2 seconds.
2 Hardware Description Figure 6. ATCA-46xx Power Subsystem ATCA Zone 1 (P10) PIM -48V A 10A Fuse -48V A -48V B 10A Fuse -48V B VRTN A 10A Fuse VRTN A VRTN B 10A Fuse VRTN B EARLY A 0.1A Fuse EARLY A EARLY B 0.1A Fuse EARLY B -48V to 12V Brick -48V -48V TPS54620 12V TPS54620 TPS7A8001 +5V_SUS TPS7A8001 TPS7A8001 TPS7A8001 IPMC FPGA CC FPGA TPS54620 2A +3_3V_SUS TPS54620 TPS54620 ISL6341A 400mA +1_2V_SUS +5V 4.
2 Hardware Description Power Input Module (PIM) A power input module (PIM) between the input fuses and main power brick (‐48V to 12V supply) provides input power conditioning and the following additional features: • Current handling up to 300W • Inrush current limit protection • Integrated filter designed to meet CISPR class B EMI limits • 11.88W of isolated auxiliary 3.3V power for IPMI circuitry. • 750mW of isolated auxiliary 5V power for IPMI circuitry.
2 Hardware Description • • • • Public key signature, encryption, and decryption to enable secure storage of data and digital secrets Storage of hashes (unique numbers calculated from pre‐runtime configuration information) that enable verifiable attestation of the machine configuration when booted An endorsement key that can be used to anonymously establish that an identity key was generated in a TPM.
2 Hardware Description eUSB Embedded Flash module (optional) The CPM supports up to two embedded USB (eUSB) NAND Flash modules. The modules are USB 2.0 compliant and currently provide sizes from 1GB to 32GB of SLC NAND Flash. The eUSB modules have read speed support of up to 35MB/s and write speed up to 17MB/s. Figure 8 shows an eUSB NAND Flash module ready for installation on the ATCA‐46xx CPM. Figure 8.
Chapter 3 Software/Firmware Description Introduction The CPM has the following software and firmware associated with it: • The system BIOS • The IPMI firmware • Software/firmware update support software • Operating system support software The following sections provide more information. System BIOS The system BIOS is designed on a base source code licensed from American MegaTrends Inc. (AMIBIOS) and is adapted to meet the requirements of the ATCA‐46xx CPM.
3 Software/Firmware Description Table 10. BIOS setup menu hierarchy.
3 Software/Firmware Description Table 12 lists the Radisys default settings for the Advanced menu of the CPM BIOS setup. Table 12.
3 Software/Firmware Description Table 12.
3 Software/Firmware Description Table 12.
3 Software/Firmware Description Table 12.
3 Software/Firmware Description Table 12.
3 Software/Firmware Description Table 12. Radisys default BIOS Advanced menu setup options (continued) Submenu Serial Port Console Redirection Serial Port Console Redirection: Redirection Via AMI Debugger Advanced Menu Setup Item COM0: Console Redirection COM1: Console Redirection Console Redirection Engineering mode.
3 Software/Firmware Description Table 13 lists the Radisys default settings for the Chipset menu of the CPM BIOS setup.. Table 13. Radisys default BIOS Chipset menu setup options Submenu North Bridge, Memory Configuration Chipset Menu Setup Item Total Memory Current Memory Mode Current Memory Speed Mirroring Sparing Spare Err Threshold DRAM RAPL BWLIMIT Perfmon and DFX devices DRAM RAPL MODE Numa DDR3 Refresh Policy Mem bandwidth throttling Memory ECC Memory VDD Oppor.
3 Software/Firmware Description Table 13.
3 Software/Firmware Description Table 13.
3 Software/Firmware Description Table 14 lists the Radisys default settings for the IPMI menu of the CPM BIOS setup.. Table 14. Radisys default BIOS IPMI menu setup options Submenu Top level: IPMI Menu IPMI menu Setup Item BMC Self Test Status POST Watchdog POST Watchdog Timeout POST Watchdog Policy O/S Watchdog Timer O/S WDT Timer Timeout Values [Default] [Dynamic update] [Enabled] Disabled 30 ... [150] ...
3 Software/Firmware Description Table 15 lists the Radisys default settings for the remaining (Security, Boot, and Exit menus) of the CPM BIOS setup.. Table 15.
Software/Firmware Description 3 RAS support The CPM uses Reliability, Availability and Serviceability (RAS) features to support enhanced boot reliability and reduce system downtime.
Software/Firmware Description 3 At runtime, the CPU triggers a System Management Interrupt (SMI) when memory errors reach a preset threshold. If the runtime error logging is enabled. then SMI determines the cause, clears the error status, and reports the memory error to IPMC. Memory errors can be either correctable or uncorrectable.
Software/Firmware Description 3 POST error handling The Power On Self Test (POST) carried out by BIOS after startup examines the functionality of the modules present on the system. It reports any errors to IPMC in the form of platform event messages. The BIOS continues the boot process as long as no errors are detected that might be essential to proper BIOS operation. Errors that might affect BIOS or system operation can cause the BIOS to halt the boot process.
Software/Firmware Description 3 BIOS recovery Refer to the Firmware and Software Update Instructions for the full BIOS image upload procedure. If the CPM BIOS fails boot due to Flash corruption of both the primary and secondary boot Flash images, the user can recover the Flash BIOS image using a BIOS crisis recovery procedure. When the “BIOS Force Recovery” jumper (pins 5‐6) is installed in the customer header (P2), the BIOS images can be reloaded.
Software/Firmware Description 3 All upgrades can be performed from the CPM’s local CPU: • Through a serial console that is directly connected to the CPM or RTM serial port. • Through a remote serial console as described in Serial‐Over‐LAN on page 65. • By a remote login to the CPM over Ethernet. The session is lost when the CPM reboots. The BIOS setup menus cannot be accessed and the bootup messages cannot be viewed.
Chapter 4 Operation and Maintenance Introduction This chapter presents the following operation and maintenance topics: • Hot swap process • IPMI over LAN • Serial over LAN • Firmware and software upgrades. Hot Swap of the CPM The CPM is hot swap capable and meets the hot swap requirements defined in the PICMG 3.0 Revision 2.0 AdvancedTCA Base Specification. The Shelf Manager controls the hot swap process, and the IPMC enables and disables payload power to the CPM when instructed by the Shelf Manager.
4 Operation and Maintenance 2. Be prepared to fill in values for these variables: The IP address of the Shelf Manager. The IPMB address of the CPM in the shelf. The IPMI channel number representing the CPM Base interface channel. Channel 5 is base interface channel 2, and channel 6 is base interface channel 1. The static IP address to assign to each CPM Base interface channel. The protocol to use (lan for RMCP or lanplus for RMCP+).
4 Operation and Maintenance rsys‐ipmitool ‐I lan ‐H ‐A none ‐t channel setaccess 01 ipmi=on link=on privilege=4 3. Repeat Step 2 to set access to the other channel. IPMI-over-LAN troubleshooting steps 1. View the current settings for a channel: rsys‐ipmitool ‐I ‐H ‐A none ‐t lan print 2. Verify that the IP address is correct. 3. Verify that a non‐zero MAC address is set.
4 Operation and Maintenance Establishing a SOL session Prerequisite: This procedure assumes that the required setup for IPMI‐over‐LAN has been done once for this CPM. For details, see Configuring IPMI‐over‐LAN access on page 63. One-time SOL configuration steps The following two steps configure the CPM for SOL: 1. In the CPM BIOS Setup, change the active serial port to COM1, as follows: Advanced Settings > Serial Port 1 Configuration > Serial port: Enabled 2.
4 Operation and Maintenance SOL session open steps To open a SOL session from a remote computer: 1. Activate the SOL console window: rsys‐ipmitool ‐I ‐H ‐A none ‐C 0 sol activate The SOL session is established. 2. Perform an action (such as pressing Enter) from the SOL console window. The SOL console window should respond. SOL session close step Note: Only one SOL session can be open at a time to a single CPM.
4 Operation and Maintenance Overview of firmware updates Table 16 summarizes the CPM’s programmable devices and the content that can be updated. Table 16.
Chapter 5 Troubleshooting and Repair Introduction The procedures presented or referenced in this chapter detail removal and replacement of CPM Field Replaceable Units (FRUs) and provide troubleshooting procedures that can be used to discover FRU’s that need to be repaired or replaced. The following CPM‐related items can be installed or replaced: • The CPM itself. For installation instructions, see the ATCA‐4xxx Compute Processing Module Installation Guide.
5 Troubleshooting and Repair Some of the multirecord area records used are: • Carrier information record • Carrier activation and current management record • Board point‐to‐point connectivity record • Carrier point‐to‐point connectivity record • Carrier clock point‐to‐point connectivity record CPM and FRU device IDs The CPM IPMC contains unique identification information. Table 17 describes those identifiers. Table 17.
5 Troubleshooting and Repair CPM replacement procedures Removing the CPM The following steps explain the hot‐swap procedure for replacing the CPM: 1. Read Electrostatic discharge on page 10 and make sure you are adequately grounded before handling any of the modules. 2. Before replacing the CPM, disconnect all cables from the front panel. 3. Loosen the two thumbscrews securing the CPM. 4. Release the module locking ejector latch that contains the hot‐swap switch.
5 Troubleshooting and Repair Removing the CPM board cover The CPM cooling shroud consists of the main board cover. The entire cover is attached to the board with a number of flat Phillips head screws. Perform the following steps to remove the CPM main board cover: 1. Power down the CPM or activate the hot swap switch so it can safely be uninstalled. 2. Remove the CPM from its slot and set it on an ESD‐safe work surface as described in Removing the CPM on page 71. 3.
5 Troubleshooting and Repair Memory module replacement procedures Note: While the ATCA‐46xx CPM can operate using a wide range of DIMM memory combinations, at least one DIMM module should be installed in each CPU DIMM bank to ensure there are not boot initialization errors.
5 Troubleshooting and Repair Figure 9. DIMM insertion/removal 3. Hold the DIMM by the edges and remove it. 4. Remove the other DIMMs in the same manner. To replace the DIMM with a new card, follow the steps under Installing DIMMs. 5. Screw the main board cover back into place. 6. Power up the CPM and check the BIOS screens to make sure all memory is detected. Installing DIMMs These instructions assume that the CPM has been removed from the shelf and the main board cover has been removed. 1.
5 Troubleshooting and Repair • • • The DIMM is incorrectly installed The DIMM is faulty The DIMM installation violates the memory population rules A DIMM that is not detected or enabled properly can be identified through the BIOS setup menu (Chipset > CPU Socket 0/1 DIMM Information). MXM module installation/replacement procedures Installing an MXM module There are two types of MXM module supported for the CPM – an MXM video module or the Radisys DSSD MXM module.
5 Troubleshooting and Repair 4. Press the MXM module into the MXM connector until it is fully seated. 5. Align the MXM mounting holes with the board standoffs (nearer the CPM front panel) and secure the module with one (DSSD module) or two (MXM video module) screws. 6. Reinstall the CPM cover (see Installing the CPM board cover on page 72). 7. Reinstall the CPM in its slot (see Installing the CPM on page 72). Removing an MXM module Perform the following steps: 1.
5 Troubleshooting and Repair Installing an eUSB module Perform the following steps: 1. Remove the CPM from its slot (see Removing the CPM on page 71). 2. Remove the CPM cover (see Removing the CPM board cover on page 72). 3. Align the first the eUSB module (between the J20 and J23 connectors at the back of the CPM) over the connectors. 4. Press the first the eUSB module into its connectors until it is fully seated. 5.
5 Troubleshooting and Repair Symptoms and recommended actions Table 18 lists possible troubleshooting scenarios. Look through the listed symptoms to see if any apply to your situation and follow the recommended actions for the applicable symptoms. When an action reveals the cause of the problem, resolve the problem as indicated. Table 18. Troubleshooting actions based on symptoms Symptom Recommendation The power LED on the CPM is not lit.
5 Troubleshooting and Repair Table 18. Troubleshooting actions based on symptoms (continued) Symptom The login prompt does not appear after BIOS has booted. Recommendation • Verify the serial cable is plugged into both the CPM and the system with the serial connection. • Verify the CPM is inserted into one of the shelf’s node slots. • Verify the terminal emulation application is set to 115200 bps, no parity, 8 data bits, 1 stop bit, with no flow control. Message: Operating system not found.
Appendix A Specifications The Radisys ATCA‐46xx CPM complies with or meets the standards and specifications presented in the following sections. Standards and interfaces Table 19 lists the standards and interfaces that apply to the CPM. Table 19. CPM Standards and Interfaces Standards/Interfaces Standards Description • PICMG 3.0 R3.0.1 AdvancedTCA • PICMG 3.1 R2.
A Specifications Environmental specifications Radisys does not provide environmental certification testing because any meaningful emissions agency certification must include the entire system. Thus, the CPM is designed and tested to pass the environmental specifications noted below, but it is not certified. WARNING! This product contains static‐sensitive components and should be handled with care. Failure to employ adequate anti‐static measures can cause irreparable damage to components.
A Specifications Safety specifications The safety specifications are measured under laboratory ambient temperature and humidity (approximately 55C and humidity between 30% and 50%). Testing was performed in partnership with a Nationally Recognized Testing Laboratory (NRTL) accredited to provide the required certifications. Table 21.
A Specifications Electromagnetic compatibility (EMC) The ESD, EMC, and Immunity specifications are measured with ambient temperature between 20C and 30C and relative humidity between 30% and 50%. Table 23.
A Specifications Network Equipment Building Standard (NEBS) The CPM is designed to meet the NEBS requirements listed in Table 24.
A Specifications Mean time between failures (MTBF) The board MTBF is designed to meet or exceed 150,000 hours @ +35C per Telcordia SR332 Issue 2, Method 1, Case 3. The calculation results in Table 25 were generated using the references and assumptions listed. This specification and its associated calculations supersede all other released mean time between failures (MTBF), annual failure rate (AFR), early return index (ERI), and dead‐on‐arrival (DOA) calculations with earlier dates.
Appendix B IPMI Commands and Managed Sensors IPMI command interfaces The following interfaces use IPMI command support: • I2C bus connections with the following links: • I2C bus 0 ‐ IPMB0‐A • I2C bus 1 ‐ IPMB0‐B • I2C bus 2 ‐ access to IPMC FPGA, CC FPGA, and RTC circuitry • I2C bus 3‐ access to voltage monitoring, MXM, and PCIe retimer • I2C bus 4 ‐ SOL access to GbE controller and base/front panel Ethernet • I2C bus 5 ‐ access to PCH (thermal monitoring), IPMB‐L, and RTM • Serial connections u
B IPMI Commands and Managed Sensors Table 26.
B IPMI Commands and Managed Sensors Table 26.
B IPMI Commands and Managed Sensors Table 26.
B IPMI Commands and Managed Sensors Get Control State command (for debug only) This command returns the current state of a control pin. Table 29 is the command description of the Get Control State command. Table 29. Get Control State OEM command Data Type Data Field Response Field Byte 1 2 1 2 Data Field FRU ID Control number Completion Code Control State 00h = de-asserted 01h = asserted Disable CFD command This command indicates to the IPMC that the CFD Watchdog Timer needs to be disabled.
B IPMI Commands and Managed Sensors Switch Active Boot Flash command This command sets the current Boot Flash and can cause a cold reset to the x86 Processor Complex portion of the board if the appropriate bits in the command data are set. The command can also set the primary Boot Flash, which is selected during a power‐on reset of the blade. Table 32 is the command description of the Switch Active Boot Flash command. Table 32.
B IPMI Commands and Managed Sensors RTM Reset Button command This command instructs the H8 IPMI Firmware to perform a COLD reset. Table 34 is the command description of the Get Active Boot Flash command. Table 34.
IPMI Commands and Managed Sensors B Set Payload Status command This command informs the H8 IPMI Firmware the current payload processor status. The IPMI firmware may use the reported information to initiate its internal processes which are dependent on resources controllable by the onboard x86 processor complex. In the CPM, the boot phases 1 and 4 specified in status byte 1 are used to report the commencement and completion of the onboard x86 processor complex boot‐up processes.
IPMI Commands and Managed Sensors B Get Payload Status (for debug only) command This command returns the H8 IPMI Firmware acknowledgement of the payload processor status. The BIOS/OS may use this command to check if the IPMI firmware has finished the internal processes for the onboard x86 Processor Complex boot phase specified in a (previously sent) Set Payload status command.
IPMI Commands and Managed Sensors B Managed sensors On the CPM, the IPMC sensors monitor voltages, temperatures, control signals, and status events. For functional information, refer to IPMI controller on page 33. The sensors are described in Table 38. Types of sensors The CPM implements the following types of sensors. • Discrete — A discrete sensor can have up to 16 bitmapped states, with one state as true.
B IPMI Commands and Managed Sensors IPMI Sensors The CPM supports a variety of sensors, each with entries in the Sensor Data Records (SDR). Table 38 lists the IPMI sensors supported by the CPM. Table 38. ATCA-46xx IPMI Managed Sensors Sensor Name Type # 0 ATCA FRU Hot ATCA FRU Hot swap swap 1 RTM FRU Hot swap ATCA FRU Hot swap 2 Version Change Version Change (IPMI 2.
B IPMI Commands and Managed Sensors Table 38.
B IPMI Commands and Managed Sensors Table 38. ATCA-46xx IPMI Managed Sensors (continued) Sensor Name # 33 +12V Voltage Reading Type Threshold Normal Reading 12.00 34 +5V Voltage Threshold 5.00 35 +5V Standby Voltage Threshold 5.00 36 +3.3V IPMI Voltage Threshold 3.30 37 +3.3V Voltage Threshold 3.30 38 +1.8V Voltage Threshold 1.80 39 +1.8V FR Voltage Threshold 1.80 Type 98 Notes LNR = 0.00 LC = 10.8 LNC = 11.4 UNC = 12.6 UC = 13.2 UNR = 13.8 LNR = 0.00 LC = 4.5 LNC = 4.
B IPMI Commands and Managed Sensors Table 38. ATCA-46xx IPMI Managed Sensors (continued) Sensor Name # 40 +1.5V PCH Voltage Reading Type Threshold Normal Reading 1.50 41 +1.2V Voltage Threshold 1.20 42 +1.1V Voltage Threshold 1.10 43 +1V Voltage Threshold 1.00 44 VCCP0 Voltage Threshold 1.04 VID = 0.75 ~1.35 45 VCCP1 Voltage Threshold 1.04 VID = 0.75 ~1.35 46 +VDDQ0 Voltage Threshold 1.50 Type 99 Notes LNR = 0.00 LC = 1.35 LNC = 1.43 UNC = 1.58 UC = 1.65 UNR = 1.
B IPMI Commands and Managed Sensors Table 38. ATCA-46xx IPMI Managed Sensors (continued) Sensor Name # 47 +VDDQ1 Voltage Reading Type Threshold 48 +VTT0 Voltage Threshold 49 +VTT1 Voltage Threshold 50 +VTT DDR0 Voltage Threshold 51 +VTT DDR1 Voltage Threshold 52 +2.5V FE Voltage Threshold 53 +VDD FE Voltage Threshold Type Normal Reading 1.50 LNR = 0.51 LC = 1.4 LNC = 1.46 UNC = 1.56 UC = 1.61 UNR = 1.66 1.05 LNR = 0.50 (SNB)/1.0(I LC = 0.96 VB) LNC = 1.00 UNC = 1.10 UC = 1.
B IPMI Commands and Managed Sensors Table 38. ATCA-46xx IPMI Managed Sensors (continued) Sensor Name # 54 +VCCPLL0 Voltage Reading Type Threshold 55 +VCCPLL1 Voltage Threshold 56 +VSA0 Voltage Threshold 57 +VSA1 Voltage Threshold 58 +1.2V Standby Voltage Threshold 59 +LVDDQ0 Voltage Threshold 60 +LVDDQ1 Voltage Threshold Type Normal Reading 1.8(SNB)/1. LNR = 0.00 7(IVB) LC = 1.68 LNC = 1.74 UNC = 1.89 UC = 1.95 UNR = 1.99 1.8(SNB)/1. LNR = 0.00 7(IVB) LC = 1.68 LNC = 1.
B IPMI Commands and Managed Sensors Table 38. ATCA-46xx IPMI Managed Sensors (continued) Sensor Name # 61 +LVTT DDR0 Voltage Reading Type Threshold Normal Reading 0.675 (LVDIMM) 62 +LVTT DDR1 Voltage Threshold 0.675 (LVDIMM) 63 Inlet Temp 1 Temperature Threshold 25 64 Inlet Temp 2 Temperature Threshold 25 65 CPU0 DIMM Temp Temperature Threshold 25 66 CPU1 DIMM Temp Temperature Threshold 25 67 PCH Die Temp Threshold 25 Type Temperature 102 Notes LNR = 0.50 LC = 0.
B IPMI Commands and Managed Sensors Table 38.
B IPMI Commands and Managed Sensors Table 38. ATCA-46xx IPMI Managed Sensors (continued) Sensor Name # 92 Processor 93 System Firmware Progress Type Processor System Firmware Progress Reading Type Sensorspecific Normal Reading N/A Sensorspecific N/A 104 Notes Offset 03h = Description FRB2/ Hang in POST failure (used hang is believed to be due or related to a processor failure. Use System Firmware Progress sensor for other BIOS hangs.
B IPMI Commands and Managed Sensors Table 38. ATCA-46xx IPMI Managed Sensors (continued) Sensor Name # 93 System Firmware (cont.) Progress (cont.) Type Reading Type Normal Reading Notes 02h = System Firmware Progress Event Data 2: 00h - Unspecified. 01h - Memory initialization.
B IPMI Commands and Managed Sensors Table 38.
B IPMI Commands and Managed Sensors Table 38. ATCA-46xx IPMI Managed Sensors (continued) Sensor Name # 99 Failover Type OEM Failover Reading Type Sensorspecific Normal Reading N/A Notes Description OEM Failover (Only applicable if redundancy is available.) eventData1: 0 eventData2: 0 & eventData3: 4' Failover Start eventData2: 1 & eventData3: FF' Failover Complete 100 HPI Event OEM HPI OEM N/A Offset Description 01h = OEM HPI (Only applicable if Radisys Shelf Manager application is installed.
Appendix C Pinouts and Mapping The ATCA‐46xx CPM pinout listings apply to front panel connectors and the Zone 1 and 2 backplane connectors. The onboard header pinouts apply to user‐accessible areas of the CPM board. Front panel connectors The following sections describe the pinouts for the front panel connectors. COM serial connector Table 39 lists the pinout for the front panel COM serial connector. Table 39.
C Pinouts and Mapping Dual Ethernet connectors Table 41 lists the pinout for each of the front panel RJ45 Ethernet GbE connectors. Note that power, ground, and the LED control signals are fed from the PCB and are not accessible at the connector pins. Refer to Table 1 on page 18 for detailed information on the connector LEDs. Table 41.
C Pinouts and Mapping Backplane interfaces Backplane connectivity summary This section describes the backplane interface connectivity. Table 43 lists the backplane connectors and summarizes their usage. Table 43.
C Pinouts and Mapping Zone 1 P10 connector pinout Table 44 lists the P10 connector pinout. Table 44.
C Pinouts and Mapping Zone 2 J20 connector pinout Table 45 lists the J20 connector pinout. Table 45. Backplane connector J20 signals Row Interface designation AB CD 1 Clks CLK1A+ CLK1A– CLK1B+ CLK1B– 2 Update Channel and Clks Tx4(UP)+ Tx4(UP)– Rx4(UP)+ Rx4(UP)– 3 4 Tx0(UP)+ Tx0(UP)– Rx0(UP)+ Rx0(UP)– 5 Fabric Channel 15 6 7 Fabric Channel 14 8 9 Fabric Channel 13 10 Note: Each differential pair has an individual L-shaped ground contact (not shown).
C Pinouts and Mapping RTM interface pinout Zone 3 J30 connector pinout Table 47 lists the Zone 3 J30 connector pinout. Table 47. RTM connector J30 signals Row AB CD EF 1 +12V_RTM +12V_RTM +12V_RTM +3.3V_IPMC 2 +12V_RTM +12V_RTM +12V_RTM IPMC_I2C_CLK 3 SERIAL_0_TX SERIAL_0_RX JTAG_TDI JTAG_TDO JTAG_TMS 4 INT_0 INT_1 RTML_TX 5 6 7 8 9 SAS0_TX+ SAS0_TX– SAS0_RX+ SAS0_RX– SAS1_TX+ 10 GE1_TX+ GE1_TX– GE1_RX+ GE1_RX– GE0_TX+ Note: Each differential pair has an individual L-shaped ground contact (not listed).
C Pinouts and Mapping Onboard switches, headers, and connectors Onboard switches There are two switches on the CPM, both on the front panel, as follow: • The reset switch • The hot swap eject switch The recessed reset push button (SW1) is located in the lower half of the front panel, just above the Base/Fabric channel Status LEDs (see Figure 2 on page 17). The hot swap eject switch is connected to the lower blade latch assembly.
C Pinouts and Mapping 5‐6BIOS Force Recovery The BIOS force recovery routine will execute on every boot when this jumper is installed. 7‐8eUSB Flash Write Protect The write protect input to installed eUSB devices is asserted when this jumper is in place. 9‐10Disable UNR Shutdown Prevents the IPMC from shutting down a blade when an upper non‐recoverable (UNR) threshold is exceeded. 11‐12ME Firmware Recovery Mode Causes the ME firmware to stay in the recovery boot loader.
C Pinouts and Mapping MXM connector The onboard MXM connector is used by two types of Mobile PCI eXpress Module (MXM) devices; a supported Type A MXM 3.0 video module or the Radisys Dual Solid State Drive (DSSD) MXM module. Both types of supported modules derive all input, output, and power resources from the MXM connector. The pinout for the MXM connector is defined in the MXM 3.0 specification and is also listed in Table 50. Table 50. MXM 3.
C Pinouts and Mapping Table 50. MXM 3.
C Pinouts and Mapping Dual micro SAS connector A dual stacked micro‐SAS connector is included on the SDDS MXM module to support the installation of two 1.8" micro SATA SSD drives. Table 51 lists the pinout for the dual micro SAS connectors. Table 51.
C Pinouts and Mapping eUSB connector There are two Embedded Universal Serial Bus (eUSB) connectors located between the J20 and J23 backplane connectors at the back edge of the CPM. These connectors can accept one or two dual stackable eUSB flash modules. Table 52 lists the pinout for each of the eUSB connectors. Table 52. eUSB connector pinout Pin 1 2 3 4 5 Signal EUSB_RST* EUSB_WP* NC NC +3.
