Dell EMC Open Networking Troubleshooting Guide May 2018
Notes, Cautions, and Warnings NOTE: A NOTE indicates important information that helps you make better use of your computer. CAUTION: A CAUTION indicates either potential damage to hardware or loss of data and tells you how to avoid the problem. WARNING: A WARNING indicates a potential for property damage, personal injury, or death. Copyright © 2018 Dell Inc. or its subsidiaries. All rights reserved. Dell, EMC, and other trademarks are trademarks of Dell Inc. or its subsidiaries.
Contents 1 About this guide............................................................................................................................................. 4 Information symbols...........................................................................................................................................................4 Related documents....................................................................................................................................................
1 About this guide This guide provides site preparation recommendations, step-by-step procedures for rack mounting and desk mounting, inserting optional modules, and connecting to a power source. CAUTION: To avoid electrostatic discharge (ESD) damage, wear grounding wrist straps when handling this equipment. WARNING: Only trained and qualified personnel can install this equipment. Read this guide before you install and power up this equipment. This equipment contains two power cords.
• Dell EMC Open Networking (ON) Installation Guide • Dell EMC Open Networking (ON) Release Notes About this guide 5
2 ONIE diagnostics This chapter describes system diagnostics and troubleshooting. After running the diagnostic tools, your system displays pass or fail test results. If all tests pass, the diagnostic tools exit normally. If a test fails, each diagnostic tool offers a different result. NOTE: The troubleshooting package includes a README file that lists the tools version and the overall troubleshooting package version. For more information, refer to this README file.
• Troubleshooting tools S4810-ON system information To view your system information; for example, the model, part number, serial number, and service tag, follow these steps. 1 Reboot your system and enter U-Boot mode. 2 Enter the sys_eeprom command.
Capturing Support Data from ONIE To capture support data from ONIE, use the following commands. 1 Capture support data to the screen. ONIE:/ # dmesg 2 Capture support data to the onie-support.tar.bz2 gzip file. ONIE:/ # onie-support The ONIE support file includes the following: • kernel_cmdline • runtime-export-env • runtime-process • runtime-set-env • log/messages • log/onie.
ONIE Rescue bypasses the installed operating system and boots the system into ONIE until you reboot the system. 2 Press ENTER to activate the console. Example of the Optional ONIE-uninstaller Command After ONIE Rescue completes, the system resets and boots to the ONIE console. NOTE: Only use the optional onie-uninstaller command if you want to remove all the network operating software on your system except for ONIE. This command removes any installed network operating system.
Erasing block: 128/128 (100%) Writing kb: 16376/16384 (99%) Verifying kb: 16376/16384 (99%) ONIE:/ # umount: can't remount rootfs read-only The system is going down NOW! Sent SIGTERM to all processes Sent SIGKILL toRestarting system. Reset via the platform CPLD 4 Start ONIE diagnostics. To start the ONIE diagnostics for the S6000–ON, use the onie-diag option from the ONIE menu.
Fan 3 speed is 12240 RPM Fan 4 speed is 12240 RPM Testing Memory Regions: Testing Memory Region 0: Address Read Test ......................................... Passed Address Write Test ........................................ Passed Address Walking 1's Test .................................. Passed Address Walking 0's Test .................................. Passed Data Read Test ............................................ Passed Data Write Test ...........................................
Problem Description and Solution The Fantool reports a failure, but the fans seem to be working correctly. Troubleshooting tools This section describes the diagnostic tools that provide debug and hardware tests. NOTE: The following ONIE diagnostic tools do not apply to the S4048–ON and S3048–ON systems. To use the troubleshooting tools manually, you must be at the ONIE prompt and navigate to /mnt/diag/. If you are not already at the ONIE prompt, reboot your system to Rescue mode.
+ Checking Dev found @ 0x49 ... ------ ------ 0x49 ..... Passed + Checking Dev found @ 0x4a ... ------ ------ 0x4a ..... Passed + Checking Dev found @ 0x4c ... ------ ------ 0x4c ..... Passed + Checking Dev found @ 0x4d ... ------ ------ 0x4d ..... Passed Checking I2C devices on bus 2: I2C Devices: Overall test results --------------------- >>> Passed Testing Programmable Devices: PL Tool test: + Checking System CPLD 0xffdf0000 Reg: 0x10 ....................
Example of the pcitool Configuation File Output # more default_pci_list.cfg Bus:Dev.Fn=00:00.0 ID=0c738086 NOT LISTED 0c738086 00000007 06000002 00000000 00000000 00000000 00000000 00000000 00008086 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 01080000 Bus:Dev.Fn=00:01.
The memory configuration file consists of lines that describe a region of memory and the tests performed on that memory region. Therefore, you can have multiple entries for a region of memory. All parameters are separated by a : character. The following describes the configuration file parameters. Parameter Description Region Name: The region name referred to in all output. Start Address: The starting address for the region of memory in hexadecimal format (without the preceding 0x).
read [b|h|w] address := read the specified physical address write [b|h|w] address data [length] := write at the specified physical address Example of the memtool Configuration File Output // Memory Configuration File // // Example: // SystemRam:-:-:w:4:1:2800:0:0:1:-1:SPD:/dev/i2c-0:50:0,ff: // This describes the SystemRam which is dynamic in location and size. It is accessed by words // and incremented addresses of 4 bytes.
C-Row Parameter Description C The row identifier. Type CPLD, FPGA, or TPM. Address The address of the device. For CPLD on an I2C bus, this is an 8–bit address. For PCI, this is the bus:dev:function in a packed 32–bit word, 8–bits each. For memory, this is the address. LPC is unknown. Name The text name for the device. Interface i2c, pci, mem, io, or lpc. Bus For devices on multiple busses, this indicates the bus number (for example, 0 for i2c indicates /dev/i2c-0.
B-Row Parameter Description Bit Number(s) This can be either a single bit number or a range starting with the highest bit number (for example, 7 or 7:3). Name The name of the bit. Access The access type of the bit; the same as the register definition. Default Value The default value of the bits. The following describes the I-row configuration file tree output. I-Row Parameter Description I The identifier row descriptor. Value The value of the collection of bits.
gpiotool The gpiotool verifies the state of the GPIO signals. This test is more commonly used in ODA Offline and Online modes. gpiotool can also change the GPIO values for I/O devices and internal CPU GPIOs. Example of the gpiotool Output Syntax: ./gpiotool
In the fantool configuration output, the first six rows describe the six fans in your system and the devices that control the fans. The second portion lists the speeds for the fans: low, medium, high, and default. The first portion of the output lists the fan IDs, name, the device that controls the fan, the fan address on the I2C bus, and the instance of the fan control within the chip. Also included is a description of which fan tray the fan is located in.
3 Dell EMC diagnostics The following describes the Dell EMC diagnostics. These instructions apply to systems for which the ONIE diagnostics are not available. ONIE expansion To view all the ONIE commands available, from the ONIE prompt, enter onie- and click twice.
INSTALLER-DND-SK-1.0 22% |****** | 6050k 0:00:10 INSTALLER-DND-SK-1.0 31% |********* | 8625k 0:00:08 INSTALLER-DND-SK-1.0 40% |************ | 10986k 0:00:07 INSTALLER-DND-SK-1.0 48% |************** | 13062k 0:00:06 INSTALLER-DND-SK-1.0 56% |***************** | 15268k 0:00:05 INSTALLER-DND-SK-1.0 66% |******************** | 18026k 0:00:04 INSTALLER-DND-SK-1.0 75% |*********************** | 20535k 0:00:02 INSTALLER-DND-SK-1.0 84% |************************** | 22803k 0:00:01 INSTALLER-DND-SK-1.
You will see a “Welcome to Grub” message at the beginning of the process and the DCLI-> prompt at the end of the process. GNU GRUB version 2.02~beta2+e4a1fe391 +---------------------------+ |ONIE: Install OS | |ONIE: Rescue | |ONIE: Uninstall OS | |ONIE: Update ONIE | |ONIE: Embed ONIE | |*DELL EMC DIAG | | | | | NOTE: The following commands are available at the DCLI prompt. 2 At the DCLI> prompt, enter the testall command to run all the Dell EMC diagnostics.
The Booted Bios Version : x.xx.x.x BIOSVersionTest .............................. PASS Starting test: BoardRevisionTest ...... System CPLD: Board Stage: 0x2, Cpld Rev: 0x7 BoardRevisionTest ............................ PASS CpldAccessTest ............................... PASS CpuSdramPresenceTest ......................... PASS CpuTypeDetectTest ............................ PASS FanAirFlowTypeTest ........................... PASS FanStatusMonitorTest ......................... PASS FanTrayPresenceTest ......
sfpPlusPhyLnkSpeedTest ......................... SsdFileCopyTest ................................ Trident2AccessTest ............................. TSensorAccessTest .............................. UsbFileCopyTest ................................ PASS PASS PASS PASS PASS LEVEL 2 DIAGNOSTIC CpuSnakeQsfpPlusExtLpbkTest .................... CpuSnakeQsfpPlusMacLpbkTest .................... CpuSnakeQsfpPlusPhyLpbkTest .................... CpuSnakeSfpPlusExtLpbkTest .....................
PsuFanStatusMonitorTest ......................... PsuPresenceTest ................................. PsuSourceTypeTest ............................... PsuStatusMonitorTest ............................ RtcPresenceTest ................................. SfpPlusModulePresenceTest ....................... ShowTemperatureTest ............................. SmcBoardRevisionTest ............................ SsdPresenceTest ................................. UsbAAccessTest ..................................
Example of the onie-syseeprom Command TlvInfo Header: Id String: TlvInfo Version: 1 Total Length: 78 TLV Name Code Len Value -------------------- ---- --- ----MAC Addresses 0x2A 2 129 Base MAC Address 0x24 6 00:05:33:6A:BF:4D Vendor Name 0x2D 4 Dell Product Name 0x21 8 S6000-ON Part Number 0x22 6 08YWFG Serial Number 0x23 12 DLCN13980015 Label Revision 0x27 3 A00 Manufacturer 0x2B 1 1 Service Tag 0x2F 2 ABC1AB2 Loader Version 0x29 8 x.xx.x.x CRC-32 0xFE 4 0xC1EB87D1 Checksum is valid.
Serial Number Sevice Tag Fantray Info: FanTray[1] AirFlow Direction Part Number Serial Number Sevice Tag FanTray[2] AirFlow Direction Part Number Serial Number Sevice Tag FanTray[3] AirFlow Direction Part Number Serial Number Sevice Tag : TW0T9FNW2829849Q0041 : AEIOU## : : : : NORMAL 0MGDH8 TW0MGDH82829849Q0002 AEIOU## : : : : NORMAL 0MGDH8 TW0MGDH82829849Q0003 AEIOU## : : : : NORMAL 0MGDH8 TW0MGDH82829849Q0001 AEIOU## ********************************************************* S6000–ON, S4048–ON, or
4 Dell EMC support Dell EMC Support provides a range of documents and tools to assist you with effectively using Dell EMC equipment and mitigating the impact of network outages. Through Dell EMC Support you can obtain technical information regarding Dell EMC products, access to software upgrades and patches, and open and manage your Technical Assistance Center (TAC) cases. Dell EMC support provides integrated, secure access to these services.