Installation Guide HP bh5700 ATCA 14-Slot Blade Server First Edition Manufacturing Part Number : AD171-9604A June 2006 Printed in the U.S.A.
Legal Notices The information in this document is subject to change without notice. Hewlett-Packard makes no warranty of any kind with regard to this document, including, but not limited to, the implied warranties of merchantability and fitness for a particular purpose. Hewlett- Packard shall not be held liable for errors contained herein or direct, indirect, special, incidental or consequential damages in connection with the furnishing, performance, or use of this material. Restricted Rights Legend.
Contents 1. Introduction Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Installation Checklist . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Safety Considerations. . .
Contents Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Tables Table 1-1. Installation Checklist . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Table 3-1. Rack Mount Kit, HP bh5700 ATCA 14-Slot Blade Server, (PN Z7678-60275) . . . . . . . . Table 6-1. Default rc Script VLAN Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Table 6-2. HP bc2100 ATCA Server Blade BIOS Default Port Assignments . . . . . . . . . . . . . . . . . . Table 7-1.
Tables 6
Figures Figure 1-1. HP bh5700 ATCA 14-Slot Blade Server, Basic Configuration . . . . . . . . . . . . . . . . . . . . Figure 1-2. HP bh5700 ATCA 14-Slot Blade Server (Front View). . . . . . . . . . . . . . . . . . . . . . . . . . . Figure 1-3. HP bh5700 ATCA 14-Slot Blade Server (Rear View) . . . . . . . . . . . . . . . . . . . . . . . . . . . Figure 1-4. ESD Wrist Strap Terminal Locations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Figure 2-1.
Figures 8
1 Introduction Chapter 1 9
Introduction Introduction Introduction NOTE There are some current issues with the HP bh5700 ATCA 14-Slot Blade Server. The issues are in the process of being solved. Please refer to www.docs.hp.com (customer-viewable) to see the latest list of issues and ways to work through them. The list of issues will be updated frequently.
Introduction Introduction Figure 1-1 Chapter 1 HP bh5700 ATCA 14-Slot Blade Server, Basic Configuration 11
Introduction Introduction Figure 1-2 HP bh5700 ATCA 14-Slot Blade Server (Front View) 1 6 7 2 8 3 4 9 10 5 Use the following list to identify the callouts from Figure 1-2: 1. Alarm display panel 2. ATCA 14-Slot backplane 3. Front card cage 4. Air filter 5. Primary Shelf Manager (left) 6. Alarm connector panel 7. Front cable tray 8. Rack mounting flange 9. ESD wrist strap terminal 10.
Introduction Introduction Figure 1-3 HP bh5700 ATCA 14-Slot Blade Server (Rear View) 5 1 6 2 3 7 8 4 9 Use the following list to identify the callouts from Figure 1-3: 1. Fan tray #2 2. Rear card cage 3. ESD wrist strap terminal 4. Power entry module (PEM B) 5. Fan tray #1 6. Fan tray #0 7. Rear cable tray 8. Shelf ground terminal (M6 studs) 9.
Introduction Introduction Installation Checklist This section provides an installation checklist to assist you with preparing the installation site and installing the 14-Slot Shelf. Table 1-1 Installation Checklist Installation Procedure (See chapters 1 through 5) In-process Completed SITE PREPARATION VERIFIED WARNING: Observe all electrical safety precautions when configuring the external power supplies. This is a high voltage hazard.
Introduction Introduction Safety Considerations All site preparation, installation, and service work on the 14-Slot Shelf must be performed by qualified personnel. WARNING Failure to follow all electrical and mechanical safety precautions in this document could result in serious personal injury. Communications Interference HP system compliance tests are conducted with HP supported peripheral devices and shielded cables, such as those received with the system.
Introduction Introduction Follow these ESD precautions: • Prepare an ESD-safe work surface large enough to accommodate the FRUs being handled during servicing. Use a grounding mat and an anti-static wrist strap, such as those included in the ESD Field Service Kit (A3024-80004). • The anti-static bag that encloses FRUs cannot function as a static dissipating mat. Do not use the anti-static bag for any other purpose than to enclose a product.
2 Unpacking the 14-Slot Shelf Chapter 2 17
Unpacking the 14-Slot Shelf Unpacking the 14-Slot Shelf Unpacking the 14-Slot Shelf This chapter provides information about unpacking and installing the bundled 14-Slot Shelf. CAUTION To minimize any possibility of physical damage to equipment, ensure that floorspace at the installation site is neat and uncluttered. Ensure that a mechanical lift can be maneuvered in the area to lift the Shelf from the shipping pallet.
Unpacking the 14-Slot Shelf Unpacking the 14-Slot Shelf Figure 2-1 Shipping Carton and Top Foam Cushions Removed Step 4. Remove both the corrugated roll-up spacer and the carton containing the front and rear cable trays along with other important documentation. These are located to either side of the unit. See Figure 2-1. Step 5. Unroll the bottom corrugated tray. See Figure 2-2 on page 20.
Unpacking the 14-Slot Shelf Unpacking the 14-Slot Shelf Figure 2-2 Bottom Corrogated Tray Unrolled Step 6. Position the mechanical lift as shown in Figure 2-3 on page 21. The lift should be in line or slightly below the top surface of the bottom foam cushions. If needed, slide the cushions that the unit is resting on outward, a little closer to the edge of the pallet, to allow the lift to slide under the unit.
Unpacking the 14-Slot Shelf Unpacking the 14-Slot Shelf Figure 2-3 Positioning the Mechanical Lift Under the 14-Slot Shelf Step 7. Push the 14-Slot Shelf onto the mechanical lift. See Figure 2-4 on page 22.
Unpacking the 14-Slot Shelf Unpacking the 14-Slot Shelf Figure 2-4 Pushing the Unit Onto the Mechanical Lift Step 8. Slide the bag down around the server. See Figure 2-5 on page 23. Install cable trays and proceed with the remainder of the installation.
Unpacking the 14-Slot Shelf Unpacking the 14-Slot Shelf Figure 2-5Ready to Attach the Front and Rear Cable Trays Step 9. Locate the front and rear cable trays from the accessory package and mount to the Shelf. To mount the cable trays, locate and remove the existing cable tray chassis screws that match the mounting holes for each tray. Reuse these screws to mount the two cable trays. See Figure 2-6 on page 24. The 14-Slot Shelf is now ready for equipment rack mounting.
Unpacking the 14-Slot Shelf Unpacking the 14-Slot Shelf Figure 2-6 Mounting the Front Cable Tray 24 Chapter 2
3 Rack Mounting the 14-Slot Shelf Chapter 3 25
Rack Mounting the 14-Slot Shelf Rack Mounting the 14-Slot Shelf Rack Mounting the 14-Slot Shelf The HP bh5700 ATCA 14-Slot Blade Server is designed to mount in a 19 in. (48.26 cm) wide, 23.62 in (600 mm) deep, Electronics Industry Alliance (EIA), seismic-rated HP rack. The 14-Slot Shelf is shipped with the rack mounting flanges correctly positioned for mounting into a 4-post seismic-rated equipment rack. This chapter provides detailed procedures for completing the rack mount installation.
Rack Mounting the 14-Slot Shelf Rack Mounting the 14-Slot Shelf Figure 3-1 Mounting Slide Attached to Cabinet Rails (Rear Hold-down Bracket in Place) (not included in this release) IMPORTANT Before installing the Rack Mount Kit, ensure there will be adequate vertical space to install the 14-Slot Shelf (13U chassis height) in addition to other equipment installed. To install the 14-Slot Shelf into the EIA Seismic rack, complete the following steps: Step 1.
Rack Mounting the 14-Slot Shelf Rack Mounting the 14-Slot Shelf Figure 3-2Nut Clip, M5x0.8 Figure 3-3Installing Nut Clip IMPORTANT The right and left rack mounting slides will support the weight of the 14-Slot Shelf when mounted in the rack. Step 3. Install two nut-clips onto the rear cabinet rail in positions that correspond to those installed in step 2 on the front cabinet rail. These nut-clips will be used to attach the rear of the right mounting slide (see note below). Step 4.
Rack Mounting the 14-Slot Shelf Rack Mounting the 14-Slot Shelf Figure 3-4M5x0.8x16, Torx Pan Screw, with Crest Cup Washer Step 6. Using four M5x0.8x16 screws (Figure 3-4), attach the left mounting slide to the front and rear left cabinet rails as a mirror image of the installed right mounting slide shown in Figure 3-1 on page 27. Use a Torx drive torque wrench to tighten each screw to 36 in-pounds (4.067 N-m). Step 7.
Rack Mounting the 14-Slot Shelf Rack Mounting the 14-Slot Shelf WARNING A 14-Slot Shelf weighs from 62.6 kg (138 lbs) for the basic configuration to 90 kg (200 lbs) with the blade cage fully populated. This is a pinch, crush, and heavy lift hazard. Step 9. Place the 14-Slot Shelf on a portable mechanical lift device that can be used to elevate and hold the Shelf in place while it is being installed into the seismic rack. Step 10.
Rack Mounting the 14-Slot Shelf Rack Mounting the 14-Slot Shelf Figure 3-6Flange Mounting Positions for 2- and 4-Post Equipment Racks Shelf Front Step 2. Slide each flange downward until the flange drops out of the five retaining clips that hold the flanges to the side panels. Step 3. Reposition each flange over the five retaining clips for the 2-post mounting position (Figure 3-6), with the clips protruding into the openings in the flange. Step 4.
Rack Mounting the 14-Slot Shelf Rack Mounting the 14-Slot Shelf 32 Chapter 3
4 Connecting Protective Earth Ground and External Power to the Shelf Chapter 4 33
Connecting Protective Earth Ground and External Power to the Shelf Connecting Protective Earth Ground and External Power to the Shelf Connecting Protective Earth Ground and External Power to the Shelf This chapter provides information on connecting protective earth ground and external system power to the installed 14-Slot Shelf. WARNING Disconnect all power cords to completely remove power. This is a shock hazard.
Connecting Protective Earth Ground and External Power to the Shelf Connecting Protective Earth Ground and External Power to the Shelf Figure 4-1 Earth Ground Lugs and PEM Location on Shelf Rear (Numbers 8, 4, & 9) 55 1 2 6 7 3 8 4 9 Use the following list to identify the callouts from Figure 4-1: 1. Fan tray #2 2. Rear card cage 3. ESD wrist strap terminal 4. Power entry module (PEM B) 5. Fan tray #1 6. Fan tray #0 7. Rear cable tray 8. Shelf ground terminal 9.
Connecting Protective Earth Ground and External Power to the Shelf Connecting Protective Earth Ground and External Power to the Shelf Figure 4-2 Earth Ground Lug Detail, Shelf Lower Right Rear Earth Ground Lugs Verifying the Power Supply Output WARNING Observe all electrical hazard safety procedures when verifying the external power supply voltages. Failure to comply can result in personal injury.
Connecting Protective Earth Ground and External Power to the Shelf Connecting Protective Earth Ground and External Power to the Shelf Figure 4-3 Typical High Availability Power Connection Block Diagram Minimum Power Connection Cable Specifications CAUTION All input power and return wiring should be specified, configure, and installed by a qualified electrician in order to prevent damage to the equipment.
Connecting Protective Earth Ground and External Power to the Shelf Connecting Protective Earth Ground and External Power to the Shelf Connecting the Power and Return Cables WARNING Ensure that each power domain supply circuit breaker is switched OFF while completing the following power connection procedure. Failure to comply can result in personal injury. To connect external power and voltage return (VRTN) leads to the 14-Slot Shelf, complete the following steps: Step 1.
Connecting Protective Earth Ground and External Power to the Shelf Connecting Protective Earth Ground and External Power to the Shelf Figure 4-5 Typical PEM Power Cable Connection Layout Step 4. On PEM B, replace and secure the PEM B terminal block cover. WARNING When external power is applied, the 14-Slot Shelf is energized at all times. Power to the 14-Slot Shelf is controlled only from the external power source, and cannot be switched off at the 14-Slot Shelf.
Connecting Protective Earth Ground and External Power to the Shelf Connecting Protective Earth Ground and External Power to the Shelf 40 Chapter 4
5 Installing 14-Slot Shelf Components Chapter 5 41
Installing 14-Slot Shelf Components Installing 14-Slot Shelf Components Installing 14-Slot Shelf Components This chapter provides information for installing memory modules into the HP bc2100 ATCA Server Blade. Server Blade Memory Modules The HP bc2100 ATCA Server Blade Dual In-line Memory Modules (DIMMs) are always installed in pairs. Each individual DIMM within a pair must also be identical in storage capacity, revision, and part number.
Installing 14-Slot Shelf Components Installing 14-Slot Shelf Components Figure 5-1 Server Blade with Access Cover Panel Installed Step 3. Remove the countersunk retaining screws that secure the blade access cover panel and remove the cover panel. Set the cover panel and retaining screws aside for reuse. Step 4. To release the DIMM from its socket, pull the sockets end levers away from the memory module (see Figure 5-2 on page 44).
Installing 14-Slot Shelf Components Installing 14-Slot Shelf Components Figure 5-2 Releasing the DIMM from DIMM Socket U5 Step 5. Grasp the memory module and remove by angling away from the socket. Place the DIMM in a static-safe location. Step 6. Repeat steps 4 and 5 to remove any remaining DIMMs. Place each memory module in a static-safe location. Replacing the Memory Modules CAUTION Observe all ESD safety precautions while completing this procedure.
Installing 14-Slot Shelf Components Installing 14-Slot Shelf Components CAUTION The DIMMs must be replaced in pairs. Each individual DIMM must be identical in storage capacity, revision, and part number. The current DIMM configurations supported by HP include any of the following: • Two 1–GB DIMMs (2 GB) • Four 1–GB DIMMs (4 GB) • Two 2–GB DIMMs (4 GB) • Four 2–GB DIMMs (8 GB) IMPORTANT DIMM sockets U5 and U12 are always populated. See Figure 5-3.
Installing 14-Slot Shelf Components Installing 14-Slot Shelf Components If loading one DIMM pair only, begin by loading the selected DIMM into DIMM socket U12 first. This will ease the installation of the second DIMM into DIMM socket U5. If you are populating all four DIMM sockets, load DIMM socket U17 first. Load DIMM socket U15 next before loading DIMM socket U12 and finish by loading DIMM socket U5 last. Step 1. Angle a selected DIMM over an empty DIMM socket. Step 2.
6 14-Slot Shelf Startup Chapter 6 47
14-Slot Shelf Startup 14-Slot Shelf Startup 14-Slot Shelf Startup This chapter provides information required to: • Access the local serial console ports of the HP bh5700 ATCA 14-Slot Blade Server. • Determine firmware release versions for the major components in the 14-Slot Shelf. • Validate and update the 14-Slot Shelf firmware. • Choose and install an operating system on the HP bc2100 ATCA Server Blades in your HP bh5700 ATCA 14-Slot Blade Server.
14-Slot Shelf Startup 14-Slot Shelf Startup Figure 6-2 RJ-45/DB-9 Adapter Pin Assignments The Shelf Manager (ShMM) Serial Ports A serial RJ-45 port to each ShMM is located on the Alarm Display Panel. The Alarm Display Panel is located on the upper-left front of the 14-Slot Shelf, and contains one serial port for the left ShMM and a second port for the right ShMM (see Figure 6-3). The RJ-45/DB-9 adapter cable connects into these ports.
14-Slot Shelf Startup 14-Slot Shelf Startup Figure 6-4 HP bc2100 ATCA Server Blade Serial Port The Ethernet Switch Blade Serial Ports A separate RJ-45 serial port is provided for both base and fabric access on the front panel of each Ethernet Switch Blade. As shown in Figure 6-5 on page 51, the base serial port is the first port at the top of the base port series, and the fabric serial port is the fist port at the top of the fabric port series.
14-Slot Shelf Startup 14-Slot Shelf Startup Figure 6-5 Chapter 6 Ethernet Switch Blade Serial Ports 51
14-Slot Shelf Startup 14-Slot Shelf Startup Determining Firmware Versions Before beginning normal operation with your HP bh5700 ATCA 14-Slot Blade Server, you should determine the version of the firmware currently installed within each major component of your 14-Slot Shelf, including each ShMM, each HP bc2100 ATCA Server Blade, and each Ethernet Switch Blade.
14-Slot Shelf Startup 14-Slot Shelf Startup Step 2. Connect your PC or laptop computer to the Server Blade serial port, activate a terminal emulator (9600 baud, 8 data bits, no parity, 1 stop bit) on your PC or laptop computer, and access the Server Blade console screen. Step 3. With the console screen displayed, cause the Server Blade to reboot. To cause a reboot you can either cycle power to the 14-Slot Shelf, or remove and reinstall (following hot swap procedures) the respective Server Blade.
14-Slot Shelf Startup 14-Slot Shelf Startup [ZX6000-OA3.2.2j]# zreg -n -v VP • For ZMON(/Sparky) firmware (repeat for both base and fabric switch components): [ZX6000-OA3.2.2j]# zreg -n -v VZ 4.45 • For OpenArchitect firmware (repeat for base and fabric component, and note you can only display the active or booted revision, not the alternate version or flash): [ZX6000-OA3.2.2j]# zreg -n -v V6 3.2.2 build j Step 5.
14-Slot Shelf Startup 14-Slot Shelf Startup Red Hat RHEL AS, Version 4, Update 3 (HP certified on the HP bh5700 ATCA 14-Slot Blade Server) Available directly from Red Hat. For information about additional versions of Red Hat that have been certified on ATCA by HP since publication of this document, see the HP Advanced Telecom Computing Architecture (HP ATCA) - Support Notes for Installing and Using Red Hat, available from HP at: http://www.docs.hp.com.
14-Slot Shelf Startup 14-Slot Shelf Startup Table 6-1 Default rc Script VLAN Configuration (Continued) Section IP Address VLAN Ports Fabric 10.0.1.42 zre 0 to zre 50 (zre 51 reserved for interswitch link) NOTE A sample alternative Ethernet Switch Blade VLAN configuration is provided in Chapter 6. It allows you to reconfigure your default VLAN topology into a more extensive and nontrivial bootup VLAN topology if desired. Additional examples are also located on the Ethernet Switch Blade at /etc/rcZ.
14-Slot Shelf Startup 14-Slot Shelf Startup To determine the most recent release of the firmware for your 14-Slot Shelf and its components, complete the following: Step 1. Ensure that power is applied to the 14-Slot Shelf. Step 2. Connect your PC or laptop computer to the Server Blade serial port, activate a terminal emulator (9600 baud, 8 data bits, no parity, 1 stop bit) on your PC or laptop computer, and access the Server Blade console screen. Step 3.
14-Slot Shelf Startup 14-Slot Shelf Startup 58 Chapter 6
7 A Sample VLAN Topology Chapter 7 59
A Sample VLAN Topology A Sample VLAN Topology A Sample VLAN Topology The HP bh5700 ATCA 14-Slot Blade Server (Figure 7-1) is a highly configurable ATCA blade enclosure. The default factory configuration of the 14-Slot Shelf components provides only a limited network interface because the individual components (Shelf Managers, Ethernet Switch Blades, and HP bc2100 ATCA Server Blades) do not come preconfigured as a production network topology.
A Sample VLAN Topology A Sample VLAN Topology CAUTION Failure to ensure proper configuration of the 14-Slot Shelf virtual local area network (VLAN) topology before connecting the 14-Slot Shelf to a functioning network can cause network failure or loss of data. Ensure that your HP bh5700 ATCA 14-Slot Blade Server is configured with a VLAN topology that is compatible with your existing network before connecting the shelf.
A Sample VLAN Topology A Sample VLAN Topology Figure 7-2 62 Ethernet Switch Blade Front Panel Chapter 7
A Sample VLAN Topology A Sample VLAN Topology In the simplest terms, a VLAN is a user-defined set of these ports, grouped together such that they live in the same subnet. A VLAN can be given an IP address that serves as an endpoint (or destination) for outside traffic, or it may live anonymously within the Switch Blade and rely on established switch routing to send or receive traffic on an anonymous VLAN.
A Sample VLAN Topology A Sample VLAN Topology This topology also allows you to hide some functionality of the ShMM behind the Switch Blades. This means that blades within the 14-Slot Shelf and devices outside of the 14-Slot Shelf are not allowed to connect directly to the ShMMs. All ShMM management traffic (SNMP / IPMI) is routed through the Switch Blades via one of these private LANs, and onto the externally-exposed management LAN.
A Sample VLAN Topology A Sample VLAN Topology • vlan5: Combines the fabric interfaces (bearer plane) into a single VLAN. The only connections to the outside enterprise are through VLAN2 (base management LAN), VLAN3 (base control LAN), and VLAN5 (fabric payload LAN). The management network takes up one base interface (zre12). The control network consumes the remaining base interfaces (zre0 to 11, zre14 to 21) and the payload network consumes all fabric interfaces (zre0 to 51 fabric).
A Sample VLAN Topology A Sample VLAN Topology Figure 7-5 RJ-45/DB-9 Console Adapter Cable Shelf Manager Command Line Interface and Basic Commands After connecting your terminal or laptop to a ShMM, login to the ShMM as root. The ShMM command line interface prompt (clia) will appear on your terminal or laptop display. Use the ShMM command line interface to enter the following basic ShMM commands and see how they work: clia help This command gives you a list of all ShMM commands.
A Sample VLAN Topology A Sample VLAN Topology Figure 7-6Identifying Jumpers B141 to B144 Installed on ShMM Carrier Board Jumpers B141, B142, B143, and B144 Step 3. Reinstall the ShMM Carrier Board in the 14-Slot Shelf. The default IP address for eth0 for the ShMM is either 10.0.0.52 (active ShMM) or 10.0.0.53 (backup ShMM). Since you are hiding the ShMMs on their own private network such that they are only known to the Switch Blade, you will move eth0 to the 192.168.0.x network. To move eth0 to the 192.
A Sample VLAN Topology A Sample VLAN Topology clia setlanconfig 1 12 192.168.0.43 This command sets the gateway address of the active and backup ShMMs for eth0. NOTE In the sample network topology, the default gateway of the ShMM must be set to the IP address of one of the Ethernet Switch Blades in the vlan4 subnet. Configuration of vlan4 is covered in the Ethernet Switch Blade configuration section found later in this document. Step 2.
A Sample VLAN Topology A Sample VLAN Topology NOTE The IP address for eth1 is the same on both ShMMs. Each ShMM resides in a private network that is created by the opposite side Ethernet Switch Blade. Step 2. In the /etc/shelfman.conf file of both ShMMs, change the value of the following variables as shown: SWITCH_OVER_TIMEOUT_ON_BROKEN_LINK needs to be set to a value of 120 or above.
A Sample VLAN Topology A Sample VLAN Topology Figure 7-7 Ethernet Switch Blade Front Panel Base Serial Ports Figure 7-8 Ethernet Switch Blade Front Panel Fabric Serial Ports 70 Chapter 7
A Sample VLAN Topology A Sample VLAN Topology IMPORTANT You must use an RJ-45/DB-9 Adapter Cable (supplied with your HP bh5700 ATCA 14-Slot Blade Server) in combination with a modem eliminator cable (with a DB-9 female connector on each end) in order to connect your terminal or laptop computer to a serial port on the 14-Slot Shelf. Serial access (RS-232) is set to 8 data bits, 1 stop bit, no parity, and 9600 baud. After connecting your terminal or laptop to the base serial port, login as root.
A Sample VLAN Topology A Sample VLAN Topology # # net-snmp (or ucd-snmp) persistent data file. # syslocation unknown syscontact unknown sysservices 72 rocommunity public rwcommunity private # # Proxy all requests intended for destination 16394 to the ShMM. # proxy -v1 -cpublic 192.168.0.2 enterprises.16394 # # SMUX peer support (gated) # smuxpeer .1.3.6.1.4.1.4.3.1.4 secret . . .
A Sample VLAN Topology A Sample VLAN Topology This is the OOB eth0 port on the Switch Blade, and you want to use it as a walk-up configuration port. You (or an administrator) are able to walk up to the ATCA 14-Slot Shelf with a laptop computer, plug into this port, get an assigned IP address, and then use telnet to the Switch Blade to perform normal Switch Blade configuration tasks. Using the dhcpd daemon provided on the Switch Blade, you can set up eth0 to be the configuration port.
A Sample VLAN Topology A Sample VLAN Topology # # If you get to here, and you are not starting the daemon, # display usage info, and exit. # [ "$1" != start ] && echo "usage: `basename $0` start | stop" && exit 0 ##################################################################### # # Bring up the OOB port and dedicated DHCP server. # Save the daemon process ID for killing the process later. ifconfig eth0 192.168.1.1 netmask $NETMASK up dhcpd -cf /etc/dhcpd.eth0.conf -pf $DHCPPID eth0# Step 3.
A Sample VLAN Topology A Sample VLAN Topology Step 9. Now that the laptop has an IP address, telnet to the Switch Blade to make all further configuration changes by entering the following: telnet 192.168.1.1 The Out-of-band port is now configured as your walk-up configuration port. IMPORTANT Make the same set of changes to the redundant, failover Ethernet Switch Blade at this time in order to maintain High Availability integrity for the 14-Slot Shelf.
A Sample VLAN Topology A Sample VLAN Topology # Tear down vlan1 that connects the Switch Blade to the # cross-connected ShMM. # ifconfig zhp0 down zconfig zhp0:teardown ;; *) echo "Usage: $0 { start | stop }" exit 1 esac exit 0 Step 3. Issue the zsync command to make the rc script persistent: zsync Step 4. Enter the following commands in order to start up zhp0/vlan1: chmod 755 /etc/rcZ.d/S21cross-shmm /etc/rcZ.d/S21cross-shmm start Step 5.
A Sample VLAN Topology A Sample VLAN Topology Figure 7-9 The zre12 Base Interface, Ethernet Switch Blade Front Panel To build the Management LAN, complete the following steps: Step 1. To build vlan2 and ensure that it is recreated when you reboot the Switch Blade, you must add an rc script. Add the following S22mgmt rc script to the /etc/rcZ.d directory: NOTE In the following script, you will want to provide an IP address for zhp1/vlan2 that is compatible with your management network.
A Sample VLAN Topology A Sample VLAN Topology set -u case "$1" in start) # Build vlan2 that contains zre12 as the only # configured port. # zconfig zhp1: vlan2=zre12 # zre12 egress is front-panel zconfig zre12=untag2 # Assign an IP address to zhp1/vlan2: # ifconfig zhp1 10.35.50.41 netmask 255.0.0.0 up ;; stop) # Tear down vlan2 that contains zre12. # ifconfig zhp1 down zconfig zhp1:teardown ;; *) echo "Usage: $0 { start | stop }" exit 1 esac exit 0 Step 2.
A Sample VLAN Topology A Sample VLAN Topology inet addr:10.35.50.41 Bcast:10.255.255.255 Mask:255.0.0.0 UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:1443377 errors:0 dropped:0 overruns:0 frame:0 TX packets:42 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:100 RX bytes:123946159 (118.2 Mb) TX bytes:1982 (1.9 Kb) Base address:0xc000 Two VLANs have now been defined; one for the cross-connect to the opposite-side ShMM, and another for the management LAN.
A Sample VLAN Topology A Sample VLAN Topology case "$1" in start) # Build vlan3 that contains zre0..11,zrezre14..21. # zconfig zhp2: vlan3=zre0..11,zre14..21 zconfig zre0..11,zre14..21=untag3 # Do not assign an IP address to this zhp2/vlan3. # Just set the state of zhp2/vlan3 to up. # ifconfig zhp2 up ;; stop) # Tear down vlan3 that contains zre0..11,zre14..21. # ifconfig zhp2 down zconfig zhp2:teardown ;; *) echo "Usage: $0 { start | stop }" exit 1 esac exit 0 Step 2.
A Sample VLAN Topology A Sample VLAN Topology zre13 = untag1 zre12 = untag2 zre0..11, zre14..21 = untag3 ifconfig zhp2 zhp2 Link encap:Ethernet HWaddr 00:11:65:09:EC:1A UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:3582 errors:0 dropped:0 overruns:0 frame:0 TX packets:0 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:100 RX bytes:549764 (536.8 Kb) TX bytes:0 (0.
A Sample VLAN Topology A Sample VLAN Topology # /etc/rcZ.d/S24isl # # rc script to build the inter-switch link (ISL) and # connection to the same-side ShMM for the ATCA shelf. # set -u case "$1" in start) # Build vlan4 that contains zre22..23. # zconfig zhp3: vlan4=zre22..23 zconfig zre22..23=untag4 # Assign an IP address to zhp3/vlan4: # ifconfig zhp3 192.168.0.43 netmask 255.255.255.0 up ;; stop) # Tear down vlan4 that contains zre22..23.
A Sample VLAN Topology A Sample VLAN Topology zhp3: vlan4 = zre22..22 zre13 = untag1 zre12 = untag2 zre0..11, zre14..21 = untag3 zre22..23 = untag4 ifconfig zhp3 zhp3 Link encap:Ethernet HWaddr 00:11:65:09:EC:1B inet addr:192.168.0.43 Bcast:192.168.0.255 Mask:255.255.255.0 UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:2228912 errors:0 dropped:0 overruns:0 frame:0 TX packets:1488770 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:100 RX bytes:156761552 (149.
A Sample VLAN Topology A Sample VLAN Topology Step 3. Enter the following command to verify that the iptables driver loaded properly: lsmod Module iptable_nat bridge if_zxe zfm_c Size 15968 37040 5586552 271160 Used by 0 (unused) 0 (unused) 5 0 Step 4. To set up the iptable rules and ensure that they are reestablished when you reboot the Ethernet Switch Blade, you must add an rc script. Add the following S25iptables rc script to the /etc/rcZ.d directory: #!/bin/sh # # /etc/rcZ.
A Sample VLAN Topology A Sample VLAN Topology SHMM_IP=192.168.0.2 # # Choose a mark your pre-routing DNAT rule. # MARK_DNAT=1 # # Initialize iptables, and flush any existing rules. # iptables -P INPUT ACCEPT iptables –P OUTPUT ACCEPT iptables -P FORWARD DROP iptables -F; iptables -F -t nat; iptables -F -t mangle iptables -X # # Set up some NAT rules to allow rmcp traffic to flow to the # SHMM, and to allow snmp-trap traffic, initiated from the SHMM, # to flow to the designated system manager.
A Sample VLAN Topology A Sample VLAN Topology iptables -t nat -A PREROUTING -p udp -d $MGMT_IP --dport \ rmcp -j DNAT --to-destination $SHMM_IP # # Change source ip of rcmp packet, so that SHMM can respond to # the correct left or right Switch Blade. # iptables -t nat -A POSTROUTING -p udp -o zhp3 \ -mmark --mark ${MARK_DNAT}/${MARK_DNAT} -j SNAT \ --to-source $ISL_IP:10000-10999 # # Allow snmp-trap traffic, initiated from the SHMM, to go to the # designated system manager.
A Sample VLAN Topology A Sample VLAN Topology # # Everything else coming into the Switch Blade, reject. # iptables -A INPUT -j REJECT # # Allow the following traffic to be forwarded from the Switch Blade. # iptables -A FORWARD -j ACCEPT -m state \ --state ESTABLISHED,RELATED iptables -A FORWARD -j ACCEPT \ -mmark --mark ${MARK_DNAT}/${MARK_DNAT} iptables -A FORWARD -j ACCEPT -i lo # # Everything else that could be forwarded, reject.
A Sample VLAN Topology A Sample VLAN Topology echo "Usage: $0 { start | stop }" exit 1 esac exit 0 NOTE The above script is one method of hiding the ShMM from the broader enterprise network. It is not the only method of achieving this objective, and network administrators may need to modify the script according to their particular network requirements.
A Sample VLAN Topology A Sample VLAN Topology udp dpt:162 0 0 ACCEPT icmp -- * icmp type 8 1442 334K REJECT all -- * reject-with icmp-port-unreachable Chain FORWARD (policy DROP 0 packets, pkts bytes target prot opt in 386 21728 ACCEPT all -- * state RELATED,ESTABLISHED 2 102 ACCEPT all -- * MARK match 0x1/0x1 0 0 ACCEPT all -- lo 0 0 REJECT all -- * reject-with icmp-port-unreachable * 0.0.0.0/0 0.0.0.0/0 * 0.0.0.0/0 0.0.0.0/0 0 bytes) out source * 0.0.0.0/0 destination 0.0.0.0/0 * 0.0.0.0/0 0.
A Sample VLAN Topology A Sample VLAN Topology # /etc/rcZ.d/S50fabric # # rc script to build the fabric interface VLAN for # the ATCA shelf. # set -u case "$1" in start) # Build vlan5 that contains zre0..51 fabric. # zconfig zhp4: vlan5=zre0..51 zconfig zre0..51=untag5 # Do not assign an IP address to this zhp4/vlan5. # Just set the state of zhp4 to up. # ifconfig zhp4 up ;; stop) # Tear down vlan5 that contains zre0..51.
A Sample VLAN Topology A Sample VLAN Topology Step 6. Enter the following commands to start up zhp4/vlan5: chmod 755 /etc/rcZ.d/S50fabric /etc/rcZ.d/S50fabric start Step 7. Enter the following commands to see if zhp4/vlan5 is properly configured: zconfig -a zhp4: vlan5 = zre0..51 zre0..
A Sample VLAN Topology A Sample VLAN Topology Figure 7-10Ethernet Switch Blade Fabric Connectors NOTE Make the same set of changes to the redundant, failover Ethernet Switch Blade at this time in order to maintain High Availability integrity for the 14-Slot Shelf. Sample Network Topology Implementation Now that vlan1 through vlan5 are configured (above in this chapter), you have an implementation of your sample network topology as shown in Figure 7-11 on page 93.
A Sample VLAN Topology A Sample VLAN Topology Figure 7-11 Configured Sample Network Topology Ethernet Switch Blade Port and IPMB Mappings Figure 7-12 on page 94 provides an Ethernet Switch Blade Port and IPMB map to help you understand how the base and fabric zre and IPMB addresses are mapped inside the HP bh5700 ATCA 14-Slot Blade Server.
A Sample VLAN Topology A Sample VLAN Topology Figure 7-12 Port IPMB Mapping Base face = 12 xx 14 15 Base rear = 16 17 18 19 Fabric face = 20 21 22 23 Fabric rear = 44 45 46 47 10Gb = 48 49 50 Configuring the HP bc2100 ATCA Server Blade Ethernet Interface Connectors To connect your HP bc2100 ATCA Server Blades into the sample network topology (created earlier in this chapter), you must configure at least some of the Ethernet interfaces included in the Server Blade.
A Sample VLAN Topology A Sample VLAN Topology IMPORTANT When setting up the eth0 through eth5 network interfaces on your Server Blades, remember that any assigned IP address on these interfaces must reside within the base or fabric network domains that you have already established for the Switch Blades.
A Sample VLAN Topology A Sample VLAN Topology To enable the high-availability (surviving partner) software, you (or an administrator) must manually create a configuration file known as zsp.conf. This file contains instructions to the surviving partner set-up software that runs on the Switch Blade. Included in the config file is a complete description for all of the Switch Blade’s VLANs, their IP addresses, sibling IP addresses, and virtual IP addresses.
A Sample VLAN Topology A Sample VLAN Topology # zhp2 does not have an assigned IP address. # sibling_addresses: zhp1 = 10.35.50.41, 10.35.50.42 netmask 255.0.0.0; sibling_addresses: zhp3 = 192.168.0.43, 192.168.0.44 netmask 255.255.255.0; # # The virtual address spans the sibling group, giving hosts and routers # a single point to connect to, or a single point to use as, a router. A # virtual address should not be specified for the interconnect interface.
A Sample VLAN Topology A Sample VLAN Topology # # Failover Modes: # switch-failover, vlan-failover or port-failover are mutually exclusive. # They describe what occurs if a port fails. For switch-failover, if any port # fails, all functionality of the current switch is moved to the backup # For vlan-failover, if a port fails in the VLAN, then all the ports that # are a member of that VLAN are failed over. For port-failover, each port # can failover independently.
A Sample VLAN Topology A Sample VLAN Topology monitors the network links that are set up for failover operation. The dhcpd daemon process enables the sibling Switch Blade to establish communication with the first Switch Blade to retrieve its surviving partner configuration file. These daemon processes can be seen by running the ps command on the Switch Blade: ps -efw root 19577 /usr/sbin/vrrpd -d 100m -c /etc/rcZ.d/surviving_partner/vrrpd.conf root 19579 /usr/sbin/vrrpd -d 100m -c /etc/rcZ.
A Sample VLAN Topology A Sample VLAN Topology zhp3 B 4 zhp0 100 224 1 192.168.0.44 It is a good idea to reboot both Switch Blades after configuring them for surviving partner mode, in order to ensure that there were are no problems with the installation.
A Sample VLAN Topology A Sample VLAN Topology NOTE You may want to put the above commands into an rc script and have them executed at boot time so that the bonding driver is always enabled. NOTE If you intend to acquire the IP address for bond0 by using DHCP, the physical MAC address that is presented to the DHCP server is the address of the first enslaved physical network interface (eth0 in this example). Step 2. When you have finished creating bond0, issue the ifconfig command on the Server Blade.
A Sample VLAN Topology A Sample VLAN Topology 102 Chapter 7
Index A adapter cable, 48 Advance Telecommunications Computer Architecture, 10 advanced mezzanine card, 10 Advanced Telecommunications Computer Architecture, 10 air filter, 12 alarm connector panel, 12 alarm display panel, 12 AMC, 10 ATCA, 10 ATCA 14-Slot backplane, 12 B bands, poly strap, 18 base ports, 55 blades, 10 C cable tray, front, 12 cable tray, rear, 13, 35 cable, console adapter, 48 card cage, front, 12 card cage, rear, 13, 35 checklist, installation, 14 command line interface, ShMM, 66 commands,
Index serial ports, Ethernet switch blade, 50 Shelf Manager, 12 ShMM Cross-connect LAN, 63 shrink wrap, 18 slide mount, 27 socket, DIMM, 43 switch blade, Ethernet, 50 switch blades, 10 Switch-to-Switch LAN, 63 T terminal, ESD wrist strap, 12, 13, 35 Topology, Sample VLAN, 60 tray, cable, 23 V VDC, 36 VLAN, 55 voltage measure, 36 VRTN, 38 104
