Dell EMC VMware Cloud Foundation 4.0 for PowerEdge Rack Server Deployment Guide May 2020 Rev.
Notes, cautions, and warnings NOTE: A NOTE indicates important information that helps you make better use of your product. CAUTION: A CAUTION indicates either potential damage to hardware or loss of data and tells you how to avoid the problem. WARNING: A WARNING indicates a potential for property damage, personal injury, or death. © 2019 2020 Dell Inc. or its subsidiaries. All rights reserved. Dell, EMC, and other trademarks are trademarks of Dell Inc. or its subsidiaries.
Contents 1 Audience and scope...................................................................................................................... 5 2 Overview..................................................................................................................................... 6 3 Pre-deployment requirements.......................................................................................................8 Management host...................................................................
VMware Cloud Foundation networking.............................................................................................................................19 Network connectivity.......................................................................................................................................................... 19 Networking and NSX-T......................................................................................................................................................
1 Audience and scope This deployment guide includes step-by-step instructions for deployment of VMware Cloud Foundation (VCF) on Dell EMC PowerEdge RX740xd rack servers. Any deviation from the listed configurations may negatively impact functionality. This deployment guide makes certain assumptions about the prerequisite knowledge of the deployment personnel.
2 Overview Deployment of VMware VCF on the PowerEdge R740xd platform provides a hyperconverged infrastructure solution incorporating best-inclass hardware from Dell EMC with core VMware products including vSphere, vSAN, NSX, vRealize Log Insight, and SDDC Manager. Virtualization of compute, storage, and networking is delivered on a single cluster of PowerEdge R740xd servers. Dell EMC has determined the compatibility and established certification across hardware and software.
Figure 1.
3 Pre-deployment requirements Management host The deployment of VMware Cloud Foundation is executed by a Cloud Builder VM that is deployed using an Open Virtualization Appliance (OVA). The virtual machine must be deployed on an ESXi host or cluster that is not a part of the Cloud Foundation cluster. If the management network is a private network ensure that the Cloud Builder VM and the Cloud Foundation management hosts have access to the same DNS and NTP services.
Domain Name Service Domain Name Service (DNS) is required to provide both forward and reverse name resolution. The IP addresses of name servers, search domains, and hostnames of all the VCF VMs must be inserted into the cloud builder deployment parameter sheet. Forward and reverse DNS entries of any hostname that are indicated in the parameter sheet should be tested and retested for both forward and reverse lookups.
4 Validated components Validated components refer to the hardware components, and the software and firmware versions that have been validated. VMware no longer maintains the VMware Compatibility Guide (VCG) for Cloud Foundation. Since VSAN is an underlying requirement of Cloud Foundation, any hardware that is specified as a vSAN Ready Node is approved for Cloud Foundation.
5 Hardware overview This section provides additional information about the hardware platform used in the development of this deployment guide. Topics: • • Dell EMC PowerEdge R740xd server Dell EMC Networking S5248F-ON Switch Dell EMC PowerEdge R740xd server The PowerEdge R740xd server provides the benefit of scalable storage performance and data set processing. This 2U, 2-socket platform brings you scalability and performance to adapt to a variety of applications.
• • 48 10 GbE SFP+ ports Six 40 GbE QSFP+ ports The QSFP28 or QSFP28-DD ports can be used for Ethernet uplink connectivity. For more information, see the Management host section. The SFP28 ports can be used for connections to both Cloud Foundation management and Cloud Foundation workload domains. Figure 5.
6 Physical layout The configuration of Cloud Foundation on PowerEdge R740xd is described in this document. The Cloud Foundation software addresses the host servers using their IP Address. The physical layout and resulting cabling are determined by the number of R740xd servers and the number of available ports on the top-of-rack switches. Cloud Foundation 4.0 incorporates NSX-T where the 3.x versions of Cloud Foundation incorporate NSX. The move from NSX to NSX-T has an impact on the network topology.
Figure 6. Management Host Network Cabling Top of Rack Connectivity to Spine The uplinks from the Top of Rack Leaf switches include separate uplinks for support of NSX-T edge node networking. To accommodate all the connections, we can utilize ports 51 and 52 on the S5248F-ON switches. These ports are 200GbE Double Density ports.
7 VCF and SDDC design considerations VMware Cloud Foundation relies on a set of key infrastructure services to be made available externally. You must configure these external services before you begin deployment. NOTE: This section is universal for VCF deployments regardless of hardware platform. The content in this section is also available in the VMware Cloud Foundation Planning and Preparation Guide, and is included here for reference.
Dynamic Host Configuration Protocol Cloud Foundation uses Dynamic Host Configuration Protocol (DHCP) to automatically configure each VM kernel port of an ESXi host that is used as a VTEP with an IPv4 address. One DHCP scope must be defined and made available for this purpose. The DHCP scope that is defined must be large enough to accommodate all the initial and future servers that are used in the Cloud Foundation solution. Each host requires two IP addresses, one for each VTEP configured.
Cloud Foundation uses an Internet Protocol Address Management (IPAM) solution to automate the IP configuration of VM kernel ports for vMotion, vSAN, and NFS (depending on the storage type being used). When a server is added to the inventory of Cloud Foundation, it goes through a process called host commissioning. During this process, the hosts are associated with an existing network pool.
Allocate hostnames and IP addresses to the following components: • • • • NTP AD DNS Certificate Authority (CA) The following table provides sample information for the external services. This example uses a DNS domain called osevcf18.local for illustration purposes only. Modify the sample information to conform to the configuration of your site. Table 4. Configuration for external services Component Group Hostname DNS IP Address Description DNS dc01sfo sfo01.osevcf18.local 172.18.11.
8 Networking requirements This section covers the networking requirements from both the Cloud Foundation software perspective and from a networking hardware connectivity perspective using the Dell EMC Networking S5248F-ON as an example. Topics: • • • VMware Cloud Foundation networking Network connectivity Networking and NSX-T VMware Cloud Foundation networking A successful VMware Cloud Foundation deployment relies heavily on networks that are constructed and allocated to Cloud Foundation.
Deploying Cloud Foundation on the R740xd servers and S5248F-ON switches follows the network connectivity as shown in Figure 6. Management Host Network Cabling. Networking and NSX-T Version 4.0 of Cloud Foundation changes to NSX-T from NSX or NSX-V in Cloud Foundation 3.9. The move to NSX-T drives changes in the physical and logical networking architecture. NSX-T introduces the concept of an Edge Node Cluster.
9 Manual switch configuration This section describes the configurations that are made on the S5248F-ON switches. The configuration information includes two Dell EMC Networking S5248F-ON switches. These switches get the same configuration except for the hostname, IP address, the VLT Backup Destination IP Address and the NSX-specific uplink VLANs. The management IP address of each switch was initially assigned using a serial console. .
NSX switch specific VLANs These VLANs are switch-specific and should not be created on both switches. On the first switch, we create VLAN 2711 and on the second switch, create VLAN 2712 on the second switch. On switch one: interface vlan2711 description 2711-NSX no shutdown mtu 9216 On switch two: interface vlan2712 description 2712-NSX no shutdown mtu 9216 NOTE: These VLANs are created on different Top of Rack switches.
Verify VLT settings Verify the VLT settings by running the following command on each switch: r10-24-s5248# show vlt 10 Domain ID Unit ID Role Version Local System MAC address Role priority VLT MAC address IP address Delay-Restore timer Peer-Routing Peer-Routing-Timeout timer VLTi Link Status port-channel1000 : : : : : : : : : : : 10 2 secondary 2.
switchport trunk allowed vlan 96,1811-1813,1815 mtu 9216 vlt-port-channel 1 r10-24-s5248# show running-configuration interface port-channel 11 ! interface port-channel11 description NSX_Uplink_1 no shutdown switchport mode trunk switchport trunk allowed vlan 2711,1814 mtu 9216 r10-24-s5248# show port-channel summary Flags: D - Down I - member up but inactive P - member up and active U - Up (port-channel) F - Fallback Activated -------------------------------------------------------------------------------G
On switch two: S5248-top(config)# S5248-top(config)# interface range ethernet 1/1/1-1/1/16 S5248-top(conf-range-eth1/1/1-1/1/16)# switchport mode trunk S5248-top(conf-range-eth1/1/1-1/1/16)# switchport trunk allowed vlan 96,1811-1815,2712 S5248-top(conf-range-eth1/1/1-1/1/16)# mtu 9216 S5248-top(conf-range-eth1/1/1-1/1/16)# no shutdown S5248-top(conf-range-eth1/1/1-1/1/16)# exit S5248-top(config)# Save your switch configuration on each S4048-ON switch by running the following command: ToR-Top-220#write mem
10 Deploy ESXi to cluster nodes Perform the following steps to install VMware ESXi on each of the PowerEdge R740xd hosts that are part of the management cluster. This guide covers the steps to install VMware ESXi remotely using iDRAC Virtual Console with Virtual Media. In this example, a static IP address is assigned to the management interface of the ESXi hosts, which is required for Cloud Foundation.
10. From the Power menu, click Power on System. 11. If the system is not turned on, click Power on System. If the system is ON, click Power Cycle System (cold boot). The server is connected to the iDRAC devices and boots into the ESXi installer. Install VMware ESXi Before installing VMware ESXi, you must connect to the iDRAC devices and boot into the ESXi installer. For more information, see Connect to iDRAC and boot installation media. 1. In the Welcome to ESXi Installation window, press Enter. 2.
Figure 9. IPv4 configuration page 8. 9. 10. 11. 12. Enter the IPv4 Address,Subnet Mask, and the Default Gateway, and then press Enter to confirm. Select DNS Configuration, and then press Enter. Enter the IP addresses of the DNS servers and FQDN of the host. Press Esc to return to the main menu, and then press Y to confirm the changes and restart the management network. From the main menu, click Test Management Network. The target IP addresses and DNS hostname are pre-populated. 13.
Figure 10. ESXi web interface—Edit settings page 5. In the Edit Port Group window, enter the Management VLAN ID, and then click Save. CAUTION: Leaving the VLAN ID at default setting causes pre-deployment validation to fail during a later step. 6. 7. 8. 9. In the Navigator pane, click Manage to set up the NTP. In the right pane, click Time & Date. Click Edit Settings and then select Use Network Time Protocol (enable NTP client). In the NTP Servers box, enter the NTP server IP addresses.
Figure 11. ESXi web interface—Edit time configuration page 11. In the Manage pane, select the Services tab and right click on the ntpd service. Set the Policy to Start and stop with the host. The resulting page is as shown in the following figure: Figure 12. ESXi settings web interface—Manage pane Once the policy is set, start the ntpd service. If the ntpd service is already running, restart the service. Repeat all the steps for each host targeted for Cloud Foundation management domain deployment.
11 Cloud Builder and SDDC deployment The primary software installation tool for Cloud Foundation 4.x is Cloud Builder. It is delivered as a virtual appliance in the standard OVA format. This section describes the steps to deploy the OVA. The Cloud Builder VM is a temporary tool to facilitate deployment of Cloud Foundation. It can be discarded after the deployment.
Figure 13. OVF customize template page 11. Review the Ready to Complete final configuration page, and then click Finish. 12. In the Recent Tasks pane, check the OVA deployment status. When the OVA deployment is complete, turn on the Cloud Builder VM. Check Time Synchronization After the Cloud Builder VM is started it will take some time to for all of the services to start and for time synchronization to complete.
12 VCF Deployment using Cloud Builder In the previous section, you deployed the Cloud Builder virtual appliance. In this section, the software within the virtual machine is used to validate the target environment and deploy the entire Cloud Foundation stack. NOTE: Before proceeding with the Cloud Builder validation process, take a snapshot of your Cloud Builder VM.
Figure 14. Cloud Builder web interface 3. 4. 5. 6. Log in using the credentials that you specified during OVA deployment. Click Check All to review the checklist of pre-bring-up steps and confirm that all the steps that are completed, and then click Next. Review the EULA, and if you agree, click Agree to End User License Agreement, and then click Next. If you have not obtained and completed the Cloud Foundation Information Spreadsheet, click Download Deployment Parameter Sheet.
Management Workload tab License keys are required for the following items: • • • • • ESXi hosts vSAN vCenter NSX-T SDDC Manager Appliance Users and Groups tab In the Users and Groups tab, you can set the passwords for your initial Cloud Foundation components. CAUTION: Do not make a mistake on this page because if any of the passwords do not meet the indicated specifications, you must redeploy your Cloud Builder VM, unless you elected to create a snapshot after you created your VM.
8. Place a checkmark to indicate that the prerequisites have been met and select Next 9. If you have not obtained and completed the Cloud Foundation Information Spreadsheet, click Download Deployment Parameter Sheet. If you have the parameter sheet click Next to move forward 10. If you have not completed your parameter sheet do it now. If the parameter sheet is complete, click Next to continue. 11. Click Select File to browse to your completed parameter sheet. 12.
Figure 16. Configure Cloud Builder validation NOTE: Validation may take 15 minutes or more. However, if there are issues such as the DNS server being down or if you provided a wrong IP address, validation may take longer. NOTE: On the Validation Report page, you can access the information about previous validation attempts. Each validation attempt is tracked with an entry that is designated by the date and time of execution. 3.
13 Post-install validation Cloud Foundation cluster verification After installing Cloud Foundation, perform the steps in the following sections to verify that the components are installed and available. SDDC Manager Log in to SDDC Manager using a web browser at: https://. The SSO user ID is administrator@vsphere.local and the password is the one you specified during installation. NOTE: Use the domain vsphere.local.
Figure 19. vCenter dashboard NSX Manager Log into the NSX Manager through your browser using the admin credentials set in your parameter sheet. Figure 20. NSX dashboard VMware Cloud Foundation installation complete Cloud Foundation has been successfully deployed and is ready for use. Typical tasks from this point would be: • • • Configure your VMware account credentials in SDDC manager. If you chose simplified networking configure your SDDC for VLAN backed Application Virtual Networks.
