Book.
Book.book Page 2 Tuesday, July 26, 2011 11:33 PM Notes, Cautions, and Warnings NOTE: A NOTE indicates important information that helps you make better use of your computer. CAUTION: A CAUTION indicates potential damage to hardware or loss of data if instructions are not followed. WARNING: A WARNING indicates a potential for property damage, personal injury, or death. _______________ Information in this publication is subject to change without notice. © 2011 Dell Inc. All rights reserved.
Book.book Page 3 Tuesday, July 26, 2011 11:33 PM Contents 1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . Before You Begin . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 Network Requirements . . . . . . . . . . . . . . . . 5 . . . . . . . . . . 6 . . . . . . . . . . . 7 Attaching to RHN/ULN Repository . . . . . . . . . . . . . 7 Installing the Dell Validated RPM . . . . . . . . . . . . . 9 Preparing Nodes for Oracle Installation . . . . . . . . .
Book.book Page 4 Tuesday, July 26, 2011 11:33 PM Primary Partition . . . . . . . . . . . . . . . . . . . . . Installing and Configuring ASMlib . . . . . . . . . . . Using ASMLib to Mark the Shared Disks as Candidate Disks . . . . . . . . . . . . 4 25 . . . . . . . . . 29 . . . . . . . . . . . . . . . . . . . . 29 Configure the System Clock Settings for All Nodes . . . . . . . . . . . . . Configuring Node One 5 6 7 8 4 . . . . . . . 29 . . . . . . . . . . . . . . . . . .
Book.book Page 5 Tuesday, July 26, 2011 11:33 PM 1 Overview This document applies to Oracle Database 11g R2 running on Red Hat Enterprise Linux 5.x AS x86_64 or Oracle Enterprise Linux 5.x AS x86_64. Before You Begin Hardware Requirements • Oracle requires 1.5 gigabytes (GB) of physical memory. • Swap space must be equal to the amount of RAM allocated to the system. • Oracle's temporary space (/tmp) must be at least 1 GB in size.
Book.book Page 6 Tuesday, July 26, 2011 11:33 PM • All public interfaces for each node should be able to communicate with all nodes within the cluster. • All private interfaces for each node should be able to communicate with all nodes within the cluster. • The hostname of each node must follow the RFC 952 standard (www.ietf.org/rfc/rfc952.txt). Hostnames that include an underscore ("_") are not permitted.
Book.book Page 7 Tuesday, July 26, 2011 11:33 PM Preparing Nodes for Oracle Installation 2 Attaching to RHN/ULN Repository NOTE: The documentation provided below discusses how to set up a local yum repository using your operating system installation media. If you would like to connect to the Red Hat Network (RHN)/Unbreakable Linux Network (ULN) channels, see the appropriate documentation. For Red Hat, see redhat.com/red_hat_network. For information relating to ULN network, see linux.oracle.com.
Book.book Page 8 Tuesday, July 26, 2011 11:33 PM 2 To install and configure the http daemon, configure the machine that will host the repository for all other machines to use the DVD image locally. Create the file /etc/yum.repos.d/local.
Book.book Page 9 Tuesday, July 26, 2011 11:33 PM The following is an example of the configuration using the RHEL 5.x Server media in the configuration file /etc/yum.repos.d/myRepo.repo [myRepo] name=RHEL5.5 DVD baseurl= http://reposerver.mydomain.com/RHEL5_5/Server enabled=1 gpgcheck=0 NOTE: Replace reposerver.mydomain.com with your server's DNS name or IP address.
Book.book Page 10 Tuesday, July 26, 2011 11:33 PM NOTE: Where, o is the operating system version, y is the year, and m is the month of the tar release.
Book.book Page 11 Tuesday, July 26, 2011 11:33 PM 4 To Untar the Dell-Oracle-Deployment release type the following command: # tar -xvf Dell-Oracle-Deployment-o-y-m.tar NOTE: Where, o is the operating system version, y is the year, and m is the month of the tar release.
Book.book Page 12 Tuesday, July 26, 2011 11:33 PM NOTE: Ensure that the Gateway address is configured for the public network interface. If the Gateway address is not configured, the Oracle Grid installation may fail. DEVICE=eth0 ONBOOT=yes IPADDR= NETMASK= BOOTPROTO=static HWADDR= SLAVE=no GATEWAY= 3 Edit the /etc/sysconfig/network file, and, if necessary, replace localhost.localdomain with the qualified public node name.
Book.book Page 13 Tuesday, July 26, 2011 11:33 PM private interconnect connections. These private network adapters can be configured during the initial install process of Oracle Grid or after the installation process using the oifcfg utility. Oracle Grid currently creates an alias IP (as known as virtual private IP) on your private network adapters using the 169.254.*.* subnet for the HAIP. If the subnet range is already in use, Oracle Grid does not attempt to use it.
Book.book Page 14 Tuesday, July 26, 2011 11:33 PM NETMASK=255.255.255.0 2 Once you have saved both the configuration files, restart your network service using service network restart. The completion of the steps above have now prepared your system to enable HAIP using the Oracle Grid Infrastructure installer. When you have completed all the Oracle prerequisites and are ready to install Oracle, you will need to select eth1 and eth2 as 'private' interfaces at the 'Network Interface Usage' screen.
Book.book Page 15 Tuesday, July 26, 2011 11:33 PM eth1:1 Link encap:Ethernet 00:1E:C9:4B:72:22 HWaddr inet addr:169.254.167.163 Bcast:169.254.255.255 Mask:255.255.0.0 UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 eth2 Link encap:Ethernet 00:1E:C9:4B:71:24 HWaddr inet addr:192.168.0.141 Bcast:192.168.0.255 Mask:255.255.255.
Book.book Page 16 Tuesday, July 26, 2011 11:33 PM IP Address and Name Resolution Requirements The steps below show how to set up your cluster nodes for using Domain Name System (DNS). For information on how to set up cluster nodes using GNS, see the Wiki article http://en.community.dell.com/dellgroups/enterprise_solutions/w/oracle_solutions/1416.aspx.
Book.book Page 17 Tuesday, July 26, 2011 11:33 PM Table 2-1 describes the different interfaces, IP address settings and the resolutions in a cluster. Table 2-1. Cluster Requirements for DNS Interface Type Resolution Public Static DNS Private Static Not required Node virtual IP Static DNS SCAN virtual IP Static DNS Configuring a DNS Server To configure changes on a DNS server for an Oracle 11g R2 cluster using a DNS (without GNS): 1 Configure SCAN NAME resolution on DNS server.
Book.book Page 18 Tuesday, July 26, 2011 11:33 PM nameserver 192.0.2.100 Where, 192.0.2.100 is a valid DNS server address in your network and ns1.domainserver.com is the domain server in your network. 2 Verify the order configuration. /etc/nsswitch.conf controls the name service order. In some configurations, the NIS can cause issues with Oracle SCAN address resolution. It is recommended that you place the NIS entry at the end of the search list and place the dns entry first.
Book.book Page 19 Tuesday, July 26, 2011 11:33 PM Preparing Shared Storage for Oracle RAC Installation 3 NOTE: In this section, the terms disk(s), volume(s), virtual disk(s), LUN(s) mean the same and are used interchangeably, unless specified otherwise. Similarly, the terms Stripe Element Size and Segment Size both can be used interchangeably.
Book.book Page 20 Tuesday, July 26, 2011 11:33 PM CAUTION: In a system running the Linux operating system, align the disk prior to being written to the Volume/Virtual Disk (VD). Failure to do so will cause all data on the disk to be destroyed.
Book.book Page 21 Tuesday, July 26, 2011 11:33 PM of data residing on each disk in the stripe. Here the stripe element size is 64 KB and the stripe size is 256 KB. Use the following formula to set the value above: Stripe Element Size in Sectors = Stripe Element Size in KB * 2 First Sector = Stripe Element Size in Sectors NOTE: The above formula assumes that 1 Sector = 512 Bytes or 0.5 KB.
Book.book Page 22 Tuesday, July 26, 2011 11:33 PM #> service multipathd restart Or #> kpartx –a /dev/mapper/ 4 Verify that the partition has been aligned by running the one of the following command: • #> fdisk –ul /dev/ • #> fdisk –ul /dev/mapper/ Where, or is the name of the disk whose partition was aligned. The following example is of a sample output of the above command on a block device that has been aligned.
Book.book Page 23 Tuesday, July 26, 2011 11:33 PM CAUTION: In a system running the Linux operating system, align the partition table before the data is written to the Volume. Failure to follow the precaution could lead to all the data on the Volume to be destroyed. At the command prompt, type: 1 #> fdisk -u /dev/ where is the name of the block device that you are adjusting. For example, if the block device is /dev/mapper/db, type: fdisk /dev/mapper/db.
Book.book Page 24 Tuesday, July 26, 2011 11:33 PM 3 Run the following command to re-scan all the partitions on node one when using device mapper: #> kpartx -a /dev/mapper/ On all other nodes run: #> kpartx -l /dev/mapper/devicename> NOTE: If your device name does not end in "p1", reboot your system. Proper name convention would append a "p1" and display as /dev/mapper/ACFSp1.
Book.book Page 25 Tuesday, July 26, 2011 11:33 PM Installing and Configuring ASMlib 1 Use oracle.com/technetwork/server-storage/linux/downloads/rhel5084877.html to download the following files: • oracleasm-support • oracleasmlib • oracleasm NOTE: If your current OS distribution is Oracle Linux, you can obtain the software from the Unbreakable Linux Network using ULN.
Book.book Page 26 Tuesday, July 26, 2011 11:33 PM Fix permissions of Oracle ASM disks on boot (y/n) [y]: y NOTE: In this setup the default user is set to grid and the default group is set to asmadmin. Ensure that the oracle user is part of the asmadmin group. You can do so by using the dell-validated and dell-oracle-utilities rpms. The boot time parameters of the Oracle ASM library are configured and a sequential text interface configuration method is displayed.
Book.book Page 27 Tuesday, July 26, 2011 11:33 PM All the instances of DISKNAME from the previous command(s) are displayed. To delete an ASM disk, run the following command: /usr/sbin/oracleasm deletedisk DISKNAME 6 To discover the Oracle ASM disks on other nodes in the cluster, run the following command on the remaining cluster nodes: /usr/sbin/oracleasm scandisks.
Book.
Book.book Page 29 Tuesday, July 26, 2011 11:33 PM Installing Oracle 11g R2 Grid Infrastructure 4 This section gives you the installation information for Oracle 11g R2 grid infrastructure for a cluster. Before You Begin Before you install the Oracle 11g R2 RAC software on your system: • Ensure that you have already configured your operating system, network, and storage based on the steps from the previous sections within this document. • Locate your Oracle 11g R2 media kit.
Book.book Page 30 Tuesday, July 26, 2011 11:33 PM Configuring Node One The following steps are for node one of your cluster environment, unless otherwise specified. 1 Log in as root. 2 If you are not in a graphical environment, start the X Window System by typing: startx 3 Open a terminal window and type: xhost + 4 Mount the Oracle Grid Infrastructure media. 5 Log in as grid user, for example: su - grid.
Book.book Page 31 Tuesday, July 26, 2011 11:33 PM 12 In the Cluster Node Information window, click Add to add additional nodes that must be managed by the Oracle Grid Infrastructure. • Enter the public Hostname information • Enter the Virtual IP name • Repeat step 12 for each node within your cluster 13 Click SSH Connectivity and configure your password less SSH connectivity by entering the OS Password for the grid user and click Setup.
Book.book Page 32 Tuesday, July 26, 2011 11:33 PM 20 In the Privileged Operating Systems Groups window, select: • asmdba for Oracle ASM DBA (OSDBA for ASM) Group • asmoper for Oracle ASM Operator (OAOPER for ASM) Group • asmadmin for Oracle ASM Administrator (OSASM) Group 21 In the Installation Location window, specify the values of your Oracle Base and Software Location as configured within the Dell Oracle utilities RPM.
Book.book Page 33 Tuesday, July 26, 2011 11:33 PM 5 Configuring Shared Oracle Home for Database Binary Using ACFS The following steps are applicable for node one of your cluster environment, unless otherwise specified: 1 Log in as root and type: xhost + 2 Log in as grid user and run the asmca utility by typing: $/bin/asmca 3 In the ASM Configuration Assistant window, select the Disk Groups tab, click Create, and perform the following steps: • Enter a name of the disk group.
Book.book Page 34 Tuesday, July 26, 2011 11:33 PM • Click Ok. 7 As root, run the acfs_script.sh mentioned in the RUN ACFS Script window. This automounts the new ACFS Home on all nodes. 8 Click Close to exit ACFS script window.
Book.book Page 35 Tuesday, July 26, 2011 11:33 PM 6 Installing Oracle 11g R2 Database (RDBMS) Software The following steps are for node one of your cluster environment, unless otherwise specified. 1 Log in as root and type: xhost +. 2 Mount the Oracle Database 11g R2 media.
Book.book Page 36 Tuesday, July 26, 2011 11:33 PM 11 In the Installation Location window, • Specify the location of your Oracle Base configured within the Dell oracle utilities RPM. • Enter the ACFS shared Oracle home address for Software Location. NOTE: The default locations used within the Dell Oracle utilities RPM are as follows: • Oracle Base—/u01/app/oracle. • Software Location—/u01/app/oracle/product/11.2.0/db_1.
Book.book Page 37 Tuesday, July 26, 2011 11:33 PM 7 Creating Diskgroup Using ASM Configuration Assistant (ASMCA) This section contains procedures to create the ASM disk group for the database files and Flashback Recovery Area (FRA). 1 Log in as grid user. 2 Start the ASMCA utility by typing: $/bin/asmca 3 In the ASM Configuration Assistant window, select the Disk Groups tab. 4 Click Create. 5 Enter the appropriate Disk Group Name, for example: DBDG. 6 Select External for Redundancy.
Book.
Book.book Page 39 Tuesday, July 26, 2011 11:33 PM Creating Database Using DBCA 8 The following steps are applicable for node one of your cluster environment, unless otherwise specified: 1 Log in as oracle user. 2 From $, run the DBCA utility by typing: $/bin/dbca & 3 In the Welcome window, select Oracle Real Application Cluster Database and click Next. 4 In the Operations window, select Create Database, and click Next.
Book.book Page 40 Tuesday, July 26, 2011 11:33 PM 10 In the Recovery Configuration window: a Select Specify Flash Recovery Area. b Browse and select the ASM disk group that you created for Flash Recovery Area. c Enter a value for Flash Recovery Area Size. d Select Enable Archiving. e Click Next. 11 In the Database Content window, click Next. 12 In the Initialization Parameters window: • Select Custom. • For the Memory Management section, select Automatic Shared Memory Management.
