Veritas Storage Foundation 5.1 SP1 for Oracle RAC Release Notes HP-UX 11i v3 HP Part Number: 5900-1513 Published: April 2011 Edition: 1.
© Copyright 2011 Hewlett-Packard Development Company, L.P. Legal Notices Confidential computer software. Valid license from HP required for possession, use or copying. Consistent with FAR 12.211 and 12.212, Commercial Computer Software, Computer Software Documentation and Technical Data for Commercial Items are licensed to the U.S. Government under vendor’s standard commercial license. The information contained herein is subject to change without notice.
Storage Foundation for Oracle RAC Release Notes This document includes the following topics: ■ About this document ■ Component product release notes ■ About Veritas Storage Foundation for Oracle RAC ■ About Symantec Operations Readiness Tools ■ Important release information ■ Changes introduced in version 5.
8 Storage Foundation for Oracle RAC Release Notes About this document About this document This document provides important information about Veritas Storage Foundation for Oracle RAC (SF Oracle RAC) version 5.1 Service Pack 1 for HP-UX 11i v3. Review this entire document before you install or upgrade SF Oracle RAC. The information in the Release Notes supersedes the information provided in the product documents for SF Oracle RAC. This is Document version: 5.1SP1.
Storage Foundation for Oracle RAC Release Notes About Veritas Storage Foundation for Oracle RAC The solution stack comprises the Veritas Cluster Server (VCS), Veritas Cluster Volume Manager (CVM), Veritas Oracle Real Application Cluster Support (VRTSdbac), Veritas Oracle Disk Manager (VRTSodm), Veritas Cluster File System (CFS), and Veritas Storage Foundation, which includes the base Veritas Volume Manager (VxVM) and Veritas File System (VxFS).
10 Storage Foundation for Oracle RAC Release Notes About Symantec Operations Readiness Tools ■ Enhanced scalability and availability with access to multiple Oracle RAC instances per database in a cluster. ■ Support for backup and recovery solutions using volume-level and file system-level snapshot technologies. SF Oracle RAC enables full volume-level snapshots for off-host processing and file system-level snapshots for efficient backup and rollback.
Storage Foundation for Oracle RAC Release Notes Important release information ■ Collect, analyze, and report on server configurations across UNIX or Windows environments.
12 Storage Foundation for Oracle RAC Release Notes Changes introduced in version 5.1 Service Pack 1 ■ For important updates regarding this release, review the Late-Breaking News TechNote on the Symantec Technical Support website: http://www.symantec.com/docs/TECH144835 ■ For the latest patches available for this release, go to: http://sort.symantec.com/ Changes introduced in version 5.1 Service Pack 1 This section describes the new features and changes in version 5.1 SP1.
Storage Foundation for Oracle RAC Release Notes Changes introduced in version 5.1 Service Pack 1 See the Veritas Storage Foundation for Oracle RAC Installation and Configuration Guide. The installer can automatically detect and configure LLT links The installer detects link connection status among all cluster nodes and chooses the most suitable links for LLT communication. It then can set the priority of the LLT private heartbeat links based on their media speed. Aggregated and bonded NICs are supported.
14 Storage Foundation for Oracle RAC Release Notes Changes introduced in version 5.1 Service Pack 1 Changes in the SF Oracle RAC installer This section lists the changes in the SF Oracle RAC installer. Option to start and stop SF Oracle RAC You can use the -start and -stop options with the SF Oracle RAC installer to start and stop SF Oracle RAC.
Storage Foundation for Oracle RAC Release Notes Changes introduced in version 5.1 Service Pack 1 To view the list of depots for this option, use -minpkgs option with the installer script. ■ Recommended Installs SF Oracle RAC without optional depots. To view the list of depots for this option, use -recpkgs option with the installer script. ■ All Installs all SF Oracle RAC depots. To view the list of depots for this option, use -allpkgs option with the installer script.
16 Storage Foundation for Oracle RAC Release Notes Changes introduced in version 5.1 Service Pack 1 ■ Updating the network configuration Installation simulator You can use the option -makeresponsefile with the installer script to simulate an installation, configuration, or uninstallation activity. Oracle RAC installation The SF Oracle RAC installer includes several enhancements for performing Oracle RAC pre-installation tasks.
Storage Foundation for Oracle RAC Release Notes Changes introduced in version 5.1 Service Pack 1 Table 1-1 Oracle task Changes in the SF Oracle RAC installer for Oracle tasks (continued) Description CSSD agent configuration A new option Configure CSSD agent is introduced in the SF Oracle RAC installer that enables you to configure the CSSD agent after installing Oracle RAC.
18 Storage Foundation for Oracle RAC Release Notes Changes introduced in version 5.1 Service Pack 1 I/O fencing configuration You can use the -fencing option with the installsfrac program to configure I/O fencing.
Storage Foundation for Oracle RAC Release Notes Changes introduced in version 5.
20 Storage Foundation for Oracle RAC Release Notes Changes introduced in version 5.1 Service Pack 1 Support for importing shared disk groups The CVMVolDg agent now imports the shared disk group from the CVM master node, if the disk group is not already imported, when the corresponding CVMVolDg resource is brought online.
Storage Foundation for Oracle RAC Release Notes Changes introduced in version 5.1 Service Pack 1 corresponding CVMVolDg resource is taken offline. Set the value of this attribute to 1 if you want the agent to deport the disk group when the CVMVolDg resource is taken offline. The default value is set to 0.
22 Storage Foundation for Oracle RAC Release Notes Changes introduced in version 5.1 Service Pack 1 ■ Veritas Volume Replicator (VVR) commands including vxibc, vxrlink, vxrsync, vxrvg, vrport, vrstat, and vradmin. ■ The vxdisk command. Changing the CVM master online Cluster Volume Manager (CVM) now supports changing the CVM master from one node in the cluster to another node, while the cluster is online. CVM migrates the master node, and reconfigures the cluster.
Storage Foundation for Oracle RAC Release Notes Changes introduced in version 5.1 Service Pack 1 Note: The disk group version must be at least 160 to create and use the cdsdisk format on disks of size greater than 1 TB. Support for intelligent monitoring of VCS resources using IMF VCS now supports intelligent resource monitoring in addition to poll-based monitoring. Intelligent Monitoring Framework (IMF) is an extension to the VCS agent framework.
24 Storage Foundation for Oracle RAC Release Notes Changes introduced in version 5.
Storage Foundation for Oracle RAC Release Notes Changes introduced in version 5.
26 Storage Foundation for Oracle RAC Release Notes Changes introduced in version 5.1 Service Pack 1 is enabled, the agent considers the current traffic on all devices and calculates a "winner" device with lower traffic. If load balancing is not enabled, the IP address is failed over to the link with the lower network ID. New CRSResource agent The CRSResource agent provides an alternative mechanism for monitoring the Oracle database in the absence of the VCS Oracle agent.
Storage Foundation for Oracle RAC Release Notes No longer supported For more information: See the Veritas Storage Foundation for Oracle RAC Administrator's Guide. See the lmxdbstat(1M) manual page. Enhancements to VRTSexplorer utility The VRTSexplorer utility includes enhancements to collect more logs, including Oracle Clusterware and Oracle database log information.
28 Storage Foundation for Oracle RAC Release Notes No longer supported ■ Option hawizard -rac for configuring service groups ■ 32-bit Oracle architectures ■ Oracle RAC 9i, Oracle RAC 10g Release 1 ■ Storage Checkpoint commands for checking and saving the configuration of the Oracle RAC database: dbed_checkconfig, dbed_saveconfig ■ Use of crossover cables Oracle does not support the use of crossover cables for cluster interconnects due to the possibility of data corruption and other software limit
Storage Foundation for Oracle RAC Release Notes System requirements ■ dbed_scheduler ■ sfua_rept_migrate with -r and -f options System requirements This section describes the system requirements for this release. Important preinstallation information Before you install SF Oracle RAC, make sure you have reviewed the following information: ■ Hardware compatibility list for information about supported hardware: http://entsupport.symantec.
30 Storage Foundation for Oracle RAC Release Notes System requirements Table 1-3 Hardware requirements for basic clusters (continued) Item Description Disks SF Oracle RAC requires that all storage disks support SCSI-3 Persistent Reservations (PR). Note: The coordinator disk does not store data, so configure the disk as the smallest possible LUN on a disk array to avoid wasting space. DMP devices are mandatory for use as coordinator disks.
Storage Foundation for Oracle RAC Release Notes System requirements 31 Hardware requirements for campus clusters Table 1-4 Item Description Storage ■ The storage switch (to which each host on a site connects) must have access to storage arrays at all the sites. ■ Volumes must be mirrored with storage allocated from at least two sites. DWDM links are recommended between sites for storage links. DWDM works at the physical layer and requires multiplexer and de-multiplexer devices.
32 Storage Foundation for Oracle RAC Release Notes System requirements Supported database software For the latest information on supported Oracle database versions, see the following Technical Support TechNote: http://www.symantec.com/docs/TECH44807 Note: SF Oracle RAC supports only 64-bit Oracle.
Storage Foundation for Oracle RAC Release Notes System requirements ■ Coordinator disks cannot be the special devices that array vendors use. For example, you cannot use EMC gatekeeper devices as coordinator disks. CP server requirements SF Oracle RAC 5.1SP1 clusters (application clusters) support CP servers which are hosted on the following VCS and SFHA versions: ■ VCS 5.1 or 5.1SP1 single-node cluster CP server requires LLT and GAB to be configured on the single-node VCS cluster that hosts CP server.
34 Storage Foundation for Oracle RAC Release Notes System requirements Table 1-5 CP server hardware requirements Hardware required Description Disk space To host the CP server on a VCS cluster or SFHA cluster, each host requires the following file system space: 550 MB in the /opt directory (additionally, the language pack requires another 15 MB) ■ 300 MB in /usr ■ ■ 20 MB in /var Storage When CP server is hosted on an SFHA cluster, there must be shared storage between the CP servers.
Storage Foundation for Oracle RAC Release Notes System requirements ■ Symantec recommends that network access from the application clusters to the CP servers should be made highly-available and redundant. The network connections require either a secure LAN or VPN. ■ The CP server uses the TCP/IP protocol to connect to and communicate with the application clusters by these network paths. The CP server listens for messages from the application clusters using TCP port 14250.
36 Storage Foundation for Oracle RAC Release Notes Fixed issues Supported replication technologies for global clusters SF Oracle RAC supports the following hardware-based replication and software-based replication technologies for global cluster configurations: Hardware-based replication Software-based replication ■ EMC SRDF ■ Hitachi TrueCopy ■ IBM Metro Mirror ■ IBM SAN Volume Controller (SVC) ■ EMC MirrorView ■ Veritas Volume Replicator ■ Oracle Data Guard Fixed issues This section cove
Storage Foundation for Oracle RAC Release Notes Fixed issues Table 1-7 37 Fixed issues in SF Oracle RAC 5.1 SP1 (continued) Incident number Description 1879412 The Low Latency Transport Multiplexer (LMX) module may cause the system to panic with the following message: kernel heap corruption detected Incorrect manipulation of the request queue corrupts the memory and causes the system to panic. When the last request is removed from the queue, the queue pointers are not updated correctly.
38 Storage Foundation for Oracle RAC Release Notes Known issues Table 1-8 Veritas Storage Foundation for Databases tools fixed issues Incident Description 1873738 The dbed_vmchecksnap command may fail 1399393 Clone command fails on an Oracle RAC database 1736516 Clone command fails for instant checkpoint on Logical Standby database 1789290 dbed_vmclonedb -o recoverdb for offhost fails for Oracle 10gr2 and prior versions 1810711 Flashsnap reverse resync command fails on offhost flashsnap cloni
Storage Foundation for Oracle RAC Release Notes Known issues 39 export OUI_ARGS=-ignoreInternalDriverError For more information, see the Oracle Metalink document: 970166.1 During installation or system startup, Oracle Grid Infrastructure may fail to start After successful installation of Oracle RAC 11g Release 2 Grid Infrastructure, while executing the root.sh script, ohasd may fail to start.
40 Storage Foundation for Oracle RAC Release Notes Known issues For example: Utility ============================================================ OUI-25031: Some of the configuration assistants failed. It is strongly recommended that you retry the configuration assistants at this time. Not successfully running any " Recommended" assistants means your system will not be correctly configured. 1. Check the Details panel on the Configuration Assistant Screen to see the errors resulting in the failures. 2.
Storage Foundation for Oracle RAC Release Notes Known issues The EULAs for Japanese and Chinese now appear in those language in the following locations: The Japanese EULAs appear in /product_dir/EULA/ja/product_eula.pdf The Chinese EULAs appear in /product_dir/EULA/zh/product_eula.pdf During product migration the installer overestimates disk space use (2088827) The installer displays the space that all the product depots and patches needs.
42 Storage Foundation for Oracle RAC Release Notes Known issues This could cause writes to checkpoints to fail. It could also trigger the removal of removable checkpoints.
Storage Foundation for Oracle RAC Release Notes Known issues Workaround: This message may be safely ignored. CVMVolDg agent may fail to deport CVM disk group The CVM disk group is deported based on the order in which the CVMVolDg resources are taken offline. If the CVMVolDg resources in the disk group contain a mixed setting of 1 and 0 for the CVMDeportOnOffline attribute, the disk group is deported only if the attribute value is 1 for the last CVMVolDg resource taken offline.
44 Storage Foundation for Oracle RAC Release Notes Known issues Stopping VCS does not unregister port f from GAB membership In an SF Oracle RAC cluster with all the CFS resources under VCS control, when you stop VCS, all the CFS resources must go down cleanly and CFS must unregister port f from GAB membership. Oracle RAC 10g Clusterware does not clean up all its processes when it is stopped. Now, when you stop VCS, all the CFS resources go down.
Storage Foundation for Oracle RAC Release Notes Known issues Node join can lead to hang if an upgrade of the cluster protocol version is in progress (2103567) If you attempt to join a node to the cluster while Cluster Volume Manager (CVM) is upgrading the cluster protocol version, the system may hang. This issue occurs if the node is attempting to join the cluster after you issue the vxdctl upgrade command to upgrade the CVM cluster.
46 Storage Foundation for Oracle RAC Release Notes Known issues 4 Reattach the detached sites: # vxdg -g dg_name reattachsite site_name 5 Verify that the state of the detached sites is now ACTIVE: # vxprint Issues related to LLT This section covers the known issues related to LLT in this release.
Storage Foundation for Oracle RAC Release Notes Known issues Preferred fencing does not work as expected for large clusters in certain cases If you have configured system-based or group-based preferred fencing policy, preferred fencing does not work if all the following cases are true: ■ The fencing setup uses customized mode with one or more CP servers. ■ The application cluster has more than eight nodes.
48 Storage Foundation for Oracle RAC Release Notes Known issues Installer is unable to split a cluster that is registered with one or more CP servers Splitting a cluster that uses server-based fencing is currently not supported. [2110148] You can split a cluster into two and reconfigure SF Oracle RAC on the two clusters using the installer. For example, you can split a cluster clus1 into clus1A and clus1B.
Storage Foundation for Oracle RAC Release Notes Known issues Upgrading Veritas Storage Foundation for Databases (SFDB) tools from 5.0.x to 5.1SP1 (2184482) When upgrading from SF Oracle RAC version 5.0 or 5.0.1 to SF Oracle RAC 5.1SP1 the S*vxdbms3 startup script is renamed to NO_S*vxdbms3. The S*vxdbms3 startup script is required by sfua_rept_upgrade. Thus when sfua_rept_upgrade is run, it is unable to find the S*vxdbms3 startup script and gives the error message: /sbin/rc3.
50 Storage Foundation for Oracle RAC Release Notes Known issues a VCS resource for the SNAP disk group. As such, when the database fails over, only the primary disk group is moved to another node. Workaround There is no workaround for this issue. The error messages depend on the timing of the database failover. To fix the problem, you need to bring the FlashSnap state to SNAP_READY. Depending on the failure, you may have to use base VxVM commands to reattach mirrors.
Storage Foundation for Oracle RAC Release Notes Known issues 51 To reattach volumes in a multiple disk group environment if the snapshot operation fails 1 Join the snapshot disk groups to primary diskgroups. The snapshot disk group name is a concatenation of “SNAPSHOT_DG_PREFIX” parameter value in snapplan and primary disk group name. Use the following command to join the disk groups: # vxdg join snapshop_disk_group_name primary_disk_group_name 2 Start all the volumes in primary disk group.
52 Storage Foundation for Oracle RAC Release Notes Software limitations Software limitations This section describes the software limitations in this release. Stopping systems in clusters with I/O fencing configured The I/O fencing feature protects against data corruption resulting from a failed cluster interconnect, or “split brain.” See the Veritas Cluster Server Administrator's Guide for a description of the problems a failed interconnect can create and the protection I/O fencing provides.
Storage Foundation for Oracle RAC Release Notes Software limitations To resize layered volumes that are associated to an RVG 1 Pause or stop the applications. 2 Wait for the RLINKs to be up to date. Enter the following: # vxrlink -g diskgroup status rlink 3 Stop the affected RVG. Enter the following: # vxrvg -g diskgroup stop rvg 4 Disassociate the volumes from the RVG. Enter the following: # vxvol -g diskgroup dis vol 5 Resize the volumes. In this example, the volume is increased to 10 GB.
54 Storage Foundation for Oracle RAC Release Notes Software limitations Performance recommendation for space-optimized volume snapshots For minimal performance impact, Symantec recommends that the Space Optimized Snapshots (SOS) be created only of data volumes. A mirror breakoff snapshot should be created of the Oracle log volume. The log volumes are typically small in size and do not have significant space overhead.
Storage Foundation for Oracle RAC Release Notes Documentation errata ■ When cloning a database using Database FlashSnap, the Oracle database must have at least one mandatory archive destination. For more information about Oracle parameters for archiving redo logs, see your Oracle documentation. ■ Only online snapshots are supported for an Oracle RAC database, when using the dbed_vmsnap, dbed_vmclonedb, and dbed_vmchecksnap commands.
56 Storage Foundation for Oracle RAC Release Notes Documentation Veritas Storage Foundation for Oracle RAC Installation and Configuration Guide Table 1-9 Documentation errata Topic Correction Section 8: Uninstallation of SF Oracle RAC The root disk encapsulation cannot be removed using the steps described in the topic. Do not perform the step. Chapter 31: Preparing to uninstall SF Oracle RAC from a cluster You may restore or create an LVM root disk image from the VxVM root disk on an other disk.
Storage Foundation for Oracle RAC Release Notes Documentation Table 1-11 Veritas Cluster Server documentation Title File name Veritas Cluster Server Installation Guide vcs_install_51sp1_hpux.pdf Veritas Cluster Server Release Notes vcs_notes_51sp1_hpux.pdf Veritas Cluster Server Administrator’s Guide vcs_admin_51sp1_hpux.pdf Veritas Cluster Server Bundled Agents Reference vcs_bundled_agents_51sp1_hpux.pdf Guide Veritas Cluster Server Agent Developer’s Guide vcs_agent_dev_51sp1.
58 Storage Foundation for Oracle RAC Release Notes Documentation Table 1-13 lists the documentation for Veritas Volume Manager and Veritas File System. Table 1-13 Veritas Volume Manager and Veritas File System documentation Document title File name Veritas Volume Manager Administrator’s Guide vxvm_admin_51sp1_hpux.pdf Veritas Volume Manager Troubleshooting Guide vxvm_tshoot_51sp1_hpux.pdf Veritas File System Administrator’s Guide vxfs_admin_51sp1_hpux.
Storage Foundation for Oracle RAC Release Notes Documentation ■ For the Bourne or Korn shell (sh or ksh), enter the following commands: MANPATH=$MANPATH:/opt/VRTS/man export MANPATH ■ For C shell (csh or tcsh), enter the following command: setenv MANPATH ${MANPATH}:/opt/VRTS/man See the man(1) manual page.
60 Storage Foundation for Oracle RAC Release Notes Documentation
