Architecture considerations and best practices for architecting an Oracle RAC solution with Serviceguard and SGeRAC

Oracle RAC supports the use of CFS for Oracle Clusterware software and Oracle RAC software.

When the software is installed on CFS, node-specific directories are created to hold log and trace

files that are generated from Oracle Clusterware processes and the RAC instance running on each

node.

CAUTION: All the log and trace files from all nodes in the cluster are stored in a shared CFS mount

point; therefore, the file system can fill up quickly. When a file system is full, especially the one that is

used for ORA_CRS_HOME, Oracle Clusterware will not be able to read the file that contains CSS

daemon PID and will either hang or crash the node. So it’s important to periodically delete old log

files, trace files, and core files in the following directories

• $ORA_CRS_HOME/log/<nodename>/crsd

to free up disk space:

• $ORA_CRS_HOME/log/<nodename>/cssd

• $ORA_CRS_HOME/log/<nodename>/evmd

• $ORA_CRS_HOME/log/<nodename>/racg

SLVM and CVM

Prior to Serviceguard A.11.17, SLVM and CVM were the only two storage management choices for

RAC environments on HP-UX 11i. These two storage management choices will continue to be offered

as options for Oracle RAC with SGeRAC.

SLVM provides an online volume group reconfiguration feature known as Single-Node Online

Reconfiguration (SNOR). Before SNOR was available, reconfiguration of a volume group, such as

adding a physical device to increase the volume group size, would require taking all RAC instances

offline. SNOR improves online configuration by allowing one instance to be online accessing the data

while the volume group holding the database data is reconfigured.

Multi-Node Online Reconfiguration (MORE), introduced in the HP-UX 11i v3 September 2009

release, allows you to change the configuration of LVM Version 2.1 or later volume groups

The key advantage of using SLVM and CVM with RAC is the I/O performance that raw volumes

provide. The disadvantage is that it’s hard to manage because there is no file system. For example,

you cannot use the file listing command like ls to see the database file’s listing; you cannot use the

bdf command to see how much space has been used.

while

the volume groups are activated in shared mode on any number of cluster nodes. Applications can

continue to use these volume groups without interruption while the volume groups are being

reconfigured.

Best practices when using SLVM for Oracle RAC

An LVM LV has an infinite I/O timeout by default, which would cause the CSS daemon to hang if the

node had no access to the storage that housed either the Oracle Clusterware software or Oracle

Clusterware voting disk. To avoid the hang problem, the LVs used for these components should be

configured with an I/O timeout value. The LV’s time out value can be changed using the lvchange

command. The recommended value is:

LV timeout value = (# of paths * PV timeout) + 10 seconds

Note that the physical volume (PV) timeout value can be changed, so make sure to use the configured

value instead of the default PV timeout of 30 seconds.

As observed in 10gR2. These directories may be changed in later versions of Oracle.

MORE is available only for LVM Version 2.1 volume groups. For earlier versions, you can still use the Single-Node Online Reconfiguration

(SNOR) method.