READ ME before using the Veritas Storage Foundation™ 5.1 SP1 for Oracle RAC Administrator's Guide (April 2011)

SGeRAC uses LLT heartbeats to determine cluster membership:
• When systems no longer receive heartbeat messages from a peer for a predetermined interval,
a protocol excludes the peer from the current membership.
• GAB informs processes on the remaining nodes that the cluster membership has changed; this
action initiates recovery actions specific to each module. For example, CVM must initiate
volume recovery and CFS must perform a fast parallel file system check.
• When systems start receiving heartbeats from a peer outside of the current membership, a
protocol enables the peer to join the membership.
Cluster Communications
GAB and Serviceguard Heartbeat provide reliable cluster communication between Serviceguard
and SGeRAC modules. GAB provides guaranteed delivery of point-to-point messages and broadcast
messages to all nodes. Point-to-point messaging involves sending and acknowledging the message.
Atomic-broadcast messaging ensures all systems within the cluster receive all messages. If a failure
occurs while transmitting a broadcast message, GAB ensures all systems have the same information
after recovery.
Low-level Communication: Port Relationship Between GAB and Processes
Most components in SGeRAC use GAB for communication. Each process wanting to communicate
with a peer process on other nodes registers with GAB on a specific port. This registration enables
communication and notification of membership changes. For example, the SGeRAC engine (had)
registers on port h. had receives messages from peer had processes on port h. had also receives
notification when a node fails or when a peer process on port h becomes unregistered.
Some processes use multiple ports for specific communications requirements. For example, CVM
uses multiple ports to allow communications by kernel and user-level functions in CVM independently.
Cluster Volume Manager
CVM is an extension of Veritas Volume Manager, the industry-standard storage virtualization
platform. CVM extends the concepts of VxVM across multiple nodes. Each node recognizes the
same logical volume layout, and more importantly, the same state of all volume resources.
CVM supports performance-enhancing capabilities, such as striping, mirroring, and mirror break-off
(snapshot) for off-host backup. You can use standard VxVM commands from one node in the cluster
to manage all storage. All other nodes immediately recognize any changes in disk group and
volume configuration with no interaction.
CVM Architecture
CVM is designed with a “master and slave” architecture. One node in the cluster acts as the
configuration master for logical volume management, and all other nodes are slaves. Any node
can take over as master if the existing master fails. The CVM master exists on a per-cluster basis
and uses GAB and LLT to transport its configuration data.
Just as with VxVM, the Volume Manager configuration daemon, vxconfigd, maintains the
configuration of logical volumes. This daemon handles changes to the volumes by updating the
operating system at the kernel level. For example, if a mirror of a volume fails, the mirror detaches
from the volume and vxconfigd determines the proper course of action, updates the new volume
layout, and informs the kernel of a new volume layout. CVM extends this behavior across multiple
nodes and propagates volume changes to the master vxconfigd. (You must perform operator-initiated
changes on the master node.) The vxconfigd process on the master pushes these changes out to
slave vxconfigd processes, each of which updates the local kernel.
12 Introducing Serviceguard Extension for RAC