HP Serviceguard Extended Distance Cluster for Linux A.11.20.10 Deployment Guide, December 2012

Table 4 Disaster Scenarios and Their Handling (continued)
Recovery ProcessWhat Happens When This
Disaster Occurs
Disaster Scenario
In this case, the package (P1) runs with
RPO_TARGET set to 60 seconds.
Package P1 is running on node N1. P1
uses a mirror md0 consisting of S1
(local to node N1, for example
/dev/hpdev/mylink-sde) and S2
(local to node N2). The first failure
occurs when all FC links between two
data centers fail, causing N1 to lose
access to S2 and N2 to lose access to
S1.
After the package resumes activity and
runs for 90 seconds, a second failure
occurs causing node N1 to fail.
In this scenario, no attempts are made to
repair the first failure until the second
failure occurs. Complete the following
procedure to initiate a recovery:
1. To recover from the first failure, restore
the FC links between the data centers.
As a result, S1 is accessible from N2.
2. After the FC links are restored, and S1
is accessible from N2, run the following
command to restart the package on N2.
# cmrunpkg <package_name>
When the package starts up on N2, it
automatically adds S1 back into the array
and starts re-mirroring from S1 to S2.
When re-mirroring is complete, the
extended distance cluster detects and
accepts S1 as part of md0 again.
For the second failure, restore N1. Once
it is restored, it joins the cluster and can
access S1 and S2.
1. Run the following command to enable
P1 to run on N1:
# cmmodpkg -e P1 -n N1
Complete the following procedure to initiate
a recovery:
1. Reconnect the FC links between the data
centers. As a result, S1
(/dev/hpdev/mylink-sde ) becomes
accessible from N2
2. Run the following command to add S1
to md0 on N2
# mdadm --add /dev/md0
/dev/hpdev/mylink-sde
This command initiates the re-mirroring
process from S2 to S1. When re-mirroring
is complete, the extended distance cluster
detects S1 and accepts it as part of md0.
If the FC links are not restored on
N2, you can only start the
package forcefully. You can
forcefully start a package only if
it is determined that the
associated data loss is
acceptable.
After you execute the force start
commands, package P1 starts on
N2 and runs with md0 consisting
of only S2
(/dev/hpdev/mylink-sdf ).
This scenario is an extension of the
previous failure scenario. In the
previous scenario, when the package
fails over to N2, it does not start as the
value of RPO_TARGET would have
been exceeded.
To forcefully start the package P1 on
N2 when the FC links are not restored
on N2, check the package log file on
N2 and execute the commands that
appear in it.
38 Disaster Scenarios and Their Handling