Front cover IBM System Storage DS6000 Series: Copy Services with IBM System z Plan, install and configure DS6000 Copy Services with System z Learn how to use the management interfaces: TSO, DS CLI, DS GUI Learn about TPC for replication support Gustavo Castets Bertrand Dufrasne Stephen Baird Werner Bauer Denise Brown Jana Jamsek ibm.
International Technical Support Organization IBM System Storage DS6000 Series: Copy Services with IBM System z December 2006 SG24-6782-02
Note: Before using this information and the product it supports, read the information in “Notices” on page xv. Third Edition (December 2006) This edition applies to features, microcode, GUI and DS CLI as announced for the DS6000 in August 2006. © Copyright International Business Machines Corporation 2005, 2006. All rights reserved. Note to U.S. Government Users Restricted Rights -- Use, duplication or disclosure restricted by GSA ADP Schedule Contract with IBM Corp.
Contents Notices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xv Trademarks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xvi Preface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xvii The team that wrote this redbook. . . . . . . . . . . . . . . . . . . . . . . . . .
4.5 Command structure. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.6 Copy Services commands . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.7 Using the DS CLI application . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.7.1 Single-shot mode . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.7.
9.2.1 Local FlashCopy management . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71 9.2.2 Remote FlashCopy management . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72 9.3 z/OS-provided interfaces. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72 9.4 Local FlashCopy using the DS CLI . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73 9.4.
vi Chapter 13. Metro Mirror options and configuration . . . . . . . . . . . . . . . . . . . . . . . . . . 13.1 High availability solutions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13.1.1 GDPS HyperSwap Manager . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13.1.2 Open systems - Clustering . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13.2 Failover and failback . . . . . . . . . . . . . . . . .
14.6 DS Storage Manager GUI . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14.6.1 Define Metro Mirror paths . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14.6.2 Create Metro Mirror pairs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14.6.3 Resume suspended pair . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14.7 ANTRQST API . . . . . . . . . . . . . . . . .
19.1.1 Peak bandwidth requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 220 19.2 Scalability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 220 19.2.1 Addition of capacity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 220 Chapter 20. Global Copy interfaces. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20.
23.3.1 Add or remove volumes to a Global Mirror session . . . . . . . . . . . . . . . . . . . . . 23.3.2 Add or remove storage disk subsystems or LSSs . . . . . . . . . . . . . . . . . . . . . . 23.3.3 Modify Global Mirror session parameters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23.3.4 Global Mirror environment topology changes . . . . . . . . . . . . . . . . . . . . . . . . . . 23.3.5 Remove a FlashCopy relationship . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23.
x 24.5.14 Delete Global Copy pairs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24.5.15 Remove all paths . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24.6 ANTRQST macro . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24.7 DS Storage Manager GUI . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24.7.
26.7.5 Delete Global Copy pairs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26.7.6 Remove Global Copy paths . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26.8 Query device information with ICKDSF. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26.9 Set up a Global Mirror environment using DS SM . . . . . . . . . . . . . . . . . . . . . . . . . . 26.9.1 Create paths . . . . . . . . . . . . . . . . . . . . . . . . . . .
28.3.3 Establish logical paths between DS8000 and DS6000 using DS CLI. . . . . . . . 28.3.4 Path creation using TSO . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28.4 Managing Metro Mirror or Global Copy pairs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28.4.1 Managing Metro Mirror or Global Copy pairs with the DS GUI . . . . . . . . . . . . . 28.4.2 Managing Metro Mirror pairs using the DS CLI. . . . . . . . . . . . . . . . . . . . . . . . . 28.4.
31.4.1 FlashCopy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31.4.2 Metro Mirror . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31.4.3 Global Copy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31.4.4 Global Mirror . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31.4.
Production and performance considerations. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 513 SMF information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 515 Examples of Concurrent Copy invocation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 515 Appendix B. SNMP notifications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . SNMP overview . . . . . . . . . . . . . . . .
Notices This information was developed for products and services offered in the U.S.A. IBM may not offer the products, services, or features discussed in this document in other countries. Consult your local IBM representative for information on the products and services currently available in your area. Any reference to an IBM product, program, or service is not intended to state or imply that only that IBM product, program, or service may be used.
Trademarks The following terms are trademarks of the International Business Machines Corporation in the United States, other countries, or both: AIX 5L™ AIX® AS/400® CICS® DB2® DFSMSdfp™ DFSMSdss™ DFSMShsm™ DFSMSrmm™ DS4000™ DS6000™ DS8000™ Enterprise Storage Server® ESCON® eServer™ FlashCopy® FICON® Geographically Dispersed Parallel Sysplex™ GDPS® HyperSwap™ HACMP™ IBM® IMS™ iSeries™ i5/OS® MVS™ NetView® OS/390® OS/400® Parallel Sysplex® PowerPC® POWER4™ POWER5™ Redbooks™ Redbooks (logo) ™ RMF™ S/390® Sy
Preface This IBM Redbook will help you plan, install, and configure the IBM System Storage DS6000 Copy Services functions in System z™ environments, and provides the details you need to implement and manage these functions. The book includes hints and tips. This document is intended to help you either to design and set up a new Copy Services installation, or to migrate from an existing installation. It also addresses functionality and terminology differences from other IBM Copy Services products.
include Copy Services solutions in Metro/Global Mirror and Incremental Re-synchronization for the DS8000. Denise holds a degree in Engineering Mathematics. Jana Jamsek is an IT Specialist with IBM Slovenia. She works in Storage Advanced Technical Support for Europe as a specialist for IBM Storage Systems and i5/OS® systems. Jana has eight years of experience in the System i™ and AS/400® areas, and six years of experience in Storage.
The team: Gustavo, Robert, Wenzel, Jana, Peter, Markus, Denise, Werner, Ying, Stephen, Bertrand Acknowledgements John Bynum and Robert Moon - Technical Support Marketing Leads IBM US We want to thank Michael Eggloff and Peter Klee for hosting us at the European Storage Competency Center in Mainz, Germany. Günter Schmitt, Uwe Schweikhard, Edgar Strubel (ATS - IBM Mainz) for their help in reserving and preparing the equipment we used.
Emma Jacobs and Alfred Schwab ITSO editorial assistance Become a published author Join us for a two- to six-week residency program! Help write an IBM Redbook dealing with specific products or solutions, while getting hands-on experience with leading-edge technologies. You'll team with IBM technical professionals, Business Partners and/or customers. Your efforts will help increase product acceptance and customer satisfaction.
Summary of changes This section describes the technical changes made in this edition of the book and in previous editions. This edition may also include minor corrections and editorial changes that are not identified. Summary of Changes for SG24-6782-02 for IBM System Storage DS6000 Series: Copy Services with IBM System z as created or updated on December 14, 2006. December 2006, Third Edition This revision reflects the addition, deletion, or modification of new and changed information described below.
xxii IBM System Storage DS6000 Series: Copy Services with IBM System z
Part 1 Part 1 Overview In Part 1 we describe the various Advanced Copy Services offerings for the IBM System Storage DS6000 series and how they relate to previous Copy Services offerings that are available on the Enterprise Storage Server® (ESS). This part also shows how the existing Copy Services functions from the ESS can coexist with the Copy Services for the DS6000 series. Similarly, we discuss their use with the DS8000 series Copy Services.
You can use either the IBM System Storage DS Storage Command-Line Interface, DS CLI, or the IBM System Storage DS Storage Manager Copy Services Graphical User Interface, DS GUI, to configure Copy Services. It should be noted that: All DS6000 installations require at least an Operating Equipment License (OEL) key to operate. However, Copy Services functions can only be used where the appropriate License Key is installed.
1 Chapter 1. Introduction This chapter introduces the business drivers for Copy Services and provides a brief summary of the various Copy Services functions available on the DS6000 series. These services are very similar to the existing Copy Services for the IBM Enterprise Storage Server, and some models of the ESS are interoperable with the DS8000 as well as the DS6000 © Copyright IBM Corp. 2006. All rights reserved.
1.1 Introduction to Copy Services We briefly describe each function of the Copy Services in this section. Copy Services are a collection of functions that provide for Disaster Recovery, data migration, and data duplication functions. There are two primary types of Copy Services functions: Point-in-Time Copy and Remote Mirror and Copy. Generally, the Point-in-Time Copy function is used for data duplication and the Remote Mirror and Copy function is used for data migration and Disaster Recovery.
When a FlashCopy operation is invoked, it takes only a few seconds to complete the process of establishing the FlashCopy pair and creating the necessary control bitmaps. Thereafter, you have access to a Point-in-Time Copy of the source volume. As soon as the pair has been established, you can read and write to both the source and target volumes. After creating the bitmap, a background process begins to copy the real data from the source to the target volumes.
Global Mirror (Asynchronous PPRC) Global Mirror provides a long-distance remote copy feature across two sites using asynchronous technology. This solution is based on the existing Global Copy and FlashCopy. With Global Mirror, the data that the host writes to the Storage Unit at the local site is asynchronously shadowed to the Storage Unit at the remote site. A consistent copy of the data is automatically maintained on the Storage Unit at the remote site.
2 Chapter 2. Copy Services architecture This chapter is an overview of the structure of the Copy Services communication architecture in either an open or System z environment. The chapter covers the following topics: Introduction to the Copy Services structure How the new structure of Copy Services works System z communication path for Copy Services © Copyright IBM Corp. 2006. All rights reserved.
2.1 Introduction to the Copy Services structure The Copy Services architecture for the IBM ESS 800 used a construct called a Copy Services Domain to manage Copy Services. The communications architecture for DS6000 and DS8000 is noticeably different. Instead of a Copy Services Domain, we now use the concept of Storage Complexes. You can perform Copy Services operations both within or between Storage Complexes.
Example 2-1 Difference between a DS6000 Storage Unit and DS6000 SFI dscli> lssu Date/Time: 13 November 2005 19:21:17 IBM DSCLI Version: 5.1.0.204 Name ID Model WWNN pw state ========================================================= 13-00247 IBM.1750-1300247 511 500507630EFFFE16 On dscli> lssi Date/Time: 13 November 2005 19:21:44 IBM DSCLI Version: 5.1.0.204 Name ID Storage Unit Model WWNN State ESSNet ============================================================================ IBM.1750-1300247 IBM.
================================================================ 2107_75ABTV1/V2 IBM.2107-75ABTV0 9A2 5005076303FFFE63 On dscli> lssi Date/Time: 13 November 2005 19:25:34 IBM DSCLI Version: 5.1.0.204 Name ID Storage Unit Model WWNN State ESSNet ============================================================================ IBM.2107-75ABTV1 IBM.2107-75ABTV0 9A2 5005076303FFC663 Online Enabled IBM.2107-75ABTV2 IBM.2107-75ABTV0 9A2 5005076303FFCE63 Online Enabled 2.1.
2.2 How the new structure of Copy Services works With the exception of System z commands (see “System z communication path for Copy Services” on page 13), communication to a DS6000 or DS8000 needs an available DS SMC or DS HMC. See Figure 2-2.
The Network Interface Server software communicates with the Network Interface Node, which resides on each controller of the DS6000. From this point, the Network Interface then interacts with the microcode that operates the controllers. 2105 ESS 800 management structure In Figure 2-2 on page 11, you can see that: The client uses either DS CLI or a ESS Copy Services Web browser GUI to issue commands to the ESS 800 Copy Services Server running on an ESS 800 cluster.
The DS GUI is accessed with a Web browser. The DS GUI cannot save tasks (unlike the ESS, which can), and you cannot use the DS GUI to initiate a series of saved tasks. Instead, you must use the DS CLI. If you wish to use a single GUI to manage multiple Storage Complexes, then, in the GUI, you must define all of the Storage Complexes. This definition allows you to manage FlashCopy or Remote Mirror and Copy on, or between, every Storage Unit in every defined and accessible Storage Complex.
14 IBM System Storage DS6000 Series: Copy Services with IBM System z
Part 2 Part 2 Interfaces In this part we discuss the interfaces available to manage the Copy Services features of the DS6000. We give an overview of the interfaces, discuss the options available, discuss configuration considerations, and give some usage examples of the interfaces. © Copyright IBM Corp. 2006. All rights reserved.
16 IBM System Storage DS6000 Series: Copy Services with IBM System z
3 Chapter 3. DS Storage Manager This chapter is an introduction to the IBM System Storage DS™ Storage Manager, which can be used to configure and administer the DS storage system. The DS Storage Manager is an interface that is used to perform logical configurations, service, Copy Services management, and firmware upgrades.
3.1 System and hardware requirements The DS Storage Manager is a Web-based graphical user interface (GUI) that is used to perform logical configuration and Copy Services management functions. It can be accessed with a Web browser from any location that has network access to the DS management console. The DS Storage Manager software must be installed on a user-provided computer. We refer to this computer as the DS Storage Management Console (SMC).
You can choose to install DS Storage Manager on the Windows operating system with either of the graphical mode or the unattended mode: The graphical mode allows you to use an online wizard that guides you through the installation process providing prompts and information necessary to complete the installation. The unattended mode (also called silent mode) allows you to customize a response file and issue a command to complete the installation process.
Clearly, you should change the IP address to match the one used by your SMC. If you are actually working on the SMC PC itself, you can also use 127.0.0.1 since this is the loopback address that connects to the local PC. Logging in to the DS Storage Manager requires that you provide your user ID and password. This function is generally administered through your system administrator and by your company policies.
3.4 Accessing the Information Center The Software Information Center provides product or application information and a GUI for browsing and searching online documentation. The broad range of topics covered includes accessibility, Copy Services, device storage, host system attachments, concurrent code loads, input/output configuration programs, and volume storage. Accessing the Information Center directly You can connect to the Information Center directly using a Web browser. If your SMC IP address is 10.0.
Figure 3-3 FlashCopy help Main page 3.5 Managing the Storage Complex When you install the DS Storage Manager software on a PC, a Storage Complex is automatically created. You can then add up to two DS6000s to the Storage Complex. Each DS6000 is a Storage Unit. Because an SMC can manage two DS6000s, a single Storage Complex could consist of one SMC and two DS6000s. Figure 3-4 shows how we connected to the Storage Complex managed by the SMC at the 10.0.0.1 IP address. 10.0.0.
SMC. There is Ethernet connectivity between the sites. Because each SMC manages only the DS6000 at its own site, this means there are two Storage Complexes. To use a single GUI to manage Copy Services at either site, then you must add these Storage Complexes to each other. Once you have done this, you can use one SMC to manage Copy Services on either Storage Unit at either site. We can also use one SMC to set up Copy Services relations between the two sites. 3.5.
10.0.0.100 10.0.0.1 Figure 3-6 Successful addition of a Storage Complex 8. Having added one DS6000 Storage Complex to another, you can use the DS6000 DS GUI to create paths and Remote Mirror and Copy pairs between any of the Storage Units. You can also use the DS6000 DS GUI to manage FlashCopy pairs on any DS6000. This is all assuming the relevant licenses are present. Note: The steps to add one DS6000 Storage Complex to another DS6000 Storage Complex cannot be performed with the DS CLI.
4 Chapter 4. DS Command-Line Interface This chapter provides an introduction to the DS Command-Line Interface (DS CLI), which can be used to configure and administer the DS6000 storage systems. It describes how it can be used to manage Copy Services relationships. In this chapter we describe: System requirements Command modes List of the commands User assistance Return codes © Copyright IBM Corp. 2006. All rights reserved.
4.1 Introduction and functionality The IBM System Storage DS Command-Line Interface (DS CLI) enables open systems hosts to invoke and manage FlashCopy and Remote Mirror and Copy functions through batch processes and scripts. While there is no support for z/OS as a server for the DS CLI, you can use the DS CLI from a supported server to control and manage Copy Services functions on z/OS volumes. Note: Most z/OS environments choose to use one of the z/OS interfaces to manage their volumes.
Note: The DS CLI cannot be installed on a Windows 64-bit operating system. Important: For the most recent information about currently supported operating systems, refer to the IBM System Storage DS6000 Information Center Web site at: http://publib.boulder.ibm.com/infocenter/ds6000ic/index.jsp The DS CLI is supplied and installed via a CD that ships with the machine. The installation does not require a reboot of the open systems host. The DS CLI requires Java™ 1.4.1 or higher. Java 1.4.
Note: A user can be assigned to more than one user group. Important: When a new user is created, the password that was initially set automatically expired and must be changed when the user first logs on. 4.4 DS CLI profile You can create default settings for the DS CLI by defining one or more profiles on the system.
4.5 Command structure This is a description of the components and structure of a Command-Line Interface command. A Command-Line Interface command consists of one to four types of components, arranged in the following order: 1. The command name: Specifies the task that the Command-Line Interface is to perform. 2. Flags: Modify the command. They provide additional information that directs the Command-Line Interface to perform the command task in a specific way. 3.
Table 4-3 Creation commands Command Description mkflash Initiates a point-in-time copy from a source to a target volume mkremoteflash Initiates a remote copy through a Remote Mirror and Copy relationship mkgmir Starts Global Mirror for a session mkpprc Establishes a Remote Mirror and Copy relationship mkpprcpath Establishes or replaces a Remote Mirror and Copy path over a Fibre Channel mkesconpprcpath Creates a Remote Mirror and Copy path over an ESCON connection mksession Opens a Global Mirr
Command Description unfreezepprc Thaws an existing Remote Mirror and Copy Consistency Group pausegmir Pauses a Global Mirror processing for a session resumegmir Resumes a Global Mirror processing for a session chsession Allows to modify a Global Mirror session Important: The Remote Mirror and Copy commands are asynchronous.
Note: When logging in to the DS CLI, you can use the hostname or the IP address of the DS SMC. 4.7.2 Script command mode Use the DS CLI script command mode, if you want to use a sequence of DS CLI commands. Administrators can use this mode to create automated processes, for example, establishing Remote Mirror and Copy relationships for volume pairs. You can issue the DS CLI script from the command prompt at the same time that you provide your login information: 1.
2. Provide the information that is requested by the information prompts. The information prompts might not appear, if you have provided this information in your profile file. The command prompt switches to a dscli command prompt. 3. Begin using the DS CLI commands and parameters. You are not required to begin each command with dscli, because this prefix is provided by the dscli command prompt. Example 4-3 shows the use of interactive command mode.
Return code Category Description 4 Server error The DS CLI server had an error. 5 Authentication error Password or user ID details are incorrect. 6 Application error The DS CLI application had an error. 4.9 User assistance The DS CLI is designed to include several forms of user assistance. The main form of user assistance is accessed from the help command, such as: help Lists all available DS CLI commands. help -s Lists all DS CLI commands with a brief description of each.
4.10 Usage examples It is not the intent of this section to list every DS CLI command and its syntax. If you need a list of all the available commands or assistance with DS CLI commands, refer to IBM System Storage DS6000: Command-Line Interface User´s Guide, GC26-7922, or you can use the online help. Example 4-6 shows some of the common commands used on a DS6000. Example 4-6 Examples of DS CLI commands # The following command establishes flashcopy pairs dscli> mkflash -dev IBM.
36 IBM System Storage DS6000 Series: Copy Services with IBM System z
5 Chapter 5. System z interfaces This chapter discusses the interfaces that are available with the System z servers for managing DS6000 Copy Services functions. This chapter covers the following topics: System z interfaces TSO ICKDSF DFSMSdss The ANTRQST macro z/TPF commands © Copyright IBM Corp. 2006. All rights reserved.
5.1 System z interfaces In addition to using the DS GUI or the DS CLI, several possible interfaces are available to System z users for managing DS6000 Copy Services relationships. These are: TSO ICKDSF DFSMSdss The ANTRQST macro Native TPF commands (for z/TPF only) These interfaces have the advantage of not having to issue their commands to the DS6000 SMC. They can instead directly send commands inband over a FICON channel connection between the DS6000 and the System z operating system.
5.3 ICKDSF The System z ICKDSF utility offers a means of control for Copy Services functions. ICKDSF is the only direct interface for z/VM and z/VSE environments. ICKDSF typically runs as a batch program and can be automatically run from batch scheduling products (for example Tivoli® Workload Scheduler). More information on ICKDSF can be found in Device Support Facilities User’s Guide and Reference, GC35-0033. 5.4 DFSMSdss The DFSMSdss commands can be used to manage Copy Services relationships.
40 IBM System Storage DS6000 Series: Copy Services with IBM System z
Part 3 Part 3 FlashCopy This part of the book describes the IBM System Storage FlashCopy for DS6000 when used in a System z environment. We discuss the features of FlashCopy and describe the options for its setup. We also show which management interfaces can be used, as well as the important aspects to be considered when establishing FlashCopy relationships.
42 IBM System Storage DS6000 Series: Copy Services with IBM System z
6 Chapter 6. FlashCopy overview FlashCopy creates a copy of a volume at a specific point-in-time, which we also refer to as a Point-in-Time copy, instantaneous copy, or time-zero copy (t0 copy). This chapter explains the basic characteristics of FlashCopy when used in a System z environment with the DS6000. The following topics are discussed: FlashCopy operational areas. FlashCopy basic concepts. FlashCopy in combination with other Copy Services. Data set level FlashCopy. © Copyright IBM Corp.
6.1 Operational environments It takes only a few seconds to establish the FlashCopy relationships for tens to hundreds or more volume pairs. The copy is then immediately available for both read and write access. In a 24x7 environment, the quickness of the FlashCopy operation allows us to use FlashCopy in very large environments and to take multiple FlashCopies of the same volume for use with different applications. Some of the different uses of FlashCopy are shown in Figure 6-1.
Integration system New application releases (for example, SAP® releases) are likely to be tested prior to putting them onto a production server. By using FlashCopy, a copy of the production data can be established and used for integration tests. With the capability to reverse a FlashCopy, a previously created FlashCopy can be used within seconds to bring production back to the level of data it had at the time when the FlashCopy was taken. 6.
While the FlashCopy relationship is being created, the DS6000 holds off the I/O activity to the volume for an interval of time by putting the source volume in an extended long busy condition. No user intervention is required. I/O activity resumes when the FlashCopy establish process is completed. If all bits for the bitmap representing the target are set to their initial values, this means that no data block has been copied so far. The data in the target is not modified during setup of the bitmaps.
Reading from the source The data is read immediately (see Figure 6-3 on page 46). Writing to the source Whenever data is written to the source volume while the FlashCopy relationship exists, the storage subsystem makes sure that the time-zero-data is copied to the target volume prior to overwriting it in the source volume. To identify if the data of the physical block on the source volume needs to be copied to the target volume, the bitmap is analyzed.
6.3.1 Full volume copy When the copy option is invoked and the establish process completes, a background process is started that copies all data from the source to the target. Once this process is finished and if there were no updates on the target, the picture we get is similar to the one in Figure 6-5. If not explicitly defined as persistent, the FlashCopy relationship ends as soon as all data is copied.
occurred on the source or on the target will stay as they were at the time when the FlashCopy was established. If the persistent FlashCopy option was specified, the FlashCopy relationship must be withdrawn explicitly. 6.4 FlashCopy in combination with other Copy Services Volume-based FlashCopy can be used in various combinations with other Copy Services, whereas the most suitable will depend on the characteristics of the environment and the requirements.
– If you create the FlashCopy first and then do a Metro Mirror of the FlashCopy target, you must monitor the progress of the FlashCopy background copy. In this case the following considerations apply: • • The Metro Mirror secondary will not be in a fully consistent state until the FlashCopy background copy process is complete. Use the copy option to ensure that the entire FlashCopy source volume data is copied to the Metro Mirror secondary.
The Global Copy secondary will not be in a fully consistent state until the FlashCopy background copy process is complete and the Global Copy is forced to the full duplex state. Issue the TSO CESTPAIR command with OPTION(SYNC) to force the Global Copy to enter the full duplex state. Use the copy option to ensure that the entire FlashCopy source volume data is copied to the Global Copy secondary. On the Global Copy secondary site a FlashCopy source volume can be based on the secondary Global Copy volume. 6.
FlashCopy volume source dataset target dataset Figure 6-10 Source data set and target data set can reside in the same volume 52 IBM System Storage DS6000 Series: Copy Services with IBM System z
7 Chapter 7. FlashCopy options This chapter discusses the options of FlashCopy when working with IBM System Storage DS6000 series in a System z environment. The following options are explained: Multiple relationship FlashCopy Consistency Group FlashCopy FlashCopy on existing Metro Mirror or Global Copy source Incremental FlashCopy Remote FlashCopy Persistent FlashCopy Data set FlashCopy Reverse restore and fast reverse restore © Copyright IBM Corp. 2006. All rights reserved.
7.1 Multiple relationship FlashCopy It is possible to establish up to 12 FlashCopy relationships using the same source. In other words, a source volume can have up to 12 target volumes. However, a target volume can still only have one source. Furthermore, cascading FlashCopy is not allowed (that is, a volume cannot be both a source and a target volume). Following is a summary of the considerations that apply: A FlashCopy source volume can have up to 12 FlashCopy target volumes.
API support data set level FlashCopy, the VTOC and catalogs are not updated as they are with DFSMSdss. 7.2 Consistency Group FlashCopy Applications might have their data spread over multiple volumes. In this case, if FlashCopy needs to be used for multiple volumes, these all have to be at a consistent level.
target source FlashCopy Local site Metro Mirror or Global Copy primary Remote site secondary Metro Mirror or Global Copy Figure 7-2 FlashCopy target is Metro Mirror (or Global Copy) primary It is possible to create either the FlashCopy relationship first or the Metro Mirror (or Global Copy) relationship first. However, in general it is better to create the FlashCopy relationship first to avoid the initial sending of unnecessary data across to the Metro Mirror (or Global Copy) secondary. 7.
FlashCopy with copy option If the original FlashCopy was established with the copy option (full volume copy), then the updates that took place on the source volume since the last FlashCopy will be copied to the target volume. Also, the updates done on the target volume will be overwritten with the contents of the source volume. .
All three bitmaps are necessary for incremental FlashCopy: Target bitmap - This bitmap keeps track of tracks not yet copied from source to target. Source Change Recording bitmap - This bitmap keeps track of changes to the source. Target Change Recording bitmap - Yhis bitmap keeps track of changes to the target. These bitmaps allow subsequent FlashCopies to transmit only those blocks of data for which updates occurred.
7.5 Remote FlashCopy There are command interfaces (see Figure 7-8 on page 61) which are able to manage a FlashCopy relationship at a remote site. The commands can be issued from the local site and they are then transmitted over the Metro Mirror or Global Copy links. This eliminates the need for a network connection to the remote site solely for the management of FlashCopy. The FlashCopy source volume at the remote site must be the secondary volume of the Metro Mirror or Global Copy pair.
7.8 Reverse restore With this option the FlashCopy relationship can be reversed by copying over modified tracks from the target volume to the source volume (see Figure 7-7). The background copy process must complete before you can reverse the order of the FlashCopy relationship to its original source and target relationship. Change recording is a prerequisite for reverse restore.
Interface Function DS front ends zOS front ends DS SM DFSMSdss DS CLI TSO ANTRQST ICKDSF Multiple relationship FlashCopy Consistency Group FlashCopy 3 Target on existing Metro Mirror or Global Copy primary Incremental FlashCopy 3 Remote FlashCopy Persistent Flashcopy 2 3 Dataset FlashCopy 2 2 1 1 Reverse restore, fast reverse restore (1) Extents can be specified, but the VTOC and the catalogs are not updated (2) Persistent relationships are available via Incremental support (3) With z/OS
62 IBM System Storage DS6000 Series: Copy Services with IBM System z
8 Chapter 8. FlashCopy ordering and activation This chapter explains how to order and activate FlashCopy for the IBM System Storage DS6000. The information presented here can be complemented with the information in Appendix C, “Licensing” on page 525. © Copyright IBM Corp. 2006. All rights reserved.
8.1 Ordering FlashCopy FlashCopy is an optional licensed function of the IBM System Storage DS6000 series. This function is also referred to as the Point-In-Time Copy (PTC) licensed function. Note: For a detailed explanation of the features involved and the factors to be considered when ordering FlashCopy we recommend you refer to the IBM System Storage DS6000 Series (IBM 1750-522) announcement letter. IBM announcement letters can be found at: http://www.ibm.
8.2 Activating FlashCopy IBM System Storage DS6000 licensed functions are managed using the Disk Storage Feature Activation (DSFA) application. Also, the DS6000 licensed functions are activated by the installation of feature activation codes into the 1750 system. The feature activation codes are made available by IBM and are obtained using the DSFA (Disk Storage Feature Activation) application at: http://www.ibm.
Example 8-1 FlashCopy example A DS6000 has a total physical capacity of 15 TB and that capacity will be configured as: 10 TB open systems (FB) 5 TB System z (CKD) Then, here's the required licenses: Operating environment ==> 15 TB (equal to total machine capacity) Parallel access volumes ==> 5 TB (equal to CKD-configured capacity) Remote mirror for z/OS ==> 5TB (equal to CKD-configured capacity) FlashCopy must either be: 10 TB if you want to use FlashCopy only with open systems (FB) data (equal to FB-config
You can also reactivate a deactivated licensed function by changing the license value to a non-zero value. A new feature activation code will be generated and when you install it into the machine, the function will be activated. Reactivation is a nondisruptive activity. When you deactivate a licensed function using the DSFA application, the 5xxx feature numbers assigned to your machine (using the order confirmation code) remain assigned to the machine.
68 IBM System Storage DS6000 Series: Copy Services with IBM System z
9 Chapter 9. FlashCopy interfaces The setup of FlashCopy in a System z environment can be done using different interfaces. This chapter explains and gives examples of the interfaces that can be used for FlashCopy management when FlashCopy is used with the IBM System Storage DS6000 in a System z environment. © Copyright IBM Corp. 2006. All rights reserved.
9.1 FlashCopy interfaces - overview There are various interfaces that can be used for the configuration and management of FlashCopy when used in a System z environment with the DS6000. There are the DS6000 front end-provided interfaces: DS Storage Manager (DS SM) - A graphical user interface (GUI) running in a Web browser that communicates with the DS6000 Storage Management Console (DS SMC).
9.2 DS CLI and DS SM - commands and options This section summarizes the commands and select options you can use when managing FlashCopy at the local and at the remote site. 9.2.1 Local FlashCopy management The commands you can use as well as the displayed panel actions and options you can select when working with the DS6000-provided interfaces DS CLI and DS SM for local FlashCopy management are listed in Table 9-1.
9.2.2 Remote FlashCopy management The commands that can be used when working with the DS6000-provided interface DS CLI for remote FlashCopy management are listed in Table 9-2.
Options Command with DFSMSdss Command with TSO Command with ICKDSF FCQUERY FLASHCPY QUERY RELATION Work with an existing FlashCopy Display a list of FlashCopy relationships Relocate data set extents on a DASD volume DEFRAG - Provide information why DFSMSdss couldn’t use FlashCopy DEBUG - DUMP with FCWITHDRAW parameter FCWITHDR Terminate FlashCopy Remove local FlashCopy FLASHCPY WITHDRAW Is automatically removed as soon as all data is copied and FlashCopy pair wasn’t established using the -pe
This section discusses the parameters that can be passed to FlashCopy when using the DS CLI and what the results are. Figure 9-1 summarizes the parameters and the corresponding DS CLI commands. When FlashCopy receives these parameters, the following actions result: freeze: Consistency Group FlashCopy With the DS CLI, it is possible to establish a Consistency Group by using the -freeze parameter and identifying all FlashCopy pairs and target volumes belonging to it.
Allows you to issue the reverseflash command before the background copy finished. cp: restrict command to FlashCopy relationships with background copy sourceLSS: reset Consistency Group for source logical subsystems s: display of FlashCopy pairs with lsflash command The shortened output of the lsflash command is returned. Only the FlashCopy pair IDs are displayed. l: display additional FlashCopy information The standard output of the lsflash command is enhanced.
The following explanations apply to the cases presented in Example 9-1: Example 1: 0000 → 0100 The FlashCopy between volume 0000 and volume 0100 is established with the property BackgroundCopy enabled. This is the default unless specified differently using the -nocp parameter. All other properties are disabled. The background copy takes place immediately. Once the copy is done, the FlashCopy relationship is automatically removed.
Example 9-2 lsflash command examples #--- Example 1 lsflash -dev IBM.1750-13ABC2A 0004 Date/Time: July 8, 2005 3:00:49 PM CEST IBM DSCLI Version: 5.0.3.134 DS: IBM.
==================================================================================================================================== 0000:0100 00 0 300 Disabled Disabled Disabled Disabled Disabled Disabled Enabled 0001:0101 00 1 300 Disabled Disabled Disabled Disabled Disabled Disabled Enabled 0002:0102 00 2 300 Disabled Enabled Enabled Disabled Disabled Disabled Enabled 0003:0103 00 3 300 Disabled Disabled Enabled Disabled Disabled Disabled Enabled The following explanations apply to the cases presented i
Example 9-3 setflashrevertible command examples #-----------------------------------------------------------#--- script to set FlashCopy property Revertible to value enabled and display values afterwards #-----------------------------------------------------------#--- Example 1 setflashrevertible -dev IBM.1750-13ABC2A 0002:0102 Date/Time: July 8, 2005 6:52:49 PM CEST IBM DSCLI Version: 5.0.3.134 DS: IBM.1750-13ABC2A CMUC00167I setflashrevertible: FlashCopy volume pair 0002:0102 successfully made revertible.
commitflash -dev IBM.1750-13ABC2A 0001-0005 Date/Time: July 10, 2005 2:46:18 PM CEST IBM DSCLI Version: 5.0.3.134 DS: IBM.1750-13ABC2A CMUC00170I commitflash: FlashCopy volume pair 0001:0001 successfully committed. CMUC00170I commitflash: FlashCopy volume pair 0005:0005 successfully committed. lsflash -dev IBM.1750-13ABC2A -l 0000-0005 Date/Time: July 10, 2005 2:46:47 PM CEST IBM DSCLI Version: 5.0.3.134 DS: IBM.
resyncflash -dev IBM.1750-13ABC2A -record -persist -seqnum 11 0001:0101 0005:0105 Date/Time: July 10, 2005 5:05:58 PM CEST IBM DSCLI Version: 5.0.3.134 DS: IBM.1750-13ABC2A CMUC00168I resyncflash: FlashCopy volume pair 0001:0101 successfully resynchronized. CMUC00168I resyncflash: FlashCopy volume pair 0005:0105 successfully resynchronized. resyncflash -dev IBM.1750-13ABC2A seqnum 13 0003:0103 Date/Time: July 10, 2005 5:06:25 PM CEST IBM DSCLI Version: 5.0.3.134 DS: IBM.
Example 9-6 reverseflash command examples #--- Example 1 lsflash -dev IBM.1750-13ABC2A 0000-0005 Date/Time: July 11, 2005 5:08:49 AM CEST IBM DSCLI Version: 5.0.3.134 DS: IBM.
Example 2: reverse a FlashCopy relationship multiple times. It is possible to reverse a FlashCopy relationship multiple times, thus recopying the contents of the original FlashCopy target volume multiple times back to the original source volume. In this example the 0002:0102 was reversed once as part of example 1. Then changes are made to data residing on volume 0002.
Example 9-8 rmflash command to create new background copy #--- Example 1 rmflash -dev IBM.1750-13ABC2A -quiet -cp 0001:0101 Date/Time: July 11, 2005 11:21:05 AM CEST IBM DSCLI Version: 5.0.3.134 DS: IBM.1750-13ABC2A CMUC00143I rmflash: Background copy process for FlashCopy pair 0001:0101 successfully started.
Reset FlashCopy Consistency Group using unfreezeflash The command unfreezeflash can be used to remove a Consistency Group for multiple volumes for which FlashCopy relations were established using the -freeze parameter. This command removes the long busy condition and allows I/Os to continue for the source volumes. See Example 9-11. The unfreezeflash command is issued to the entire logical subsystem (LSS). Example 9-11 unfreezeflash command example #--- Example 1 unfreezeflash -dev IBM.
9.5.
9.6 FlashCopy management using the DS SM To use the DS SM front end GUI, a supported Web Browser needs to be installed on the workstation. The DS SM communicates with the DS SMC. To start using the DS SM, enter in your Web browser the IP address of the DS SMC, as shown in Example 9-12. The password is set up by the administrator and needs to be changed during the first use of the DS SM.
4. Select common options Figure 9-4 on page 88 shows the options that can be chosen. The data provided in this window will be used for all defined FlashCopy pairs. Figure 9-4 FlashCopy options 5. Verification. In the following verification window, all information regarding the FlashCopy definition is displayed. If it isn’t possible to establish the FlashCopy relationship (if, for example, a volume was requested to be offline but it is not offline) then an information message is displayed on this window.
Options Parameter with DS CLI command mkflash Parameter with DS SM FlashCopy create Remark FlashCopy target can be Metro Mirror or Global Copy primary tgtpprc Establish target on existing Metro Mirror source Permit target volume to be online in z/OS environment tgtoffline Permit FlashCopy to occur if target is online for host access Only for CKD volumes (z/OS) Inhibit writes to target volume tgtinhibit - Resync target for z/OS Identification of FlashCopy pair dev or source:target Selected i
Figure 9-5 FlashCopy display properties Note: In case you are selecting multiple FlashCopy relationships, the Select Action → Properties will not be presented. Select only one FlashCopy relationship to view its properties. This selection will give you a window, with two folders: General In this folder all properties of the selected FlashCopy are presented. See Figure 9-6.
Out-of-synch tracks The window displaying the out-of-synch tracks can be used to monitor how the FlashCopy performs in the background; see Figure 9-7. A refresh interval can be set to refresh the display after a preselected period of time. Figure 9-7 Out-of-synch tracks folder Properties display - DS CLI vs.
Properties with DS CLI command lsflash Properties with DS SM FlashCopy Persistent Relationship will remain Enabled or disabled property Yes or no Identifies if the FlashCopy relationship can be changed by copying the contents of the target volume to the source volume Revertible Enabled or disabled Restorable Yes or no Source is write inhibited Yes or no Identifies if the source can be used to write on it SourceWriteEnabled Enabled or disabled Identifies it the target can be used by another se
Figure 9-8 Properties that can be changed with the reverse action 9.6.4 Initiate background copy for a persistent FlashCopy relationship To initiate a background copy for a persistent relationship, start on the DS SM front end window and select Real Time Manager on the left, then FlashCopy. Identify the DS6000 (Storage Unit and LSS, for example) for which you would like to increment a FlashCopy. This will give you a list of all active FlashCopy relationships similar to Figure 9-9 on page 94.
Figure 9-9 Initiate background copy The next window displayed is shown in Figure 9-10. It prompts for the FlashCopy pairs for which the FlashCopy should run. Figure 9-10 Prompt window for background copy 9.6.5 Resynchronize target To resynchronize a target volume, start by displaying the list of active FlashCopy relationships as shown in Figure 9-11 on page 95. Then check the box at the left of the FlashCopy you want to resynchronize.
Figure 9-11 Resynchronize the FlashCopy relationship The following prompt window asks for more details for the resync request; see Figure 9-12. Figure 9-12 Prompt window to detail resync request for FlashCopy relationship Chapter 9.
9.6.6 Delete existing FlashCopy relationship To delete an existing FlashCopy relationship, start by displaying the list of active FlashCopy relationships, as shown in Figure 9-13. Then check the box at the left of the FlashCopy relationship you want to terminate. Doing so, the available Select Actions for this FlashCopy relationship will be shown. Then select Delete.
9.7 z/OS interfaces for local FlashCopy The following z/OS-provided interfaces can be used for FlashCopy management: DFSMSdss utility TSO commands ANTRQST macro ICKDSF utility 9.7.1 Initiating FlashCopy using DFSMSdss DFSMSdss has the COPY command that uses FlashCopy for volume copies and data set copies. Detailed information can be found in z/OS DFSMSdss Storage Administration Reference, SC35-0424.
Table 9-9 Options and parameters used for FlashCopy with DFSMSdss COPY FULL Options Parameter with DFSMSdss COPY FULL Remark Options for the source volume Multiple relationship FlashCopy selected pairs Consistency Groups for FlashCopy FCCGFREEZE CGCREATE FCCGVERIFY see note (1) Options for the target volume FlashCopy target can be Metro Mirror or Global Copy primary FCTOPPRCPrimary Permit target volume to be online —z/OS environment no specific parameter required Inhibit writes to target volume
Group has been formed. During the time period between the first and last volumes being frozen, no dependent write updates will occur, which allows a consistent copy of logically related data that spans multiple volumes. Freezing the source volumes: You can use the FCCGFREEZE keyword on the COPY FULL or COPY TRACKS CPVOLUME command to specify that the FlashCopy source volume is to be part of a FlashCopy Consistency Group.
Incremental FlashCopy, Persistent FlashCopy, Change Recording option Incremental FlashCopy operates at the full volume level. You can use Incremental FlashCopy to create an initial point-in-time copy of a source volume and refresh the target volume by copying only the changed data.
FCNOCOPY to get a point-in-time copy and then perform FCNOCOPYTOCOPY later to start background copy. If you want a permanent copy and do not want to delay background copy, do not specify FCNOCOPY. FlashCopy as preferred copy method for DFSMSdss The parameter FASTREPLICATION can be used to identify whether the DS6000 FlashCopy should be used or not. See Table 9-11 for the possible values and the corresponding actions.
Table 9-13 Data mover for data set copy (to device of same geometry) Data set type Data mover Notes Sequential DFSMSdss Partitioned (not PDSE) DFSMSdss / IEBCOPY All partitioned data sets that are not load modules are compressed during a copy. Specify NOPACKING if FlashCopy is to be used. Partitioned (not PDSE) load modules DFSMSdss / IEBCOPY If copying partitioned load modules with REBLOCK, DFSMSdss calls IEBCOPY to copy the data set to a like device.
Data set type Data mover Integrated catalog facility user catalogs IDCAMS (EXPORT/IMPORT) Undefined DSORG DFSMSdss Notes 9.7.2 FlashCopy using TSO commands In this section we describe the FlashCopy functions invoked via TSO commands and parameters. For more detailed information about the TSO FlashCopy commands, refer to DFSMS Advanced Copy Services, SC35-0428.
Example 9-14 Incremental FlashCopy invoked with FCESTABL and INCREMENTAL(YES) //********************************************************************* //* ESTABLISH FLASHCOPY RELATIONSHIP * //* SDEVN - SOURCE FLASHCOPY VOLUME * //* TDEVN - TARGET FLASHCOPY VOLUME * //********************************************************************* //STEP1 EXEC PGM=IKJEFT01,REGION=256K //SYSTSPRT DD SYSOUT=* //SYSUADS DD DSN=SYS1.UADS,DISP=SHR //SYSLBC DD DSN=SYS1.
All specified source device tracks are updated. The DS6000 copies all tracks from source to target when a threshold number of source tracks are updated. A FlashCopy withdraw (using the FCWITHDR command) is issued to remove the FlashCopy relationship. A NOCOPY relationship is converted to a COPY relationship through the NOCOPY2COPY mode option.
FCESTABL SDEVN(X'3501') TDEVN(X'4501') MODE(NOCOPY2COPY) FlashCopy to Metro Mirror or Global Copy primary volume This FlashCopy option allows you to establish a FlashCopy pair where the FlashCopy target is the primary device in a Metro Mirror (or Global Copy) pair. In this way you can create a point-in-time copy and then make a copy of that point-in-time copy at a remote site.
Inband FlashCopy can be useful if the host at the recovery site is not online. The Inband option eliminates the need for a host connection from local to remote exclusively for FlashCopy backup. The FlashCopy request must be issued at a host processor connected to the remote mirror or copy primary volume, with the remote mirror or copy secondary volume specified as the FlashCopy source.
As shown in Example 9-20, Fast Reverse Restore is invoked with the FCESTABL command and the ACTION(FRR) parameter.
Example 9-21 FlashCopy withdraw target only //********************************************************************* //* WITHDRAW FLASHCOPY RELATIONSHIP * //* TARGET ONLY * //* TDEVN - TARGET FLASHCOPY VOLUME * //********************************************************************* //STEP1 EXEC PGM=IKJEFT01,REGION=256K //SYSTSPRT DD SYSOUT=* //SYSUADS DD DSN=SYS1.UADS,DISP=SHR //SYSLBC DD DSN=SYS1.
/* The main benefit of the DDSW(YES) parameter is that it can be used to easily free up tracks on both the source and the corresponding target volumes that are in FlashCopy relationships that are no longer needed because the original source data—that justified the relationship—has been deleted. Another example might be that prior to starting a backup cycle using DFSMSdss, you might want to make sure that all relationships have been cleaned up on the subject source volume.
10 Chapter 10. FlashCopy performance This chapter discusses best practices when configuring FlashCopy for specific environments or scenarios. The following topics are covered: FlashCopy performance overview FlashCopy establish performance Background copy performance FlashCopy impact to applications FlashCopy options FlashCopy scenarios © Copyright IBM Corp. 2006. All rights reserved.
10.1 FlashCopy performance overview Many parameters can affect the performance of FlashCopy operations. It is important to review the data processing requirements and hence select the proper FlashCopy options. This chapter examines when to use the COPY versus the NOCOPY mode and where to place the FlashCopy source and target volumes/LUNs. We also discuss when and how to use incremental FlashCopy—which should definitely be evaluated for use in most applications.
Tip: If the FlashCopy target volume is on the same rank as the FlashCopy source volume, you run the risk of a rank failure causing the loss of both the source and the target volume.
10.2 FlashCopy establish phase performance The FlashCopy of a volume has two distinct periods: The initial logical FlashCopy (also called establish) The physical FlashCopy (also called background copy) The FlashCopy establish phase, or logical FlashCopy, is the period of time when the microcode is preparing things, such as the bitmaps, necessary to create the FlashCopy relationship so the microcode can properly process subsequent reads and writes to the related volumes.
Tip: The DS6000 gives higher priority to application performance than background copy performance. This means that the DS6000 will throttle the background copy if necessary, so that applications are not unduly impacted. The recommended placement of the FlashCopy source and target volumes, regarding the physical FlashCopy phase, was already discussed in the previous section. Refer to Table 10-1 on page 113 for a summary of the conclusions.
typically the preferred option. However, some workloads that contain a large number of random writes and are not cache friendly might benefit from using the COPY option, which is typically the preferred option. Another important performance consideration is whether or not to use incremental FlashCopy.
When backing up to disk, it is important to take the necessary steps to protect your data. Remember that, until the background copy is complete, you still only have one physical copy of the data, and that copy is vulnerable. Therefore, it is important to always establish the FlashCopy with the COPY option. Otherwise, in the unlikely event you have a failure of your production volumes, you will also lose your backup. One method for protecting against failure is to use multiple FlashCopy targets.
type of workload. This topic has been discussed in 10.4, “FlashCopy impact on applications” on page 115. Still, in general, NOCOPY is the preferred method, but you should also think about the following considerations when choosing either COPY or NOCOPY: Using NOCOPY - The argument here is that the impact caused by the I/O resulting from the COPY option is more significant than that of the NOCOPY option, where less I/O activity resulting from collisions occur.
of the point-in-time refresh to complete the copy of the changed data. Finally, take the required point-in-time copy with the incremental refresh at the required point in time. 10.6.4 Scenario #4: Ranks reserved for FlashCopy Another configuration worth considering is the one where 50% of the ranks (capacity) are all FlashCopy source volumes—and where the application write I/Os take place—and the remaining 50% of the ranks (capacity) are all FlashCopy target volumes.
120 IBM System Storage DS6000 Series: Copy Services with IBM System z
11 Chapter 11. FlashCopy examples This chapter presents examples of the use of FlashCopy in the following scenarios: Fast setup of test systems or integration systems Fast creation of volume copies for backup purposes © Copyright IBM Corp. 2006. All rights reserved.
11.1 Create a test system or integration system Test systems or integration systems are needed to perform application tests or system integration tests. As it is assumed that with test systems or integration systems many write operations will occur over the time period involved, we recommend doing a copy in the background environment. 11.1.
Example 11-2 Create a multiple setup of a test system Part 1 //********************************************************************* //* ESTABLISH FLASHCOPY RELATIONSHIP * //* SDEVN - SOURCE FLASHCOPY VOLUME * //* TDEVN - TARGET FLASHCOPY VOLUME * //********************************************************************* //STEP1 EXEC PGM=IKJEFT01,REGION=256K //SYSTSPRT DD SYSOUT=* //SYSUADS DD DSN=SYS1.UADS,DISP=SHR //SYSLBC DD DSN=SYS1.
Example 11-3 Create a backup copy using TSO FlashCopy commands Part 1 //********************************************************************* //* ESTABLISH FLASHCOPY RELATIONSHIP WITH NOCOPY * //* SDEVN - SOURCE FLASHCOPY VOLUME * //* TDEVN - TARGET FLASHCOPY VOLUME * //********************************************************************* //STEP1 EXEC PGM=IKJEFT01,REGION=256K //SYSTSPRT DD SYSOUT=* //SYSUADS DD DSN=SYS1.UADS,DISP=SHR //SYSLBC DD DSN=SYS1.
Example 11-5 Create a full volume copy with incremental parameter //********************************************************************* //* ESTABLISH INCREMENTAL FLASHCOPY RELATIONSHIP * //* SDEVN - SOURCE FLASHCOPY VOLUME * //* TDEVN - TARGET FLASHCOPY VOLUME * //********************************************************************* //STEP1 EXEC PGM=IKJEFT01,REGION=256K //SYSTSPRT DD SYSOUT=* //SYSUADS DD DSN=SYS1.UADS,DISP=SHR //SYSLBC DD DSN=SYS1.
Example 11-7 Reverse the FlashCopy volumes Part 2 //******************************************************** //* THIS JOB WILL REVERSE THE DIRECTION OF FLASHCOPY //* ORIGINAL SOURCE = 3200, FRR SOURCE = 3400 //* ORIGINAL TARGET = 3400, FRR TARGET = 3200 //******************************************************** //STEP1 EXEC PGM=IKJEFT01,REGION=256K //SYSTSPRT DD SYSOUT=* //SYSUADS DD DSN=SYS1.UADS,DISP=SHR //SYSLBC DD DSN=SYS1.
Part 4 Part 4 Metro Mirror This part of the book describes IBM System Storage Metro Mirror for DS6000 when used in a System z environment. Here we discuss the characteristics of Metro Mirror and describe the options for its setup. We also show which management interfaces can be used, as well as the important aspects to be considered when establishing a Metro Mirror environment. This part ends with examples of Metro Mirror management and setup. © Copyright IBM Corp. 2006. All rights reserved.
128 IBM System Storage DS6000 Series: Copy Services with IBM System z
12 Chapter 12. Metro Mirror overview This chapter explains the basic characteristics of Metro Mirror for DS6000 when used in a System z environment. © Copyright IBM Corp. 2006. All rights reserved.
12.1 Metro Mirror overview Metro Mirror (previously known as synchronous Peer-to-Peer Remote Copy, or PPRC) provides real-time mirroring of logical volumes between two DS6000s that can be located up to 300 km from each other. It is a synchronous copy solution where write operations are completed on both copies (local and remote site) before they are considered to be complete. It is typically used for applications that cannot suffer any data loss in the event of a failure.
12.2 Metro Mirror volume state Volumes participating in a Metro Mirror session can be found in any of the following states: Copy pending: volumes are in copy pending state after the Metro Mirror relationship is established, but the primary and secondary volumes are still out of sync. In that case, data still needs to be copied from the primary to the secondary volume of the Metro Mirror pair.
Data consistency, dependent writes, extended long busy, are all discussed in detail in 13.3, “Consistency Group function” on page 136. Note: Dduring normal Metro Mirror processing, the data on disk at the secondary site is an exact mirror of that at the primary site. During or after an error situation this depends on the options specified for the pair and the path. Remember that any data still in buffers or processor memory is not yet on disk and so will not be mirrored to the secondary site.
13 Chapter 13. Metro Mirror options and configuration This chapter discusses the options available when using Metro Mirror for DS6000. It also discusses the configuration guidelines that should be considered when planning the Metro Mirror environment. © Copyright IBM Corp. 2006. All rights reserved.
13.1 High availability solutions Because the disk subsystem attached to the server is being mirrored using Metro Mirror, this offers some improved opportunities for high availability solutions. 13.1.1 GDPS HyperSwap Manager The GDPS services offering includes the GDPS HyperSwap™ Manager function.
The switch back is completed with one more sequence of a Metro Mirror Failover followed by a Metro Mirror Failback operations, both given at the now-recovered production site. Figure 13-1 summarizes the process. Primary site (A) Normal operation site (A) production site site (B) recovery site When planned/unplanned outage at (A): At (B): Metro Mirror Failover restart application processing With site (A) recovered and operable. At (B): 1. Metro Mirror Failback 2.
13.3 Consistency Group function In order to restart applications at the remote site successfully, the remote site must have consistent data. In normal operation, Metro Mirror keeps data consistency at the remote site. However, as mentioned in 12.3, “Data consistency” on page 131 and in 12.4, “Rolling disaster” on page 132, in case of a rolling disaster, a particular procedure is necessary to keep data consistency even in a synchronous remote copy environment.
In other words, the order of updates was not the same as it was for the source data, but the order of dependent writes was preserved. 13.3.2 Consistency Group function - how it works In the operation of the Consistency Group function of Metro Mirror we distinguish two parts. One is the invocation of the Consistency Group option, and the other one is the freeze/run operation. Together, they make it possible for the disk subsystem to hold I/O activity and subsequently to thaw the held I/O activities.
1st LSS11 LSS21 LSS12 LSS22 LSS13 LSS23 Wait Application on Servers 2nd Wait 3rd Wait dependent write operation Source DS6000 Target DS6000 Figure 13-2 Consistency Group: Example 1 In Figure 13-2, there are two Metro Mirror paths between LSS11 and LSS21. There are another two Metro Mirror paths for each of the other LSS pairs (LSS12;LSS22 and LSS13:LSS23). In a disaster, the paths may fail at different times.
1st LSS11 LSS21 LSS12 LSS22 LSS13 LSS23 Completed Application on Servers 2nd Wait 3rd Wait dependent write operation Source DS6000 Target DS6000 Figure 13-3 Consistency Group: Example 2 In all cases, if each write operation is dependent, the Consistency Group option can keep the data consistent in the Metro Mirror target volumes until the Consistency Group time-out occurs. After the time-out value has been exceeded, all held I/O will be released.
1st LSS11 LSS21 LSS12 LSS22 LSS13 LSS23 Completed Application on Servers 2nd Wait 3rd Wait Completed independent write operations Source DS6000 Target DS6000 Figure 13-4 Consistency Group: Example 3 In this case, the Metro Mirror target volumes reflect only the first and third write operation, not including the second operation. Typical database management software can recover its databases by using its log files if dependent write operations are kept.
The ICKDSF PPRCOPY FREEZE and PPRCCOPY RUN commands provide the same functionality as the TSO CGROUP FREEZE and CGROUP RUN commands. If you are using the DS Command-Line Interface (DS CLI), you can use the freezepprc and unfreezepprc commands to provide the same functionality as the TSO CGROUP FREEZE and CGROUP RUN commands. It is not possible to issue Consistency Group (freeze/unfreeze) type commands from the DS Storage Manager Graphical User Interface.
The volume pair is suspended and further writes to the source volume are not accepted if data cannot be sent to the target volume. 13.3.5 Consistency Group and critical mode combination Table 13-1 shows what happens when the CRIT parameter of the CESTPAIR command, and the CGROUP parameter of CESTPATH command are combined. These options can also be set using the DS CLI, or with the DS Storage Manager GUI.
LSS 0 LSS 1 LSS 2 LSS 3 Physical Fibre Channel link : LSS 08 : LSS 0 LSS 1 LSS 2 LSS 3 : 256 logical paths per FCP link LSS nn LSS 08 : LSS nn Figure 13-5 Logical paths Paths are unidirectional, that is they are defined to operate in either one direction or the other. Still, Metro Mirror is bidirectional. That is, any particular pair of LSSs can have paths defined among them that have opposite directions—each LSS holds both source and target volumes from the other LSS.
Metro Mirror Pairs LSS1A LSS14 Server0 1A01 Server0 1A00 Fibre Channel links LSS1B DS6000#1 1401 LSS15 Server1 1B01 Server1 1B00 Fibre Channel links 1400 1500 1501 DS6000#2 Figure 13-6 DS6000 FC ports Dedicating Fibre Channel ports for Metro Mirror use guarantees no interference from host I/O activity. This is recommended with Metro Mirror, which is time-critical and should not be impacted by host I/O activity.
DS6000 1 DS6000 2 3-9 logical paths LSS 1 LSS 1 1 logical path 1 logical path switch LSS 2 1 logical path Port LSS 3 1 logical path 1 Link Metro Mirror paths Port LSS 2 1 logical path LSS 3 1 logical path Figure 13-7 Logical paths for Metro Mirror Metro Mirror paths have certain architectural limits, as follows: A primary LSS can maintain paths to a maximum of four secondary LSSs. Each secondary LSS can reside in a separate DS6000.
13.7 Distance The distance between your primary and secondary DS6000 subsystems will have an effect on the response time overhead of the Metro Mirror implementation. The maximum supported distance for Metro Mirror is 300 km. 13.8 Symmetrical configuration When planning your Metro Mirror configuration, consider the possibility of a symmetrical configuration, in terms of both physical and logical elements. This will have the following benefits: Simplifying management.
In addition to making the maintenance of the Metro Mirror configuration easier, the symmetrical implementation has the added benefit of helping to balance the workload across the DS6000. Figure 13-8 shows a logical configuration, but this idea applies equally to the physical aspects of the DS6000. You should attempt to balance workload and apply symmetrical concepts to other aspects of your DS6000 (for example, the Extent Pools). 13.
13.10 Hardware requirements Metro Mirror is an optional licensed function of the IBM System Storage DS6000 series. All DS6000 series licensed functions are enabled, authorized, managed, activated, and enforced based upon the physical capacity contained in a 1750 system. Particularly for Metro Mirror, the DS6000 must have the Remote Mirror and Copy (RMC) licensed function, with the corresponding feature #531x, based on the physical capacity.
14 Chapter 14. Metro Mirror interfaces This chapter discusses, and provides examples of, the interfaces that can be used for Metro Mirror management, setup and control when used with the IBM System Storage DS6000 in a System z environment. © Copyright IBM Corp. 2006. All rights reserved.
14.
Similar functions of the interfaces for Metro Mirror management Table 14-1 lists the commands and selections for the different interfaces that provide similar Metro Mirror control functions.
The TSO commands communicate with the DS6000 through a device number specified in the command. IP connectivity is not required. TSO commands can be integrated into REXX programs for automation purposes. Tip: if you are controlling open systems volumes with TSO commands, you must have a z/OS volume in the same server to direct your command to. For example, for an open systems LUN in LSS 16, a z/OS volume in LSS 00 can be used for the device address.
This command has parameters that allow it to indicate whether the operation is an initial establish of volumes that were in the simplex state, whether it is a resynchronization of a suspended pair of volumes, whether it is a Metro Mirror or Global Copy pair, or whether Failover or Failback processing is being requested. Example 14-1 shows the command necessary to establish an initial Metro Mirror pair.
14.2.4 CDELPAIR The CDELPAIR command is used to specify the primary and secondary volumes to be removed from a Metro Mirror pairing. This command should be directed to the primary device; see Example 14-3. Example 14-3 CDELPAIR command CDELPAIR DEVN(X'6400') PRIM(X'0002' AAVCA X'00' X'00') SEC(X'0003' AAVCA X'00' X'01') Before issuing a CDELPAIR command, you should verify that there are some active paths between the respective primary and secondary LSSs.
Tip: The CGROUP FREEZE command deletes the paths between the specified LSS pair. To restart normal Metro Mirror operation again, the paths will have to be reestablished and the pairs will have to be resynchronized. Example 14-5 illustrates the CGROUP command.
* SECONDARY.1 500507630EFFFCA0 * ******************************************************************** ANTP0001I CQUERY COMMAND COMPLETED FOR DEVICE 6400. COMPLETION CODE: 00 Also, from the output information, you can check the number of primary updated tracks, which at that moment were not reflected on the secondary volume (TRACKS OUT OF SYNC); there were none in our example. In addition, you can see the Metro Mirror volume state of the device and how many Metro Mirror paths are established.
Example 14-10 CRECOVER command CRECOVER DEVN(X’6400’) PRIM(‘X0002’ AAVCA X’00’ X’00’) SEC(X’0003’ AAVCA X’00’ X’01’) ID(VOL002 VOL001) MSGREQ(YES) In Example 14-10, the CRECOVER command brings device x’6400’ (on the recovery DS6000) to simplex state. It also changes the volume label from VOL002 to VOL001. 14.2.9 CSUSPEND This command is used to suspend Metro Mirror operations between a volume pair.
14.3.1 Metro Mirror management with ICKDSF Table 14-2 lists the Metro Mirror commands available with ICKDSF.
14.3.2 Display the Fibre Channel Connection Information Table The ANALYZE command allows the user to specify the WWNN of the secondary Metro Mirror volume connected by Fibre Channel links. The analyze pathing reports include the Fibre Channel Connection Information Table. This information indicates the potential connectivity of the Fibre Channel ports in the DS6000 where the I/O is issued to each system adapter port in the DS6000 that is specified by the secondary WWNN.
Example 14-17 ICKDSF DELPAIR example //IKJEFT01 JOB MSGLEVEL=(1,1),MSGCLASS=A,NOTIFY=user id //STEP01 EXEC PGM=ICKDSF //SYSPRINT DD SYSOUT=* //VOL1 DD UNIT=3390,VOL=SER=DS6400,DISP=SHR //SYSIN DD * PPRCOPY DELPAIR DDNAME(VOL1) PRI(X'0002',AAVCA,X'00') SEC(X'0003',AAVCA,X'00') LSS(X'00',X'01') 14.3.4 PPRCOPY DELPATH The DELPATH command is used to delete the defined Metro Mirror paths between a primary LSS and a secondary LSS.
This specifies the World Wide Node Name of the primary and secondary DS6000. Each WWNN is an 8-byte hexadecimal value X’wwwwwwwwwwwwwwww’ where X’pwwnn’ represents the primary WWNN and X’swwnn’ represents the secondary WWNN. 14.3.6 PPRCOPY ESTPAIR The ESTPAIR command is used to establish a Metro Mirror relationship between a primary and a secondary volume. Example 14-20 shows the PPRCOPY ESTPAIR command.
A host system that is attached only to a primary volume cannot obtain the status of the secondary volume for that pair. In the same way, a host that is attached only to the secondary volume cannot obtain the status of the primary volume. Example 14-22 shows an example of the PPRCOPY QUERY command.
ICK04029I DEVICE IS IN SUSPENDED PPRC STATE QUERY REMOTE COPY - PATHS PRIMARY CONTROL UNIT INFORMATION SERIAL WORLD WIDE NUMBER SSID LSS NODE NAME ------- ---- --- ---------------AAVCA 0002 00 500507630EFFFCA0 ICKDSF - MVS/ESA DEVICE SUPPORT FACILITIES 17.0 TIME: 17:48:02 SECONDARY CONTROL UNIT INFORMATION SERIAL NUMBER ------AAVCA SSID ---0003 WORLD WIDE LSS NODE NAME --- ---------------01 SSID ---0003 .... .... .... LSS --01 ... ... ... PATHS: 1ST: 2ND: 3RD: 4TH: SERIAL NUMBER ------AAVCA .......
ICK00002I ICKDSF PROCESSING COMPLETE. MAXIMUM CONDITION CODE WAS 0 14.3.9 PPRCOPY RECOVER The RECOVER command is used to allow the recovery system to regain control of a DASD volume. This command is issued from the recovery system. It signals the recovery site DS6000 to remove the volume from the Metro Mirror relationship (the volume becomes simplex), and thus gives the volume control back to the recovery system. During this process the volser can be verified and also the volume can be relabeled. 14.3.
Note: In order to run the job to refresh the VTOC of the secondary volume, the Metro Mirror volume pair must first be deleted. 14.4 DS Command-Line Interface Although the DS CLI will not run on a System z processor, it can be used to perform operations on mainframe disks if you have a supported server environment (for example, Windows XP). The DS CLI can be used to create scripts for setup and control of DS6000 functions. It is a flexible and powerful interface.
The port IDs for source and target. Up to eight port pair IDs can be specified. – The lsioport lists the ports on the system. – A port ID will have the format I0102, for example. 14.5.1 Set up a Metro Mirror environment This section gives examples of the commands you can use to set up a Metro Mirror environment. This includes the commands needed to set up the Metro Mirror paths, as well as the commands needed to create the Metro Mirror relationships.
-tgtlss the LSS number of the target I0003:I0003 one of the source and target port ID pairs mkpprc The mkpprc command establishes a Metro mirror volume pair. Example 14-28 illustrates how it is used. Example 14-28 mkpprc command dscli> mkpprc -remotedev IBM.1750-1300819 -type mmir -mode full 0600:0100 0601:0101 Date/Time: 23 November 2005 23:46:02 IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300247 CMUC00153I mkpprc: Remote Mirror and Copy volume pair relationship 0600:0100 successfully created.
14.5.3 Manage a Metro Mirror environment This section describes the commands you can use to manage a Metro Mirror environment. This includes the commands to manage Metro Mirror relationships and Consistency Groups, as well as commands to display information about Metro Mirror relationships and paths. freezepprc The freezepprc command is used to remove Metro Mirror paths and suspend volume pairs between a pair of LSSs, and is usually used to provide data consistency.
1/1 pair 0601:0101 state: Full Duplex. failoverpprc The failoverpprc command is used to start a failover operation. It changes a secondary device into a primary suspended device while leaving the primary device in its current state. This command succeeds even if the paths are down and the volume at the production site is unavailable or nonexistent. Example 14-35 shows how it can be used for multiple volume pairs.
Important: If you failback so that updates are copied from the remote disk subsystem to the local disk subsystem (in other words, from the backup site to the production site), then you must have the corresponding remote copy paths established from the remote site to the local site. If you do not have these paths defined, the command fails.
Example 14-36 failbackpprc command Following command issued to SMC at backup site dscli> failbackpprc -dev IBM.1750-1300819 -remotedev IBM.1750-1300247 -type mmir 0100:0600 0101:0601 Date/Time: 24 November 2005 0:39:55 IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300819 CMUC00197I failbackpprc: Remote Mirror and Copy pair 0100:0600 successfully failed back. CMUC00197I failbackpprc: Remote Mirror and Copy pair 0101:0601 successfully failed back.
14.6 DS Storage Manager GUI The DS Storage Manager (DS SM) is a graphical user interface (GUI) that can be used to set up and control Metro Mirror for DS6000 functions. It is user friendly; however, you cannot use it for automation activities, and certain remote mirror and copy functions are not supported from this interface. The DS Storage Manager interface is simpler to use, however, it is usually slower than the other interfaces, and you cannot save actions for later use.
Figure 14-2 Select source LSS panel Now you must select the target LSS, using the panel shown in Figure 14-3. From the menus provided, select the device to which you want to establish the path. In this example, we chose LSS 01 on a different DS6000. Then click Next. Figure 14-3 Select target LSS panel Next, you select the source I/O ports, using the panel shown in Figure 14-4 on page 174. Use the check boxes to select the I/O ports you want to use for the Metro Mirror paths that you are establishing.
Figure 14-4 Select source I/O ports Now you must define a target I/O port for each source port. In the example shown in Figure 14-5, there are no selections; however, for your setup you might see menus. Click Next. Figure 14-5 Select target I/O ports Next, the panel shown in Figure 14-6 on page 175 opens. Here you can indicate whether the paths will be a Consistency Group path by using the check box. Click Next.
Figure 14-6 Select path options panel Next, the verification panel (see Figure 14-7) opens. Here you can verify that everything has been selected correctly. If so, click Finish. If not, click Back or Cancel as appropriate. Figure 14-7 Create paths verification panel 14.6.2 Create Metro Mirror pairs To establish a Metro Mirror pair from the GUI, you can follow the process below.
Figure 14-8 Initial Metro Mirror panel, Create pull-down selected The panel shown in Figure 14-9 opens. Select either automated or manual volume pairing, and click Next. Figure 14-9 Select volume pairing method If you select Automatic volume pair assignment, then the panel in Figure 14-10 on page 177 opens. You then page through the list of volumes listed, and check the one you want to establish.
Figure 14-10 Automatic volume assignment source volume selection panel In our example, we selected volumes 0600 and 0601. After clicking Next, the panel shown in Figure 14-11 opens. On this panel you select a target volume. Because we selected automatic volume pairing, we are taken straight to the target LSS so that we can select the target volumes. Figure 14-11 Select target volume auto pairing In our example, we selected volumes 0100 and 0101 and clicked Next.
Figure 14-12 Metro Mirror options After making these selections, the Verification panel (see Figure 14-13) opens so that you can check your selections. If all is correct, you can click Finish. Figure 14-13 Metro Mirror pair verification panel The Metro Mirror panel indicating the successful establishment of the Metro Mirror pairs opens, as shown in Figure 14-14.
14.6.3 Resume suspended pair To resume a suspended pair from the DS Storage Manager, you can follow a procedure similar to the procedure described in this section. First, you must access the Metro Mirror panel. Then, select the volume you want to resume, then click Resume from the Select Action menu, as shown in Figure 14-15. Figure 14-15 Metro Mirror resume select volume and action The panel shown in Figure 14-16 opens. This is where you confirm that you want to resume the volume in question.
180 IBM System Storage DS6000 Series: Copy Services with IBM System z
15 Chapter 15. Metro Mirror performance and scalability In this chapter, we discuss performance and scalability considerations when using Metro Mirror for DS6000. © Copyright IBM Corp. 2006. All rights reserved.
15.1 Performance Because Metro Mirror is a synchronous mirroring technology, it introduces a performance overhead (for write operations) as compared to a similar environment with no remote mirroring. As part of the planning process for Metro Mirror, you should understand this. Bandwidth analysis and capacity planning for your Metro Mirror links can help define how many links you need, and when you need to add more, to ensure best possible performance.
15.2 Scalability The DS6000 Metro Mirror environment can be scaled up or down as required. If new volumes are added to the DS6000 that require mirroring, they can be dynamically added. If additional Metro Mirror paths are required, they also can be dynamically added. Remember that in the DS6000, you have only eight Fibre Channel ports available, four per controller. Therefore, we recommend that you use two Fibre Channel links exclusively for Metro Mirror, one on each controller, for availability.
184 IBM System Storage DS6000 Series: Copy Services with IBM System z
16 Chapter 16. Metro Mirror examples In this chapter we show examples using TSO, ICKDSF and DS CLI commands to manage Metro Mirror. The examples presented are: Resynchronization of a suspended volume pair with TSO commands Metro Mirror Failover and Failback procedure using TSO commands Define a Metro Mirror volume pair using TSO commands for open systems Define a Metro Mirror path using ICKDSF Use of the freezepprc and unfreezepprc DS CLI commands © Copyright IBM Corp. 2006.
16.1 Resynchronization of suspended volume using TSO To resynchronize a volume that is in a suspended state we need to obtain information such as serial numbers, WWNNs, CCA and LSS numbers. We can obtain this information using the CQUERY command, and then use it to reestablish the Metro Mirror pairs. Example 16-1 shows the CQUERY command and the output information it provides.
Example 16-2 Resume suspended Metro Mirror pair from TSO CESTPAIR DEVN(X'6400') PRIM(X'0002' AAVCA X'00' X'00') SEC(X'0003' AAVCA X'00' X'01') OPTION(SYNC) MODE(RESYNC) ANTP8802I CESTPAIR DEVN(X'6400') PRIM(X'0002' AAVCA X'00' X'00') SEC(X'0003' AAVCA X'00' X'01') OPTION(SYN ANTP8802I (CONT) C) MODE(RESYNC) ANTP0001I CESTPAIR COMMAND COMPLETED FOR DEVICE 6400.
Syslog messages Example 16-4 shows the syslog messages from a failover operation. We are doing a failover from device 6030 to device 6430. In this example, both primary and secondary devices are available to this processors LPAR; however, this might not be the case in all Metro Mirror environments. If both devices are not available from one processor LPAR, you must issue commands on the system where the device is located.
ANTP0090I CQUERY FORMATTED LVL 3 239 VOLUME REPORT ************** PPRC REMOTE COPY CQUERY - VOLUME ******************** * (PRIMARY) (SECONDARY) * * SSID CCA LSS SSID CCA LSS* *DEVICE LEVEL STATE PATH STATUS SERIAL# SERIAL# * *------ --------- ---------- ----------- ----------------- * * 6030 PRIMARY.. SUSPEND(3) ACTIVE.. 2060 30 00 0002 30 00 * * CRIT(NO)....... CGRPLB(NO).
recorded on the B volumes back to the A volumes. You must vary the A volumes offline, if they are online at the primary site. 4. If you established a FlashCopy relationship (to the C volumes), you can now withdraw it. 5. Check that Metro Mirror paths exist between the A and B volumes. Reestablish, if necessary. 6. Allow Metro Mirror pairs to reach full Duplex state (use the CQUERY command, or check the syslog for IEA494I messages). 7.
*DEVICE LEVEL STATE PATH STATUS SERIAL# SERIAL# * *------ --------- ---------- ----------- ----------------- * * 6030 PRIMARY.. SUSPEND(3) ACTIVE.. 2060 30 00 0002 30 00 * * CRIT(NO)....... CGRPLB(NO). 0000000AAGXA 0000000AAVCA* * PATHS PFCA SFCA STATUS: DESCRIPTION * * ----- --------- ------ ------------------* * 1 0000 0000 13 PATH ESTABLISHED... * * ---- ---00 NO PATH............ * * ---- ---00 NO PATH............ * * ---- ---00 NO PATH............
* SUBSYSTEM WWNN LIC LEVEL * * ----------- -------------------------* * PRIMARY.... 500507630EFFFCA0 5.0.00.0000 * * SECONDARY.1 500507630EFFFC6F * ******************************************************************** ANTP0001I CQUERY COMMAND COMPLETED FOR DEVICE 6030.
* CRIT(NO)....... CGRPLB(NO). 0000000AAVCA 0000000AAGXA* * PATHS PFCA SFCA STATUS: DESCRIPTION * * ----- --------- ------ ------------------* * 1 0100 0100 13 PATH ESTABLISHED... * * ---- ---00 NO PATH............ * * ---- ---00 NO PATH............ * * ---- ---00 NO PATH............ * * SUBSYSTEM WWNN LIC LEVEL * * ----------- -------------------------* * PRIMARY.... 500507630EFFFCA0 5.0.00.0000 * * SECONDARY.1 500507630EFFFCA0 * * SECONDARY.
* SECONDARY.1 500507630EFFFCA0 * * SECONDARY.2 500507630EFFFC6F * ******************************************************************** ANTP0001I CQUERY COMMAND COMPLETED FOR DEVICE 6430. COMPLETION CODE: 00 ANTP8802I CESTPAIR DEVN(X'6030') PRIM(X'2060' AAGXA X'30' X'00') SEC(X'0002' AAVCA X'30' X'00') ACTION(FAI ANTP8802I (CONT) LBACK) ANTP0001I CESTPAIR COMMAND COMPLETED FOR DEVICE 6030.
Establish a volume pair Similar to System z volumes, we first have to collect the required information. In this example, we already had a path defined to the secondary subsystem. We issued a CQUERY command with the ODEVN parameter to check the status of the volume, as seen in Example 16-8.
Example 16-11 CESTPAIR and CQUERY output ANTP8802I CESTPAIR DEVN(X'6030') PRIM(X'FF16' AAGXA X'00' X'16') SEC(X'FF16' AAVCA X'00' X'16') OPENDVCS(YES) ANTP8802I (CONT) OPTION(SYNC) MODE(COPY) ANTP0001I CESTPAIR COMMAND COMPLETED FOR DEVICE 6030.
Example 16-12 on page 196 shows an ICKDSF job that is defining the Metro Mirror path. Notice that the PPRCOPY ESTPATH command replaces any path definitions that currently exist. In Example 16-13 we show the output from the ESTPATH command. Example 16-13 PPRCOPY ESTPATH output ICKDSF - MVS/ESA DEVICE SUPPORT FACILITIES 17.
4TH: ....... .... ................ . .... .... 00 S*=PATH STATUS 00=NO PATH ...... 13=ESTABLISHED FIBRE CHANNEL PATH ...... ICK02206I PPRCOPY QUERY FUNCTION COMPLETED SUCCESSFULLY ICK00001I FUNCTION COMPLETED, HIGHEST CONDITION CODE WAS 0 16.5 DS CLI freezepprc and unfreezepprc commands When you want to suspend all Metro Mirror pairs (and remove paths) between a pair of LSSs, you use the freezepprc command (or its equivalent in the other interfaces).
Part 5 Part 5 Global Copy In this part of the book, we describe IBM System Storage Global Copy for DS6000. After presenting an overview of Global Copy, we discuss the options available, the interfaces you can use, and the configuration considerations. We also provide examples of the use of Global Copy. © Copyright IBM Corp. 2006. All rights reserved.
200 IBM System Storage DS6000 Series: Copy Services with IBM System z
17 Chapter 17. Global Copy overview In this chapter we describe the characteristics and operation of Global Copy. Also discussed are the considerations for its implementation with the IBM System Storage DS6000. © Copyright IBM Corp. 2006. All rights reserved.
17.1 Global Copy overview Global Copy (formerly known as PPRC Extended Distance, or PPRC-XD) is a non-synchronous remote copy function for System z and open systems for longer distances than are possible with Metro Mirror. It is appropriate for remote data migration, offsite backups, and transmission of inactive database logs at virtually unlimited distances.
17.2 Volume states and change logic Figure 17-2 illustrates the basic states and the change logic of a volume that is in either a Metro Mirror or Global Copy relationship. The following considerations apply to the volume states when the pair is a Global Copy pair: Simplex: The volume is not in a Global Copy relationship. Suspended: In this state, the writes to the primary volume are not mirrored onto the secondary volume. The secondary volume becomes out-of-sync.
You can also request that a pair be suspended as soon as it reaches the full-duplex state (see go to sync and suspend in Figure 17-2). Pairs cannot change directly from full duplex state to copy pending state. They must go through an intermediate suspend state. You can go from suspended state to copy pending state during an incremental copy (copying out-of-sync tracks only).
18 Chapter 18. Global Copy options and configuration This chapter discusses the options available when using Global Copy. It also discusses the configuration guidelines that should be considered when planning for Global Copy with the IBM System Storage DS6000. © Copyright IBM Corp. 2006. All rights reserved.
18.1 Global Copy basic options First we review the basic options available when working with Global Copy. You will see that many of these options are common to Metro Mirror. However, keep in mind that the results may differ because Metro Mirror and Global Copy have differences in the way they work. 18.1.1 Establish Global Copy pair This is the operation where you establish a Global Copy relationship between a primary (source) and a secondary (target) volume—that is, you establish a Global Copy pair.
18.1.4 Terminate Global Copy pair This operation ends the remote copy relationship between the volume pair; the volumes return to the simplex state. 18.1.5 Convert a Global Copy pair to Metro Mirror This operation is known as the Go-to-sync operation.
Example 18-1 TSO initiated go-to-sync transition CESTPAIR DEVN(X'400A') OPTION(SYNC) MODE(RESYNC) PRIM(X'4000' ABTV1 X'0A' X'00') SEC(X'8000' 20781 X'8A' X'80') Note: The CESTPAIR command does not support the go-to-SYNC and suspend operation (ICKDSF and the DS CLI do). When using the TSO interface, to suspend as soon as the duplex state is reached, you must automate the process by triggering on the state change messages that the system issues. 18.2.
18.2.4 Go-to-sync using the DS CLI You can also use the DS CLI command mkpprc with the option -mmir to synchronize a Global Copy pair, as shown in Example 18-3. Example 18-3 mkpprc command dscli> mkpprc -dev ibm.1750-13aagxa -remotedev ibm.1750-13aavca -type mmir 000d:000d Date/Time: 17. Juni 2005 00:01:00 CEST IBM DSCLI Version: 5.0.3.110 DS: IBM.1750-13AAGXA CMUC00153I mkpprc: Remote Mirror and Copy volume pair relationship 000D:000D successfully created. 18.2.
corresponds to the sectors that were updated since the last volume bitmap scan was done. These are the out-of-sync sectors. Because of the bitmap scan method, writes might not be applied on to the secondary volume in the same sequence as they are written to the primary volume. When terminating the Global Copy relationship to establish host access to secondary volumes, the first issue might cause you to lose transactions.
The steps in the procedure shown in Figure 18-3 are the following: 1. Quiesce the application updates. 2. Synchronize the volume pairs by one of these methods: – Perform the catch-up by doing a go-to-sync operation. This can be done with the Storage Manager as shown in 18.2.3, “Go-to-sync using the DS Storage Manager” on page 208. The volume pair leaves the copy pending state and reaches the full duplex state.
done using an ESS 800 (or ESS 750) as an intermediate unit, because the ESS 800 supports, with PPRC Version 2, both ESCON and FCP links. In this setup, the source volumes on the ESS F20 would be copied to intermediate volumes on the ESS 800 using Global Copy (or Metro Mirror), and they in turn would be copied to the target volumes on the DS6000 using cascading Global Copy. The intermediate ESS has to be at LIC level 2.4.3.65 or later in order to support Global Copy with the DS6000.
18.5 Hardware requirements Global Copy is an optional licensed function of the IBM System Storage DS6000 series. The licensed function is the Remote Mirror and Copy (RMC), and is the same that is used to acquire the Metro Mirror and the Global Mirror licensed functions. You have to purchase the corresponding licensed function for the primary and secondary DS6000 systems.
Therefore, for licensed functions authorized on the basis of physical capacity—as in the case of RMC—the authorization level established with the acquisition of the 53xx feature number on the Model 522 must also cover the physical capacity of the attached Model EX2 expansion enclosures. Interoperability Global Copy pairs can only be established between disk subsystems of the same (or similar) type and features.
18.7 Global Copy connectivity Global Copy pairs are set up between volumes in LSSs, usually in different disk subsystems, and these are normally in separate locations. A path (or group of paths) needs to be defined between the source LSS and the target LSS. These logical paths are defined over physical links between the disk subsystems.
18.7.2 Logical paths We recommend that you read the discussion presented in 13.4.2, “Logical paths” on page 144, in the Metro Mirror chapter, because it is also valid for Global Copy. 18.8 Distance considerations The maximum distance for a direct Fibre Channel connection is 10 km.
18.9 Other planning considerations Figure 18-6 illustrates the use of Global Copy for point-in-time backup solutions.
218 IBM System Storage DS6000 Series: Copy Services with IBM System z
19 Chapter 19. Global Copy performance and scalability In this chapter, we discuss performance and scalability considerations when using Global Copy for DS6000. © Copyright IBM Corp. 2006. All rights reserved.
19.1 Performance As the distance between DS6000s increases, Metro Mirror response time is proportionally affected, and this negatively impacts the application performance. When implementations over extended distances are needed, Global Copy becomes an excellent trade-off solution. You can estimate the impact of Global Copy on an application as that of the application when working with Metro Mirror suspended volumes.
20 Chapter 20. Global Copy interfaces The setup of Global Copy can be done using different interfaces. This chapter explains and gives examples of the interfaces that can be used for Global Copy management when Global Copy is used with the IBM System Storage DS6000 in System z environments.
20.1 Global Copy interfaces - overview There are various interfaces available for the setup and management of Global Copy for DS6000 in a System z environment.
Table 20-1 Comparison of commands Task TSO ICKDSF Command with DS CLI Select option with DS SM lsavailpprcport This information is shown during the process when a path is established. Global Copy paths commands List available I/O ports that can be used to establish Global Copy paths. List established Global Copy paths.
20.2 TSO commands for Global Copy management In z/OS systems, Global Copy can be managed using TSO commands. In this section, we discuss the Global Copy commands that TSO provides. For a detailed description of these commands, see the IBM publication z/OS DFSMS Advanced Copy Services, SC35-0428. 20.2.1 Commands summary and use A summary of the available TSO commands for Global Copy management and their corresponding use is presented in Table 20-2.
Table 20-3 CESTPAIR parameter Operation Volume pair state CESTPAIR parameter values from to OPTION MODE Establish initial copy pair simplex copy pending XD COPY Re-establish suspended pair suspended copy pending XD RESYNC 20.2.3 CESTPATH When establishing a path between DS6000s for Global Copy, use the CESTPATH command. There must be a physical Fibre Channel connection between the two DS6000 subsystems.
Example 20-4 CDELPATH command CDELPATH DEVN(X'6030') PRIM(X'2060' 500507630EFFFC6F X'00') SEC(X'0002' 500507630EFFFCA0 X'00') - Before issuing a CDELPATH command, a CDELPAIR command to all Global Mirror and Copy volume pairs should be given. The CDELPATH command may cause an ANTP0121I message if this sequence is not followed. 20.2.
20.2.7 CQUERY The CQUERY command is used to query the status of one volume of a Global Copy volume pair or all paths that are associated with the LSS for the device number that is specified. The CQUERY command can be issued to either a primary or secondary Global Copy volume. A host system that is attached only to a primary volume cannot obtain the status of the secondary volume for that pair. In the same way, a host attached only to the secondary volume cannot obtain the status of the primary volume.
During this procedure, the volume serial number can be verified and optionally relabeled. The volume can be varied online after this command is complete. Example 20-8 CRECOVER command CRECOVER DEVN(X’2242’) PRIM(‘X6060’ 62019 X’42’ X’00’) SEC(X’8061’ 68006 X’42’ X’00’) ID(VOL002 VOL001) MSGREQ(YES) In Example 20-8, the CRECOVER command brings device x’2242’ (on the recovery DS6000) to simplex state. It also changes the volume label from VOL002 to VOL001. 20.2.
Table 20-4 ICKDSF commands Command Description PPRCOPY ESTPATH Establishes Global Copy paths between a primary and secondary LSS PPRCOPY DELPATH Deletes Global Copy paths between primary and secondary LSSs PPRCOPY ESTPAIR Establishes a Global Copy primary-to-secondary volume relationship PPRCOPY DELPAIR Deletes a volume pair PPRCOPY RECOVER Allows a system to regain control of a volume on the secondary DS6000 PPRCOPY SUSPEND Puts a Global Copy volume pair in suspended state PPRCOPY FREEZE Sus
For the most current list of DS CLI supported environments, refer to the Interoperability Matrix that is found at the IBM Products site at: http://www-03.ibm.com/servers/storage/disk/ds6000/interop.html 20.4.1 Define Global Copy paths You can use the following commands to define and manage Global Copy paths. lsavailpprcport The lsavailablepprcport command displays the Fibre Channel ports, which can be used to establish Global Copy paths, see Example 20-11.
Example 20-14 rmpprcpath command dscli> rmpprcpath -dev IBM.1750-13AAGXA -remotedev IBM.1750-13AAVCA 16:16 Date/Time: June 12, 2005 5:58:25 AM CDT IBM DSCLI Version: 5.0.3.110 DS: IBM.1750-13AAGXA CMUC00152W rmpprcpath: Are you sure you want to remove the Remote Mirror and Copy path 16:16:? [y/n]:y CMUC00150I rmpprcpath: Remote Mirror and Copy path 16:16 successfully removed. You can suppress the CMUC00152W confirmation message by adding the -quiet parameter to the command. 20.4.
mkpprc With the mkpprc command, you can create a Global Copy pair. To create a Global Copy pair, specify parameter -type gcp, as shown in Example 20-18. Example 20-18 mkpprc command dscli> mkpprc -dev IBM.1750-13AAGXA -remotedev IBM.1750-13AAVCA -type gcp -mode full 1600:1600 Date/Time: June 14, 2005 11:25:49 AM CDT IBM DSCLI Version: 5.0.3.110 DS: IBM.1750-13AAGXA CMUC00153I mkpprc: Remote Mirror and Copy volume pair relationship 1600:1600 successfully created.
Example 20-22 rmpprc command dscli> rmpprc -dev IBM.1750-13AAGXA -remotedev IBM.1750-13AAVCA 1600:1600 Date/Time: June 14, 2005 12:48:09 PM CDT IBM DSCLI Version: 5.0.3.110 DS: IBM.1750-13AAGXA CMUC00160W rmpprc: Are you sure you want to delete the Remote Mirror and Copy volume pair relationship 1600:1600:? [y/n]:y CMUC00155I rmpprc: Remote Mirror and Copy volume pair 1600:1600 relationship successfully withdrawn. You can suppress the CMUC00160W confirmation message by adding the -quiet parameter.
Figure 20-1 Paths panel To create a new path, select from the Select Actions pull-down menu Create, and a process will then guide you to establishing a path. Alternatively, if in the Paths panel you select one or more of the existing paths, then the Select Actions pull-down menu will give you the following additional possible actions: Delete, to delete the paths you have selected before. LSS copy options, to modify the copy options for an LSS; see Figure 20-2. Figure 20-2 LSS Copy Options 20.5.
Figure 20-3 Metro Mirror panel To create a new Global Copy pair, choose Create from the Select Action pull-down menu. The DS Storage Manager will guide you through the process to create a new pair. Figure 20-4 Metro Mirror panel If you select an existing Global Copy pair (see Figure 20-4), then the Select Action pull-down menu will show additional actions you can perform with the selected volume pair: Delete: Use this action to delete a Global Copy pair.
236 IBM System Storage DS6000 Series: Copy Services with IBM System z
21 Chapter 21. Global Copy examples This chapter includes sample Global Copy scenarios and examples of the corresponding management activities using TSO and the DS CLI. The examples presented are: How to establish and manage Global Copy volume pairs with TSO Use Global Copy to migrate volumes from an ESS 800 to a DS6000 using the DS CLI © Copyright IBM Corp. 2006. All rights reserved.
21.1 Define and manage Global Copy pairs using TSO Figure 21-1 shows a diagram of the test environment that we used for our example. DS6800 BOX 1 1750-13AAGXA WWNN 500507630EFFFC6F DS6800 BOX 2 1750-13AAVCA WWNN 500507630EFFFCA0 LCU 00 LCU 00 I0000 I0100 I0000 I0100 FC Switch Figure 21-1 Configuration used to setup Global Copy pairs in our example In our example, using TSO commands, we established a path between two LSSs and then established the Global Copy pairs.
Example 21-3 CQUERY command ANTP8802I CQUERY DEVN(X'6030') ANTP0090I CQUERY FORMATTED LVL 3 161 VOLUME REPORT ************** PPRC REMOTE COPY CQUERY - VOLUME ******************** * (PRIMARY) (SECONDARY) * * SSID CCA LSS SSID CCA LSS* *DEVICE LEVEL STATE PATH STATUS SERIAL# SERIAL# * *------ --------- ---------- ----------- ----------------- * * 6030 PRIMARY.. PENDING.XD ACTIVE.. 2060 30 00 0002 30 00 * * CRIT(NO)....... CGRPLB(NO).
ANTP0001I CQUERY COMMAND COMPLETED FOR DEVICE 6030. COMPLETION CODE: 00 3. After the volumes were fully copied, we suspended the pairs; see Example 21-5. Example 21-5 CSUSPEND Global Copy pair ANTP8802I CSUSPEND DEVN(X'6030') PRIM(X'2060' AAGXA X'30' X'00') SEC(X'0002' AAVCA X'30' X'00') ANTP0001I CSUSPEND COMMAND COMPLETED FOR DEVICE 6030.
SSID(0002) MODE(COPY) 6. The final task was to reestablish Global Copy for the volume pairs; see Example 21-8. Example 21-8 Restart Global Copy CESTPAIR DEVN(X'6030') PRIM(X'2060' AAGXA X'30' X'00') SEC(X'0002' AAVCA X'30' X'00') OPTION(XD) MODE(RESYNC) CESTPAIR DEVN(X'6031') PRIM(X'2060' AAGXA X'31' X'00') SEC(X'0002' AAVCA X'31' X'00') OPTION(XD) MODE(RESYNC) 21.
Example 21-10 Check ports C:\IBM\DSCLI>dscli -cfg ess-22399.prf lsavailpprcport -dev ibm.2105-22399 -remotedev ibm.1750-1300247 -remotewwnn 500507630efffe16 05:02 Date/Time: November 22, 2005 4:08:10 PM EET IBM DSCLI Version: 5.1.0.204 DS: IBM.2105-22399 Local Port Attached Port Type ============================= I000C I0103 FCP I00AC I0003 FCP 3. We established paths from source LSS 05 to target LSS 02. The CLI script file to do this is shown in Example 21-11. Example 21-11 Script estpath.
=========================================================================================== 050A:023A Copy Pending Global Copy 48148 Disabled Unknown Unknown 050B:023B Copy Pending Global Copy 50085 Disabled Unknown Unknown 5. We synchronized the Global Copy pairs so that the target volume would be consistent. This should not be done until the number of out-of-sync tracks for most of the volumes is close to zero.
Date/Time: November 22, 2005 4:45:34 PM EET IBM DSCLI Version: 5.1.0.204 DS: IBM.2105-22399 CMUC00234I lspprc: No Remote Mirror and Copy found. After the volume pairs have been deleted, the host systems can be IPLed from the target volumes on the DS6000 and the applications can start. To avoid duplicate volser messages and the related operator replies during IPL, physically disconnect the ESS 800 source system from the host, or change ESCON/FICON director configurations as relevant. 7.
22 Chapter 22. Global Mirror overview In this chapter we provide an overview of what Global Mirror is. We also discuss the need for data consistency at a distant site when synchronous data replication such as Metro Mirror is not possible. This chapter then explains how Global Mirror works in a similar manner to a distributed application—in a server and client relationship. Finally, in this chapter you will find a step-by-step process to establish a Global Mirror environment.
22.1 Synchronous and non synchronous data replication When replicating data over long distances, beyond the 300 km, asynchronous data replication is the valid approach. This is basically so because with the asynchronous techniques, the application I/O processing at the local storage disk subsystem (storage server) remains independent of the process of data transmission to the remote storage disk subsystem.
In synchronous data replication methods such as Metro Mirror, an application write always goes through the following four steps; see Figure 22-1 on page 246: 1. Write the data to the primary storage disk subsystem cache and present channel end to free up the channel for further I/O. Note this does not end the I/O and does not present an I/O complete to the application. 2. Replicate the data from the primary storage disk subsystem cache to the secondary storage disk subsystem cache. 3.
A database update usually involves three dependent write I/Os to ensure data consistency, even in the event of a failure during this process: 1. Update intent to the logging files—logging may happen to two logging files. 2. Update the data. 3. Indicate update complete to the logging files. This sequence of I/Os is also called a two phase commit process. When you are in a remote copy environment, in an outage situation, the following sequence may occur; see Figure 22-2 on page 247: 1.
Database Subsystem 1 Primary 6 Database Subsystem 7 Fail over to secondary site Recover A2 Secondary Restart Hold I/O 3 Storage Disk Subsytem 1 2C00 A Primary A1 A Primary Primary 4 Synchronous Primary Log Redrive I/O Primary DB Replicate 2 Storage Disk Subsystem 5 2D00 2E00 A A A3 Primary Primary Primary Primary Primary Primary RECON 8 8 Log' Storage Disk Subsystem 4 3D00 4 A B2 Synchronous 4 8 B1 1 5 Storage Disk Subsystem 3 2D00 2D00 A Primary Primary A Primary A2 Primar
Summary In summary, the following characteristics are typical of a synchronous data replication technique: Application write I/O response time is affected —this can be modeled and predicted. Local and remote copies of data are committed to both storage disk subsystems before host write I/O is complete. Data consistency is always maintained at the remote site as long as no failures occur. If a rolling disaster occurs, freeze/run is needed to maintain consistency.
2. Present channel end and device end and acknowledge to the application successful I/O completion. The application can then immediately schedule the next I/O. 3. Replicate the data from the primary storage disk subsystem cache to the secondary storage disk subsystem cache. 4. Acknowledge to the primary storage disk subsystem that data successfully arrived at the secondary storage disk subsystem.
Database Subsystem Primary local site Secondary remote site 3 1 Storage Disk Subsystem 2 Storage Disk Subsystem 1 2C00 2C00 A Primary A Primary Primary 3C00 non synchronous Primary Log 3 B 1 Log' Replicate Storage Disk Subsystem 3 2D00 2D00 A A A Primary Primary Primary Primary Primary Primary DB 2 non synchronous Replicate Storage Disk Subsystem 4 3D00 A B Primary Primary Primary DB' Figure 22-6 Global Copy non synchronous data replication involving multiple disk subsystems Notice th
To accomplish the necessary activities with minimum impact on the application write I/O, Global Mirror introduces a smart bitmap approach in the primary storage disk subsystem. With this, Global Mirror can resume the application I/O processing immediately after a very brief serialization period for all involved primary storage disk subsystems —this brief serialization periodically occurs at the very beginning of a sequence of events that resemble the ones outlined above.
Global Mirror works like a distributed application. A distributed application is usually built on a server to client relationship. The server functions as a supervisor and instructs the client. The client is able to do some work in an autonomic fashion but relies on the coordination efforts from the server; see Figure 22-7 on page 253. The server distributes the work to its clients. The server also coordinates all individual feedback from the clients and decides based on this feedback further actions.
When the Master and Subordinate are in a single storage disk subsystem the Subordinate is internally managed by the Master. With two or more storage disk subsystems at the local site, which participate in a Global Mirror session, the Subordinate is external and needs separate attention when creating and managing a Global Mirror session or environment. The following sections explain how Global Mirror works and how Global Mirror ensures consistent data at any time at the remote site.
Host Write I/O A A Primary Primary Primary A B Primary Primary Primary Global Copy path Primary Primary Local site FCP port Remote site Figure 22-10 Establish Global Copy connectivity between both sites Note in Figure 22-10that we establish Global Copy paths over an existing network. This network may be based on a FCP transport technology or on an IP based network. Global Copy paths are logical connections that are defined over the physical links that interconnect both sites.
In the following paragraphs we refer to the primary volume as the A volume and to the secondary volume as the B volume for simplicity. Global Copy does not immediately copy the data as it arrives to the A volume. Instead, this is an asynchronous process. As soon as a track is changed by an application write I/O, it is reflected in the out-of-sync bitmap as with all the other changed tracks.
Start change recording: Apply changes only from the source volume to the target volume which occurred to the source volume in between FlashCopy establish operations —except for the first time when FlashCopy is initially established. Persist: Keep the FlashCopy relationship until explicitly or implicitly terminated. This parameter is automatic due to the nocopy property.
22.3.6 Populate Global Mirror session with volumes Host Add Global Copy primary volume to Global Mirror session 01 A Primary Primary A Primary PENDING A B Primary Primary Global Copy Secondary PENDING S B M O O S Local site FlashCopy MODE(ASYNC) Remote site Primary Primary C Tertiary T B M Figure 22-14 Add Global Copy primary volume to Global Mirror session The next step is the definition of volumes in the Global Mirror session. The focus is still on the local site; see Figure 22-14.
Host Start Global Mirror session 01 A Primary Primary A A B FlashCopy MODE(ASYNC) Primary Primary Global Copy Primary Secondary PENDING PENDING Primary Primary C Tertiary C R O O S CR: Change recording bit map Local site S B M C R T B M Remote site Figure 22-15 Start Global Mirror This start command triggers events that involve all the volumes within the session.
Start 1 Done Start Serialize all Global Copy primary volumes C R A1 Primary 2 O O S Done Drain data from local to remote site Start 3 Done Perform FlashCopy B1 Secondary C1 Tertiary C R A2 Primary Local O O S B2 Secondary C2 Tertiary Remote Figure 22-16 Formation of consistent set of volumes at the secondary site Note that before step 1 and after step 3, Global Copy constantly scans through the out-of-sync bitmaps and replicates data from A volumes to B volumes as described in “Crea
Once step 3 is complete, a consistent set of volumes have been created at the secondary site. This set of volumes, the C volumes, represents the Consistency Group. At this very moment also, the B volumes and the C volumes are, but only for this brief moment, equal in their content. Immediately after the FlashCopy process is logically complete, the primary systems’s microcode is notified to continue with the Global Copy process.
If this persists for a significant period of time, then eventually Global Mirror will force the formation of a new Consistency Group. In this way Global Mirror ensures that during periods when the bandwidth is insufficient, production performance is protected and data is transmitted to the secondary site in the most efficient manner possible. When the peak activity has passed, consistency group formation will resume in a timely fashion.
264 IBM System Storage DS6000 Series: Copy Services with IBM System z
Part 6 Part 6 Global Mirror This part of the book describes IBM System Storage Global Mirror for DS6000 when used in a System z environment. We discuss the characteristics of Global Mirror and describe the options for its setup. We also show which management interfaces can be used, as well as the important aspects to be considered when establishing a Global Mirror environment. This part ends with examples of Global Mirror management and setup. © Copyright IBM Corp. 2006. All rights reserved.
266 IBM System Storage DS6000 Series: Copy Services with IBM System z
23 Chapter 23. Global Mirror options and configuration In this chapter we provide a detailed description of Global Mirror options, including how to create a Global Mirror environment and how to remove it. We discuss how to change Global Mirror tuning parameters and how to modify an active Global Mirror session. We also discuss a scenario where a sites switch is performed due to a primary site failure.
23.1 Terminology used in Global Mirror environments First, let us review and further define some of the terms and new elements we have presented so far, and that are of common use when working in a Global Mirror context. Dependent writes If the start of one write operation is dependent upon the completion of a previous write, the writes are dependent. Application examples for dependent writes are databases with their associated logging files.
When you start or resume a session, Consistency Groups are created, and the Master storage disk subsystem controls the session by communicating with the Subordinate storage disk subsystems. There is also a session concept at the LSS level. But all LSS sessions are combined and grouped together within a Global Mirror session.
Subordinate 01 A Primary Primary A Global Copy Network A B Primary Primary Secondary Primary PENDING PENDING Primary Primary C Tertiary paths 01 A Primary Master Primary A Primary A B Primary Primary Network PENDING PENDING Local site Secondary Global Copy Remote site Primary Primary C Tertiary Figure 23-1 Global Mirror basic configuration with Master and Subordinate disk subsystems The order of commands to create a Global Mirror environment is not completely fixed and allows for some
disk subsystem at the local site. If you did not establish these paths in the very first step, then this is the time to create these paths before you continue with the next step. 5. Define a token that identifies the Global Mirror session. This is a session ID with a number between 1 and 255.
volumes, you may consider adding the new volumes to an existing Global Mirror session in stages. Suspending a Global Copy pair that belongs to an active Global Mirror session will impact the formation of Consistency Groups. When you intend to remove Global Copy volumes from an active Global Mirror session, follow these steps: 1. Remove the desired volumes from the Global Mirror session; Example 23-2 on page 274 shows how to remove Global Copy volumes from a Global Mirror session, using TSO commands. 2.
Important: When setting new values for the tuning parameters, be sure to check for errors in Consistency Group formation and in draining the out-of-sync bitmaps. A few errors are not significant and do not jeopardize the consistency of your Global Mirror.
The RSESSION command contains the topology information for the Subordinate and also the communication paths specification.
volumes from the Global Mirror session, Global Copy keeps running and replicating newly arrived write I/Os over the paths to the remote storage disk subsystem. The termination of FlashCopy relationships, may be needed when you want to change the FlashCopy targets within a Global Mirror configuration and choose, for example, another LSS for the FlashCopy targets. This may be needed because you want to replace the FlashCopy targets due to a skew in the load pattern in the remote storage disk subsystem.
When you create a Global Mirror environment that spans multiple storage disk subsystems at the local site, and probably also at the remote site, then you need to establish communication paths between the involved local storage disk subsystems. Figure 23-2 on page 275 shows a symmetrical configuration with a one-to-one mapping. You have to define the corresponding session, with its number, to all potentially involved LSSs at the local site.
Note that when you configure links over a SAN network, the same FCP ports of the storage disk subsystem may be used for the Global Mirror session communication, as well as for the Global Copy communication, and for host connectivity. However, for performance reasons, and to prevent host errors from disrupting your Global Mirror environment, it is often a good idea to use separate FCP ports.
Subordinate 01 A Primary Primary A A B Primary Primary Secondary Primary PENDING Subordinate PENDING Global Copy links 01 A Primary Primary A A B C Tertiary Primary Primary Secondary Primary PENDING Primary Primary PENDING Global Copy links Primary Primary C Tertiary GM links 01 A Master Primary Primary A Primary PENDING Local site Network A B Primary Primary Secondary PENDING Global Copy links Remote site Primary Primary C Tertiary Figure 23-5 Dedicated and shared links In Figure
Subordinate 01 A Primary Primary A A B Primary Primary Secondary Primary PENDING Subordinate PENDING Global Copy links 01 A Primary Primary A A B 01 A Master Primary Primary A Primary PENDING Local site C Tertiary Primary Primary Secondary Primary PENDING Primary Primary PENDING Global Copy links GM links Network A B Primary Primary C Tertiary Primary Primary Secondary PENDING Global Copy links Remote site Primary Primary C Tertiary Figure 23-6 Dedicated Global Mirror links and dedi
01 A Primary Primary A A B Primary Primary Secondary Primary PENDING PENDING A B Primary Primary Secondary Primary PENDING PENDING Host channels Primary Global Mirror session links Global Mirror session links ISL FCP Local site Primary Primary C Tertiary PENDING FICON C Tertiary 01 A Primary Primary A 01 A Primary Primary A Primary Primary A B Primary Primary Secondary PENDING FCP Global Copy links Host Primary Primary Host channels C Tertiary FICON Remote site Host Figure
23.6.
Figure 23-8 shows more than a single FICON/FCP director at both sites, for availability reasons. Note that the Master and Subordinate paths are defined over a local SAN fabric, represented by the FICON/FCP directors. 23.7 Recovery scenario after primary site failure This section covers the steps that you need to follow in a Global Mirror environment, when a primary site failure requires you to recover at the secondary site.
Host FlashCopy A Primary A Primary Primary Primary A Primary PENDING Local site Global Copy A A B Primary Primary Primary Primary Secondary PENDING A A C Primary Primary Primary Primary Tertiary Remote site Figure 23-10 Primary site fails The goal is to swap to the remote site and restart the applications. This requires, first, to make the set of consistent volumes at the remote site available for the application, before the application can be restarted at the remote site.
commands. In our example, we assume that there is a host connection to the remote storage disk subsystem; see Figure 23-11.
Note that this command just changes the state of the secondary volumes from secondary pending to primary suspended. This command does not communicate to the other storage disk subsystem at all, even though it is specified in the SEC parameter of the command. Once all the Failover commands are successfully executed, we can move on to the next step. 23.7.
FCQUERY Formatted -2 DEVN SSID LSS CCA CU SERIAL ACT MAX XC PC CC RV SEQNUM 6403 0002 00 03 1750 0000000AAVCA 1 50099 N S N N 42B03957 FCQUERY COMMAND COMPLETED FOR DEVICE 6403. COMPLETION CODE: 00 Example 23-5 shows, for each single FlashCopy relationship, a column titled RV, which stands for revertible. An N indicates that this FlashCopy relationship is not revertible and all changes are committed. All volumes show the same information for RV.
Are all FC relationships revertible? Are all FC sequence numbers equal? Action to take Comments Case 2 SOME - Some FlashCopy pairs are revertible and others are not revertible. Revertible FlashCopy pairs’ sequence numbers are equal. And non-revertible FlashCopy pairs sequence numbers are equal, but do not match the revertible FlashCopies sequence number. revert FC relations. Some FlashCopy pairs are running in a Consistency Group process and some have not yet started their incremental process.
These indications suggest that you have to return the revertible FlashCopy relationships to the previous Consistency Group doing a FlashCopy FCWITHDR command with the REVERT action to restore the FlashCopy relations to their prior state. You may run this command against all volumes. As stated before, this will do nothing to the non-revertible FlashCopy pairs but return an error message.
Example 23-8 Withdraw Global Mirror FlashCopy relationship with ACTION(COMMIT) //* -------------------------------------------------------------- *** //FCWITHDR EXEC PGM=IKJEFT01 //SYSPRINT DD SYSOUT=* //SYSTSPRT DD SYSOUT=* //SYSTSIN DD DDNAME=SYSIN FCWITHDR SDEVN(X'6400' TDEVN(X'6500') ACTION(COMMIT) 23.7.5 Set consistent data on B volumes At this point only the C volumes comprise a set of consistent data volumes.
FRR replicates all data from C to B making a physical copy of C, with some exceptions. Note that the previous nocopy relationship between B and C caused some I/Os from B to C. Changed tracks between two Consistency Group creation points will be preserved on the C volume after being copied over from the B volumes before they change there.
//SYSTSIN FCESTABL DD DDNAME=SYSIN SDEVN(X'3C00') TDEVN(X'3E00') MODE(ASYNC) Now you may restart the applications at the remote site using the B volumes. Note the B volumes are Global Copy primary volumes in suspended state, which implies that change recording takes place. Later this allows you to resynchronize from B to A, before returning to the local site.
23.7.7 Restart the application at the remote site At this stage you may restart the application at the remote site and work with the consistent set of B volumes; see Figure 23-15.
Note that the Failback operation is issued to the B volumes as the primary and the A volumes as the secondary. This command changes the A volume from its previous primary pending state to secondary pending and starts the resynchronization of the changes from B to A. Before doing the Failback operation, ensure that paths are established from the remote site LSS to its corresponding LSS at the local site. Note that with Fibre Channel links you can define paths in either direction on the very same FCP link.
Once the local site is ready, quiesce the application at the remote site. Then a sequence of Failover - Failback operations from A to B will reestablish Global Copy back as it was originally before the local site outage. Figure 23-17 shows the action at the local site, the Failover operation from A to B. This will change the state of the A volumes from secondary pending to primary suspended, and start to keep a bitmap record of the changes to the A volumes.
Last but not least, if you did not already establish the FlashCopy relationship from B to C during the Failover - Failback sequence at the remote site, then you have to do it now. This may be an inband FlashCopy as shown in Figure 23-19. A TSO command example is provided in Example 23-13. Example 23-13 Establish FlashCopy from B to C via inband TSO command //* ---------------------------- TSO ------------ CREATE (3) ----//* FLASHCOPY B -> C START CHANGE RECORDING NOCOPY ETC.
23.7.10 Conclusions This concludes the sequence of steps you have to go through for a swap to the remote site and coming back to the local site after service is restored at the local site. In particular, the check on a valid Consistency Group after a primary site failure is a challenge when you consider a large configuration with many volume pairs. Each command usually addresses a single pair of volumes.
24 Chapter 24. Global Mirror interfaces In this chapter we provide an overview of the interfaces you can use to manage and control Global Mirror environments. We explain the TSO command support for Global Mirror. We also discuss the use of the DS CLI for managing a Global Mirror configuration in a System z environment, and provide examples. Then we describe the ICKDSF utility and the ANTRQST macro support.
24.1 Global Mirror interfaces - overview Global Mirror combines Global Copy and FlashCopy, which work together in an autonomic fashion under microcode control. There are commands intended for Global Copy and FlashCopy, as well as commands that address Global Mirror sessions.
Host Write I/O A A Primary Primary Primary A B Primary Primary Primary Global Copy Secondary Primary PENDING PENDING Local site O O S FlashCopy MODE(ASYNC) Primary Primary C Tertiary SBM: Source Bit Map TBM: Target Bit Map S B M Remote site T B M Figure 24-1 Establish FlashCopy relationship within Global Mirror 24.2.1 Establish FlashCopy using TSO Example 24-1 shows the TSO command that creates the FlashCopy relationship between the B and the C volumes.
24.2.2 Establish FlashCopy using DS CLI Example 24-3 shows how to create the FlashCopy relationship between the B and the C volumes using the DS CLI command mkflash. Example 24-3 Create FlashCopy between B and C volumes using DS CLI dscli mkflash -record -persist -nocp -seqnum 0000 6500:6502 -cfg $DSCLI/profile/DS8300-box2.
24.2.3 Establish FlashCopy using ICKDSF ICKDSF is usually the interface that is used to manage remote copy configurations in z/VM and z/VSE environments. It is a batch-oriented approach and requires the JCL to invoke ICKDSF. Example 24-7 shows the FlashCopy command used to create a Global Mirror FlashCopy relationship.
TSO commands TSO commands are submitted using an inband approach over the host channel directly to the storage disk subsystem. No active SMC or DS SM is required to trigger Copy Services functions. You can issue the TSO commands from procedures that are similar to the scripting approach in open systems environments. Note that TSO requires a z/OS system at the recovery site when you recover a Global Mirror environment at the remote site. DS CLI The strength of DS CLI is its scripting capability.
24.3.1 Establish a Global Mirror environment The following sections discuss the setup of a Global Mirror environment using TSO commands in a step-by-step approach. The sequence of steps in our example is the recommended one, although it is not a mandatory one. Still, it guarantees a clean and straightforward creation of the Global Mirror environment. 24.3.2 Define paths When you establish Global Copy relationships between volume pairs, you have to first establish the Global Copy paths.
//EPATHS EXEC //SYSPRINT DD //SYSTSPRT DD //SYSTSIN DD CESTPATH CQUERY PGM=IKJEFT01 SYSOUT=* SYSOUT=* DDNAME=SYSIN DEVN PRIM SEC LINK (X'2C00') + (X'2C00' 5005076303FFC228 X'0C') + (X'3C00' 5005076303FFC422 X'0C') + (X'00300030' X'01300130' + X'02300230' X'03310330') CGROUP(NO) DEVN (X'2C00') PATHS Primary and secondary LSSs are identified by their respective subsystem IDs (SSIDs), the WWNN of the storage disk subsystem, and the LSS number.
For redundancy reasons it is better to define two paths between the primary disk subsystems. Note that the FCP ports may be shared using a SAN fabric. See Figure 24-3. 24.3.3 Establish Global Copy volume pairs After defining the paths, you create the Global Copy volume pairs.
Example 24-11 Query Global Copy pair immediately after ESTABLISH CQUERY DEVN(X'2C00') CQUERY FORMATTED LVL 3 VOLUME REPORT ********************** COPY CQUERY - VOLUME *********************** * (PRIMARY) (SECONDARY) * * SSID CCA LSS SSID CCA LSS* *DEVICE LEVEL STATE PATH STATUS SERIAL# SERIAL# * *------ --------- ---------- ----------- ----------------- * * 2C00 PRIMARY.. PENDING.XD ACTIVE.. 2C00 00 0C 3C00 00 0C * * CRIT(NO)....... CGRPLB(NO).
Example 24-13 Global Copy after first replication phase is completed CQUERY DEVN(X'2C00') CQUERY FORMATTED LVL 3 VOLUME REPORT ********************** CQUERY - VOLUME **************************** * (PRIMARY) (SECONDARY) * * SSID CCA LSS SSID CCA LSS* *DEVICE LEVEL STATE PATH STATUS SERIAL# SERIAL# * *------ --------- ---------- ----------- ----------------- * * 2C00 PRIMARY.. PENDING.XD ACTIVE.. 2C00 00 0C 3C00 00 0C * * CRIT(NO)....... CGRPLB(NO).
The inband modality is used when the remote site is not connected to the local site. DEVN refers to the primary volume, and SOURCE and TARGET refer to remote storage disk subsystem. MODE(ASYNC) implicitly includes the specific FlashCopy attributes that are required for Global Mirror. 24.3.5 Define a Global Mirror session Before adding volumes to the Global Mirror session, a token is required under which the session can be identified and addressed.
The TSO command RSESSION is used to set up and manage a Global Mirror session; see Example 24-15. This includes the ability to define a Global Mirror session by means of the ACTION(DEFINE) parameter option. Other options that can be specified for the ACTION parameter are: START and STOP a Global Mirror session, as well as PAUSE and RESUME of the session, or to UNDEFINE the session.
24.3.7 Start a Global Mirror session The new RSESSION command is used to start the Global Mirror session. With the ACTION(START) parameter, you indicate the start of the Consistency Groups formation. Example 24-17 Start Global Mirror session //* ---------------------------- TSO ------------ CREATE (6) ----//* START SESSION X'01' THROUGH LSS X'0C' //* //* THEN LSS X'0C' BECOMES THE MASTER LSS (DS) //* NOTE: LSS SYNTAX IS WITHOUT X AND QUOTES ...
ACTION is required and specifies which kind of query report is requested. Three different query reports can be requested with this parameter: – GMLSTAT provides summary information pertaining to the Global Mirror session. This includes information about the Master storage disk subsystem, the Subordinate storage disk subsystems, as well as the number of successfully created Consistency Groups and how often Consistency Group creation failed. It also includes the parameter values for the session.
Example 24-19 shows for this status Running. Refer to z/OS DFSMS Advanced Copy Services, SC35-0428, for a description of different status indications. Example 24-20 shows Global Mirror status information for a particular LSS and its Global Copy primary volumes, when ACTION(GMPSTAT) is used with the RQUERY command.
Example 24-22 Global Copy primary volumes in Global Mirror session READY RQUERY SNBR(01) VOLSER(XX2C00) ACTION(DVCSTAT) RQUERY Output Volser(XX2C00) Action(DVCSTAT) Version(001) SNbr LSS Dvc VolStat PriPPRCStat SecCascStat -- -- -- -------------------- ---------- ---------01 0C 00 InSession DplxPendng Simplex READY RQUERY SNBR(01) VOLSER(XX2D00) ACTION(DVCSTAT) RQUERY Output Volser(XX2D00) Action(DVCSTAT) Version(001) SNbr LSS Dvc VolStat PriPPRCStat SecCascStat -- -- -- -------------------- ---------- ----
Query All ICKDSF Establish Paths Establish Pairs Manage Global Mirror Sessions Define, Pause, Resume, Terminate Global Mirror Sessions z/OS z/VM VSE Subordinate LSS Host I/O LSS Long Distance Global Copy Global Copy paths LSS Global Copy paths LSS LSS Global Mirror Session paths LSS LS S Master Local Site Remote Site Figure 24-7 ICKDSF support for Global Mirror Figure 24-7 provides the complete picture of a Global Mirror session.
Paths LSS Subordinate LSS Host Global Mirror session paths Global Copy paths LSS LSS LSS Master WWNN: 5005076300C09517 Global Copy paths LSS WWNN: 5005076300C0A2ED Local site (primary) Remote site (secondary) Figure 24-8 Define all needed paths for the Global Mirror environment through ICKDSF Example 24-23 shows the JCL and command lines used to define paths between two storage disk subsystems.
24.5.3 Establish Global Copy pairs When you set up a Global Mirror environment, there is nothing particularly different concerning the Global Copy pairs. The command syntax used to establish Global Copy pairs with ICKDSF has no changes, whether used for Global Mirror setup or not. See Figure 24-9.
hex notation. In hex notation the third parameter refers to the channel connection address (CCA) of the corresponding device. 24.5.4 Establish FlashCopy relationships The next step is to create the FlashCopy relationship between the B volume and the C volumes at the remote site; note in Figure 24-10 the particular attributes for FlashCopy because it is being used in a Global Mirror environment.
The ESTABLISH inband command has some complexity. UNIT points to the local primary volume. SOURCEVOL identifies the FlashCopy source volume at the remote site. Note this volume is also the Global Mirror secondary volume. TARGETVOL identifies the FlashCopy target volume. Both parameters require certain positional values. All parameters are in hex notation except the serial number in SOURCEVOL as the last positional parameter.
Primary volumes Secondary volumes = Flashcopy source Session number Subordinate LSS LSS Global Mirror session paths Long distance Global Copy volume pairs LSS LSS LSS Master Global Copy paths Flashcopy target LSS Local site (primary) Global Copy paths LSS LSS Remote site (secondary) Figure 24-11 ICKDSF - define Global Mirror session Example 24-27 shows an ICKDSF job that defines a Global Mirror session involving the LSS that volume RS7000 belongs to.
Secondary volumes = Flashcopy source Primary volumes Host Subordinate LSS LSS Global Mirror session paths Global Copy paths Long distance Global Copy volume pairs LSS LSS Global Copy paths LSS Master Flashcopy target LSS Local site (primary) LSS LSS Remote site (secondary) Figure 24-12 ICKDSF - populate a Global Mirror session with primary volumes Example 24-28 shows the JCL and commands to add volumes to a Global Mirror session.
Note that the VOLCNT parameter counts the number of ranges you specify in the RVOLLIST parameter. In Example 24-29 the VOLCNT is 2 because RVOLLIST contains two ranges. Instead of RVOLLIST, you can select dedicated volumes that are going to become a part of Global Mirror session through the IVOLLIST parameter. Volume count VOLCNT has a similar meaning with IVOLLIST as with RVOLLIST; see Example 24-30.
Example 24-31 ICKDSF example to start a Global Mirror session //* -------------------------------------------------------------- *** //STEP01 EXEC PGM=ICKDSF //SYSPRINT DD SYSOUT=* //SYSIN DD * PPRC PPRC /* UNIT(7000) STARTASYNCCOPY START SESSIONNO(001) MAXCOORDTIME(05) MAXDRAINTIME(60) CGINTERVALTIME(0) - UNIT(7000) QUERY ASYNCCOPY 24.5.8 Query an active Global Mirror session Figure 24-14 shows an ICKDSF command to query the devices and their session status.
/ / / / / / / / / / * -- - -- -- ----- --- -- ----- --- -- - - - ------ -- -- - ---- ---- - - ---- ---- - -- *** STEP01 EXEC PGM=I CKDSF SYSPRI NT DD SYSOUT=* DD01 DD UNI T=3390, VOL=SER=RSED00, DI SP=SHR PPRC XD Pr i mar y SYSI N DD * PPRCOPY DDNAME( DD01) /* QUERY ASYNCCOPY SESSI ON I NFORMATI ON CURRENT TI ME = 2004. 05. 20.
400 : FYY : 7ZZZ: F005: F01D: INVALID PARAMETER FORMAT 0X0F ERROR, WHERE YY IS THE REASON CODE MICROCODE LOGIC ERROR - ZZZ DESCRIBES ERROR TEMPORARILY UNAVAILABLE LONG BUSY MASTER STATE 01: PAUSE/TERMINATE ASYNCHRONOUS PPRC IN PROGRESS 02: START/RESUME ASYNCHRONOUS PPRC IN PROGRESS 03: ASYNCHRONOUS PPRC IS BETWEEN Consistency Group FORMATIONS 04: XDC START INCREMENT IN PROGRESS 05: XDC RUN IN PROGRESS 06: DRAIN IN PROGRESS 07: FLASHCOPY ESTABLISH WITH REVERTIBLE IN PROGRESS 08: FLASHCOPY WITHDRAW WITH COM
Example 24-33 Stop Global Mirror session with ICKDSF //* -------------------------------------------------------------- *** //TERMSESS EXEC PGM=ICKDSF //SYSPRINT DD SYSOUT=* //DD01 DD UNIT=3390,VOL=SER=AA6000,DISP=SHR //SYSIN DD * PPRCOPY DDNAME(DD01) TMASYNC SESSNO(001) TERMINATE MASTER /* - Note also that an ongoing process to form a Consistency Group may not come to a successful end when you TERMINATE a session.
Example 24-35 Remove session from LSS with ICKDSF //* -------------------------------------------------------------- *** //CLOSESS EXEC PGM=ICKDSF //SYSPRINT DD SYSOUT=* //DD01 DD UNIT=3390,VOL=SER=AA6000,DISP=SHR //SYSIN DD * PPRCOPY DDNAME(DD01) DEFSESS SESSNO(001) CLOSE /* - Now, in the next steps, we will remove the FlashCopy relationships, and later we will delete the Global Copy pairs that belonged to the Global Mirror session we just deleted. 24.5.
24.5.14 Delete Global Copy pairs Example 24-37 shows how to delete the Global Copy volume pairs.
RQUERY There are several action parameters to determine what output information to obtain from a Global Mirror session. The API allows you to get more information out of a Global Mirror session than the respective TSO or ICKDSF commands. The ANTRQST macro allows you to direct the query output to a preallocated data set, with a logical record length or 120-byte fixed length. The ANTRQST macro is not described in detail here.
24.7.1 View Global Mirror volumes in session From the panel in Figure 24-16 on page 328, we select the View session volumes from the pull-down list, as shown in Figure 24-17. Figure 24-17 View Global Mirror volumes in session1 - select action The next panel, shown in Figure 24-18, provides the requested list of Global Copy primary volumes that are defined to session number 01. Figure 24-18 View Global Mirror volumes in session1 - volume list Chapter 24.
Figure 24-18 suggests you choose meaningful nicknames for the volumeswhen you define them to the DS6000. These nicknames indicate that the volumes belong to Extent Pool2, which is a CKD pool, and the corresponding z/OS device numbers are 2000+. The syntax of the nicknames is a bit limited and does, for example, not allow you to use numerics in the prefix part of the nickname when defined through the GUI. 24.7.
Next you receive a confirmation panel as shown in Figure 24-20. Figure 24-20 Pause selected session number 1 - confirmation panel Click OK and you receive the next panel; see Figure 24-21. Figure 24-21 Session is paused Figure 24-21 shows the result as well as the status information indicating that session number 01 is paused. Chapter 24.
We now resume the session. We select the corresponding action parameter, Resume, from the Select Action pull-down list; see Figure 24-22. Figure 24-22 Resume paused session number 1 through the GUI You receive another confirmation panel asking whether to continue with the resume operation; see Figure 24-23.
We click OK and then receive the next panel; see Figure 24-24. Figure 24-24 Resume paused session number 1 through the GUI - session is running The session is back and running. There is a more extensive exercise based on the GUI in Chapter 26, “Global Mirror examples” on page 341. Chapter 24.
334 IBM System Storage DS6000 Series: Copy Services with IBM System z
25 Chapter 25. Global Mirror performance and scalability This chapter discusses performance considerations for planning and configuring Global Mirror for DS6000. It also explains the potential impact that the three phases of Consistency Group formation might have on application write I/Os. Finally, it covers the distribution of B volumes and C volumes across different ranks, and how to provide extra care for very busy volumes. © Copyright IBM Corp. 2006. All rights reserved.
25.1 Performance aspects for Global Mirror Global Mirror is basically comprised of Global Copy and FlashCopy, and it combines both functions to create a solution that provides consistent data at a distant site. Global Copy has, at most, only a minimal impact on the response time of an application write I/O to the Global Copy primary volumes. In the Global Mirror environment, FlashCopy is used with the nocopy attribute.
There is potential impact on the Global Copy data replication operation, depending on whether persistent memory or non-volatile cache is over-committed in the secondary storage disk subsystem. In this situation, the FlashCopy source tracks might have to be preserved first to the FlashCopy target volume, before the Global Copy write completes. Usually, however, all writes are quick writes to cache and persistent memory.
The following example addresses the impact of the coordination time when Consistency Group formation starts, and whether this impact has the potential to be significant or not. Assume a total aggregated number of 5000 write I/Os over two primary storage disk subsystems, with 2500 write I/Os per second to each storage disk subsystem. Each write I/O takes 0.5 ms. You specified 3 ms maximum to coordinate between the master storage disk subsystem and its subordinate storage disk subsystem.
Figure 25-3 on page 339 proposes spreading the B and C volumes over different ranks at the remote storage disk subsystem.
A B1 A Primary Primary A1 Primary Primary Se condar y Primary Rank 1 Rank 1 links FCP FCP A B2 Primary Primary Se condar y Primary Host channels C3 Tertiary PENDING PENDING A Primary Primary A2 Primary Primary PENDING PENDING Rank 2 Rank 2 A Primary Primary A3 Primary Primary A B3 Se condar y Primary C1 Tertiary Primary Primary C2 Tertiary PENDING PENDING Rank 3 Rank 3 FICON Host Host Local site Primary Primary Remote site A D1 Primary Primary A D3 Primary Primary D2
26 Chapter 26. Global Mirror examples In this chapter we provide examples that illustrate how to set up and manage a Global Mirror environment, using the available interfaces: TSO commands, ICKDSF, DS SM GUI, and DS CLI. The examples show how to: Query Global Mirror sessions. Establish a Global Mirror environment (paths, pairs. session). Deal with a primary site failure and the subsequent recovery at the backup site. Also, the management of a planned outage is discussed.
26.1 Global Mirror examples - configuration In order to illustrate the basic use of commands to manage a Global Mirror environment, the following examples are limited to a few volumes only to keep these examples as simple as possible and avoid confusing complexity. Figure 26-1 shows the volumes with their device numbers and their SSIDs, which are used throughout most of the examples within this chapter. Some other configuration data may be used for the DS CLI examples, or examples based on ICKDSF.
DVCSTAT GMLSTAT GMPSTAT The RQUERY command and its options have been explained in 24.3.8, “Query a Global Mirror session” on page 310. 26.2.2 Query Global Mirror volume status - DVCSTAT option The DVCSTAT option returns information pertaining to the volumes in the LSS where VOLSER points to. Note that the specified VOLSER itself does not have to be a Global Mirror volume. You direct this command to an LSS that is part of a Global Mirror session.
Example 26-3 RQUERY GMLSTAT output when not addressing an LSS with an active session RQUERY SNBR(01) VOLSER(XX2D00) ACTION(GMLSTAT) RQUERY Output Volser(XX2D00) Action(GMLSTAT) Version(001) SNbr GMLStat GoodCg Pct CrnBadCG TotBadCG LastGoodCGSCntlClock -- ---------- -------- --- -------- -------- -------------------01 NoConfig LSS 2D00 has an active Global Mirror volume but it is not the Master LSS.
26.2.5 Timing information Creating consistency groups relies also on internal timing information. It is important to understand that the internal timer in the DS6000 is not synchronized with an external clock. Example 26-6 shows an RQUERY command example with a TIME command just before the RQUERY command. The TSO TIME command displays the CPU based timer, which displays 10:09:32. Consistency Group drain time limit is 60 seconds or one minute.
Note: If you still have not defined the paths between the Master LSS and the Subordinate storage disk subsystem LSSs, then this is the time to do that before you continue on. 4. Define the Global Mirror session and create a session ID between 1 and 255. You define this session ID to the LSS that will become the Master, as well as to the LSSs in the Subordinate storage disk subsystems that are going to participate in this session.
Example 26-7 Define paths for Global Copy from A to B //* ---------------------------- TSO ----------------------------- *** //* ESTABLISH PATH(S) for Global Copy pairs *** //* -------------------------------------------------------------- *** //EPATHS EXEC PGM=IKJEFT01 //SYSPRINT DD SYSOUT=* //SYSTSPRT DD SYSOUT=* //SYSTSIN DD DDNAME=SYSIN CQUERY CESTPATH CQUERY CQUERY CESTPATH CQUERY DEVN DEVN PRIM SEC LINK (X'2C00') PATHS (X'2C00') + (X'2C00' 5005076303FFC228 X'0C') + (X'3C00' 5005076303FFC422 X'0C')
You may also consider defining paths in the reverse direction. After a site failover and before you return to the local site it may be necessary to resynchronize the volumes from the remote site to the local site. This requires paths from the remote site to the local site. In contrast to ESCON, FCP links allow the definition of paths in either direction over the very same link. 26.3.2 Establish Global Copy volume pairs Create Global Copy pairs as the next step.
Example 26-10 Verify that the establish Global Copy succeeded ************** PPRC REMOTE COPY CQUERY - VOLUME ******************** * (PRIMARY) (SECONDARY) * * SSID CCA LSS SSID CCA LSS* *DEVICE LEVEL STATE PATH STATUS SERIAL# SERIAL# * *------ --------- ---------- ----------- ----------------- * * 2D00 PRIMARY.. PENDING.XD ACTIVE.. 2D00 00 0D 3D00 00 0D * * CRIT(NO)....... CGRPLB(NO).
FCQUERY FCQUERY FCESTABL FCQUERY MODE (ASYNC) DEVN (X'3D00') DEVN (X'3D01') SDEVN(X'3D01') MODE (ASYNC) DEVN (X'3D01') ONLINTGT(YES) TDEVN(X'3F01') ONLINTGT(YES) + Note the ONLINTGT parameter. Although we used here YES to make sure that the FlashCopy command successfully completes, independent of whether the target volume is online or offline, we still recommend to use the default of NO. This ensures that the FlashCopy command is only successful when the target volume is offline.
RSESSION SNBR(01) VOLSER(XX2D00) ACTION(DEFINE) LSSTYPE(CKD) LSSNBR(0D) ESSSERIAL(27131) MSSERIAL (27131) + + + + Note that the VOLSER parameter is used to point to the LSSs that hold the Global Copy primary volumes that will be part of this Global Mirror configuration. These volumes have to be online to the system on which this job executes. This asks for a kind of utility volume within each LSS.
26.3.6 Start Global Mirror session The last step is to start the Global Mirror session. This will start forming Consistency Groups at the remote site. Also with this command, you are appointing the Master LSS. Example 26-15 shows the TSO command RSESSION, but this time with the ACTION(START) parameter option. As this parameter option implies, the microcode now starts to create Consistency Groups at the secondary storage disk subsystem.
FICON A 2C00 Primary A Primary Primary Primary A ty ac tiv i 2D01 2D00 FlashCopy 3D01 A A B 3D00 I/O Local site Host 3C00 Primary Primary Primary Primary Global Copy 3F01 A A C 3F00 3E00 Primary Primary Primary Primary Secondary Primary Tertiary 73081 27131 FCP links Remote site Figure 26-2 Global Mirror configuration before unplanned primary failure The production applications write to three primary volumes that are part of a Global Mirror session. 26.4.
Example 26-16 Simulate primary site failure through CGROUP FREEZE //* ---------------------------- TSO ------------ FAILO (2) ----//* SIMULATE PRIMARY FAILURE WITH CGROUP FREEZE/RUN //* HARDER PRIMARY FAILURE THROUGH VARY OFFLINE,FORCE //* -------------------------------------------------------------//FREEZE EXEC PGM=IKJEFT01 //SYSPRINT DD SYSOUT=* //SYSTSPRT DD SYSOUT=* //SYSTSIN DD DDNAME=SYSIN CGROUP DEVN (X'2C00') PRIM (X'2C00' 27131 X'0C') SEC (X'3C00' 73081 X'0C') FREEZE + + + CGROUP DEVN (X'2D00'
MSSERIAL (27131) Note: Address all Global Mirror session commands to the Master LSS. This is the LSS that was used when the Global Mirror START command was given 26.4.3 Failover from B to A volumes Once the primary volumes fail and the application I/O stops at the primary site, the status of the secondary volumes (B) is changed from secondary pending to primary suspended. This is done with the ACTION(FAILOVER) on the secondary volumes; see Example 26-18.
26.4.5 Create a data consistent set of B volumes The next step is to create a set of consistent volumes. This requires a specific FlashCopy operation, which is fast reverse restore, FRR; see Figure 26-4.
Example 26-20 Create optional second copy of data consistent D volumes //* ---------------------------- TSO ------------ CREATE (5) ----- *** //* FLASHCOPY B -> D TO CREATE COPY TO WORK (TEST) WITH *** //* -------------------------------------------------------------- *** //FCESTBL EXEC PGM=IKJEFT01 //SYSPRINT DD SYSOUT=* //SYSTSPRT DD SYSOUT=* //SYSTSIN DD DDNAME=SYSIN FCESTABL FCQUERY SDEVN(X'3C00') DEVN (X'3C00') TDEVN(X'3E02') FCESTABL FCQUERY SDEVN(X'3D00') DEVN (X'3D00') TDEVN(X'3F02') FCESTABL
26.4.8 Prepare to return to the local site At some time you will plan to return to the local site. The intention here is to apply the changes from the B volumes to the A volumes before resuming operations at the local site. Before you can apply these changes from the B to the A volumes, you have to establish paths from B to A. 26.4.
CESTPAIR MSGREQ(NO) OPTION(XD) DEVN (X'3D01') PRIM (X'3D00' 73081 X'01' SEC (X'2D00' 27131 X'01' ACTION(FAILBACK) MSGREQ(NO) OPTION(XD) CRIT(NO) + CASCADE(NO) + X'0D') + X'0D') + + CRIT(NO) + CASCADE(NO) 26.4.10 Return to the local site and resume Global Mirror At some time you will plan to return the application to the local site. This requires first that you quiesce the application at the remote site. Then, you execute an ACTION(FAILOVER)/ ACTION(FAILBACK) command sequence to the A volumes.
PRIM (X'2C00' 27131 X'00' SEC (X'3C00' 73081 X'00' ACTION(FAILBACK) MSGREQ(NO) OPTION(XD) CESTPAIR CESTPAIR DEVN (X'2D00') SEC (X'3D00' 73081 X'00' PRIM (X'2D00' 27131 X'00' ACTION(FAILBACK) MSGREQ(NO) OPTION(XD) DEVN (X'2D01') SEC (X'3D00' 73081 X'01' PRIM (X'2D00' 27131 X'01' ACTION(FAILBACK) MSGREQ(NO) OPTION(XD) X'0C') X'0C') + + + CRIT(NO) + CASCADE(NO) + + + + CRIT(NO) + CASCADE(NO) + X'0D') + X'0D') + + CRIT(NO) + CASCADE(NO) X'0D') X'0D') From now on the normal operation is restored, and as the
ACTION(STOP) LSSTYPE(CKD) LSSNBR(0C) ESSSERIAL(27131) MSSERIAL (27131) RQUERY RQUERY RQUERY + + + SNBR(01) VOLSER(XX2C00) ACTION(DVCSTAT) SNBR(01) VOLSER(XX2C00) ACTION(GMLSTAT) SNBR(01) VOLSER(XX2C00) ACTION(GMPSTAT) The GMPSTAT query shown in Example 26-25 informs that session 01 is not active any longer. Numbers are reset to zeros.
Example 26-27 Query Global Mirror volumes after removal RQUERY SNBR(01) VOLSER(XX2C00) ACTION(DVCSTAT) RQUERY Output Volser(XX2C00) Action(DVCSTAT) Version(001) SNbr LSS Dvc VolStat PriPPRCStat SecCascStat -- -- -- -------------------- ---------- ---------01 NoVolumes READY RQUERY SNBR(01) VOLSER(XX2D00) ACTION(DVCSTAT) RQUERY Output Volser(XX2D00) Action(DVCSTAT) Version(001) SNbr LSS Dvc VolStat PriPPRCStat SecCascStat -- -- -- -------------------- ---------- ---------01 NoVolumes 26.5.
Example 26-29 Remove FlashCopy relationships //* ------------------ TSO ----------------- CLEANUP (4) --------- *** //* Remove FlashCopy relationship from B -> C *** //* -------------------------------------------------------------- *** //FCWDRW EXEC PGM=IKJEFT01 //SYSPRINT DD SYSOUT=* //SYSTSPRT DD SYSOUT=* //SYSTSIN DD DDNAME=SYSIN FCQUERY FCWITHDR FCQUERY DEVN (X'3C00') SDEVN(X'3C00') DEVN (X'3C00') TDEVN(X'3E00') FCQUERY FCWITHDR FCQUERY DEVN (X'3D00') SDEVN(X'3D00') DEVN (X'3D00') TDEVN(X'3F00')
Example 26-31 Delete Global Copy volume pairs //* ------------------ TSO ---------------- CLEANUP (5) ---------//* Delete Global Copy volumes pairs //* //* DELETE PRIMARY X'2C00' //* X'2D00' //* X'2D01' //* //* PRIM(X'2C00' 27131 X'00' X'0C') //* SSID ##### CCA LSS //* -------------------------------------------------------------//DPAIR EXEC PGM=IKJEFT01 //SYSPRINT DD SYSOUT=* //SYSTSPRT DD SYSOUT=* //SYSTSIN DD DDNAME=SYSIN //SYSTSIN DD DDNAME=SYSIN CQUERY CQUERY CQUERY DEVN(X'2C00') DEVN(X'2D00') DEVN(X'
* PRIMARY.... 5005076303FFC228 5.0.03.0097 * * SECONDARY.1 5005076303FFC422 * ******************************************************************** Example 26-33 CQUERY after deleting the Global Copy pair READY CQUERY DEVN(X'2C00') CQUERY FORMATTED LVL 3 VOLUME REPORT ************** PPRC REMOTE COPY CQUERY - VOLUME ******************** * (PRIMARY) (SECONDARY) * * SSID CCA LSS SSID CCA LSS* *DEVICE LEVEL STATE PATH STATUS SERIAL# SERIAL# * *------ --------- ---------- ----------- ----------------- * * 2C00 ..
Example 26-35 Remove the paths //* ------------------ TSO ----------------- CLEANUP (6) --------- *** //* Delete path(s) this completes the GM cleanup *** //* -------------------------------------------------------------- *** //DPATH EXEC PGM=IKJEFT01 //SYSPRINT DD SYSOUT=* //SYSTSPRT DD SYSOUT=* //SYSTSIN DD DDNAME=SYSIN CQUERY CDELPATH CQUERY CQUERY CDELPATH CQUERY DEVN(X'2C00') DEVN(X'2C00') PRIM(X'2C00' SEC (X'3C00' DEVN(X'2C00') PATHS DEVN(X'2D00') DEVN(X'2D00') PRIM(X'2D00' SEC (X'3D00' DEVN(X'2D
Figure 26-6 summarizes the scenario. The numbers indicate the sequence of events and main considerations, and also relate to Figure 26-7 where the corresponding ICKDSF commands are shown. The sequence of steps for a Global Mirror planned outage is as follows: 1. Pause the Global Mirror session. 2. Suspend Global Copy pairs. 3. Application I/O continues all the time. 4. Failover from B to A. This turns the B volumes into Global Copy primary suspended. 5. Create a data consistent set of B volumes.
For ICKDSF command details, refer to Device Support Facility User’s Guide and Reference, SC35-0033. 26.7 Remove a Global Mirror environment using ICKDSF This section shows how to use ICKDSF to end and remove a Global Mirror environment that was previously established using TSO commands. The steps of the procedure to follow are similar for any of the interfaces. In our example in this section, we use ICKDSF to accomplish the associated task for each step.
Example 26-37 Remove volumes from a Global Mirror session with ICKDSF //* -------------------------------------------------------------- *** //REMOVE EXEC PGM=ICKDSF //SYSPRINT DD SYSOUT=* //DD01 DD UNIT=3390,VOL=SER=AA6000,DISP=SHR //SYSIN DD * PPRCOPY DDNAME(DD01) POPSESS REMOVE SESSNO (001) VOLCOUNT (1) RANGE (YES) RVOLLIST ((X'00',X'03')) - 26.7.3 Withdraw FlashCopy relationships We are going to withdraw the FlashCopy relationships between the B and C volumes using ICKDSF inband commands.
Note the different positional parameters for SRCVOL and TGTVOL. It is possible to mix these up. First comes the LSS number and then the CCA. This is the same for both parameters. The third positional parameter for SRCVOL is the SSID, followed by the serial number of the secondary storage disk subsystem. The third parameter in TGTVOL is the actual z/OS device number of the FlashCopy target volume. Withdraw can be abbreviated as WD.
PPRCOPY DDNAME(DD03) DELPAIR LSS(X'00' X'00') PRI(X'2060' AAGXA X'02') SEC(X'0002' AAVCA X'02') PPRCOPY DDNAME(DD04) DELPAIR LSS(X'00' X'00') PRI(X'2060' AAGXA X'03') SEC(X'0002' AAVCA X'03') For each single Global Copy pair you must specify a corresponding PPRCOPY command. The syntax for the ICKDSF command is slightly different from the equivalent TSO command.
ICK04030I ICK02204I ICK02230I ICK00001I ADDITIONAL DEVICE INFORMATION = 4800243D TRKS/CYL = 15, # PRIMARY CYLS = 3339 DEVICE IS A PEER TO PEER REMOTE COPY VOLUME PPRCOPY DELPAIR FUNCTION COMPLETED SUCCESSFULLY DEVICE IS NOW IN SIMPLEX STATE FUNCTION COMPLETED, HIGHEST CONDITION CODE WAS 0 09:36:00 06/16/05 26.7.6 Remove Global Copy paths Finally, remove the Global Copy paths. Use the DDNAME parameter to point to the primary LSS, as shown in Example 26-43.
26.9 Set up a Global Mirror environment using DS SM You may establish a Global Mirror environment with the DS Storage Manager (DS SM) that provides a graphical user interface (GUI). A Global Mirror session relies on the following components, which are independent of the interface used, but are repeated here in case you are only reading this section: Paths for Global Copy communication between primary and secondary LSSs.
26.9.1 Create paths When using the DS Storage Manager to create paths between two LSSs on two different Storage Images, it is necessary to go through a six-step process creation wizard. This wizard needs to be repeated for each data path to be defined. Remember that you have to define paths between site 1 and site 2, for Global Copy replication, and eventually you will also have to define paths within site 1 if the Master and Subordinates are on different Storage images.
The creation wizard then displays the “Select target LSS” panel; see Figure 26-10. Here you select from the pull-down lists the Storage complex, then the Storage unit, then the Storage Image and finally the LSS, which contains the corresponding target volumes of the Global Copy pairs. Click Next to proceed with the third step of this wizard. Figure 26-10 Global Copy paths creation step 2 - select the target LSS Then the creation wizard displays the “Select source I/O ports” panel; see Figure 26-11.
Then the creation wizard displays the Select target I/Os ports panel; see Figure 26-12. Here you select from the pull-down list the target I/O ports that you intend to connect with the source I/O ports. Figure 26-12 Global Copy paths creation step 4 - select the target I/O ports Click Next to proceed with the next step of this wizard. Then the creation wizard displays the “Select path options” panel; see Figure 26-13. Here you have a box that can be checked for the option Define as Consistency Group.
This brings you to the Verification panel; see Figure 26-14. Here you can check all the components of your path definitions and, if necessary, click Back to correct any of them, or click Finish to validate the configuration and end the wizard. Figure 26-14 Global Copy path creation step 6 - verification 26.9.2 Create Global Copy volume pairs To create Global Copy volume pairs for a Global Mirror session, using the DS Storage Manager, you have to go through another staged process with corresponding panels.
Then the creation wizard displays the Volume Pairing Method panel; see Figure 26-16. Here you select Manual volume pair assignment. Figure 26-16 Global Copy creation step 1 - choose volume pairing method Click Next to proceed with the second step of this wizard. Then the creation wizard displays the “Select source volumes” panel; see Figure 26-17.
In the “Select source volumes” panel (see Figure 26-17), select from the pull-down lists the Storage complex, then the Storage unit, then the Storage Image, then the Resource type and, if necessary, its appropriate parameter to display the list of volumes. If you have chosen the Resource type LSS, select from the pull-down list the LSS number that contains the source volumes you intend to use. Then, check the boxes for the source volumes you plan to use as Global Copy pairs in the Global Mirror session.
Select from the pull-down lists the Storage complex, then the Storage unit, then the Storage Image, then the Resource type and, if necessary, its appropriate parameter to display the list of volumes. Click the required Storage Unit, then on the required LSS check the box for the volume you want to use as target. Click Next to proceed with the selection of the second target volume. Again the “Select target volumes” panel is displayed.
Figure 26-20 Global Copy creation step 4 - select the copy options Click Next to proceed with the last step of this wizard. The Verification panel is displayed; see Figure 26-21. In this panel check all the components of your Global Copy session configuration, and, if necessary, click Back to correct any of them, or click Finish to validate. In Figure 26-20 you may move back to correct the Permit read access from target. Figure 26-21 Global Copy creation step 5 - verification 26.9.
To launch this wizard, you need to first go to the FlashCopy panel under the Copy Services menu of DS Storage Manager GUI; see Figure 26-22. In the Select Action pull-down list choose Create to proceed with the first step of the wizard. Figure 26-22 FlashCopy creation - launch the creation process Note: If FlashCopy pairs are in several LSSs, do not forget to select all of them during this wizard, or run the wizard again on each LSS.
use. Select with the check boxes the source volumes you plan to use for FlashCopy within the Global Mirror session. Figure 26-24 FlashCopy creation step 2 - select the source volumes Note: Although the following panels show open systems volumes, the panels and their order apply also to CKD volumes. Click Next to proceed with the third step of this wizard. The creation wizard then displays the “Select target volumes” panel; see Figure 26-25.
When the creation wizard displays the “Select target volumes” panel (see Figure 26-25), select from the pull-down lists the Resource type and if necessary its appropriate parameter to display the list of volumes. If you have chosen the Resource type LSS, select from the pull-down lists the LSS number that contains the target volumes you plan to use. Check the boxes of the volumes you want to use as target FlashCopy volumes. Click Next to proceed with the fourth step of this wizard.
Figure 26-27 FlashCopy creation step 5 - verification 26.9.4 Create a Global Mirror session To define a Global Mirror session with the DS Storage Manager, you go through another staged panel process. To launch this wizard you first need to go to the Global Mirror panel under the Copy Services menu of DS Storage Manager GUI; see Figure 26-28. In the Select Action pull-down list, choose Create to proceed with the first step of the wizard.
Figure 26-29 Global Mirror session creation step 1 - select volumes 386 IBM System Storage DS6000 Series: Copy Services with IBM System z
Click Next to proceed with the second step of this wizard. The creation wizard then displays the “Define properties” panel; see Figure 26-30. Figure 26-30 Global Mirror creation step 2 - define properties When the creation wizard displays this panel, complete the “Enter session number” field with the appropriate session number. You can click Get Available Session Numbers if you have forgotten the one you want to use.
When Global Mirror cannot complete replicating all outstanding data within this time window, then the process to form a new Consistency Group fails. After five contiguous failures due to this condition, Global Mirror will force the formation of a new Consistency Group independent of how long it takes to replicate all remaining data from the primary to secondary volumes. See also 22.4, “Consistency Groups” on page 260. Click Next to proceed with the third and last step of this wizard.
Figure 26-32 Global Mirror - visualize the session status 26.10 Set up a Global Mirror environment using the DS CLI You may consider implementing a standard task such as creating a Global Mirror session, through a more automated process than the DS SM graphical interface approach. The DS CLI provides a good approach, which is similar to what you may do with REXX procedures, or other software-based tools in System z environments, to assist when managing remote copy configurations.
26.10.1 DS CLI profile files To simplify DS CLI commands, we use configuration files. Example 26-45 and Example 26-46 show the configuration files that we are using in our environment. These profiles are stored in the default directory c:\Program Files\IBM\dscli\Profile for Windows systems. Because we are using two storage disk subsystems, and there is a local entry and a remote entry in these files indicated by devid and remotedevid, we created two profile files.
Example 26-46 Sample DS CLI configuration file DS-02.profile hmc1: xxx.yyy.zzz.yyy username: admin password: passw0rd devid:IBM.2107-7573731 remotedevid:IBM.2107-7506551 fullid: off banner: on verbose: off paging: off olc:off 26.10.2 Create paths When using the DS CLI to create new paths from a source LSS to a target LSS, use the procedure we discuss here. This process must be repeated for every path you need to define.
To define a path, use the following command: dscli mkpprcpath -remotewwnn -consistgrp -srclss -tgtlss -cfg : : ... Note: The consistgrp option is intended for Metro Mirror. Example 26-48 illustrates how to create a path from the source LSS 65 in the storage disk subsystem at site 1, to the target LSS 65 in the storage disk subsystem at site 2.
Once you identify your Global Copy pairs, use the following DS CLI command to create the pairs: dscli mkpprc -type gcp -tgtread -mode full -cfg : : ... Note: The type gcp option is required to create Global Copy pairs. Use the option mode full to force a first full replication of the Global Copy pair.
Example 26-52 shows the command that creates two FlashCopy relationships. Example 26-52 mkflash to establish FlashCopy relationships for Global Mirror dscli mkflash -record -nocp -cfg $DSCLI/profile/DS-02.profile 6500:6502 6501:6503 Date/Time: June 14, 2005 3:28:58 PM EDT IBM DSCLI Version: 5.0.3.134 DS: IBM.2107-7573731 CMUC00137I mkflash: FlashCopy pair 6500:6502 successfully created. CMUC00137I mkflash: FlashCopy pair 6501:6503 successfully created.
drain - Specifies the maximum Consistency Group drain time in seconds, and is the maximum amount of time that the consistent set of data is allowed to drain to the remote site before failing the current Consistency Group formation. The default is 30 seconds. For a detailed discussion, see 22.4, “Consistency Groups” on page 260. The command in Example 26-54 starts the session and the actual Consistency Group creation process. It uses the default session parameters.
If we query the volumes 6500 and 6501, that are in the storage disk subsystem at site 1, and have target volumes 6500 and 6501 in the storage disk subsystem at site 2, we will see the result shown in Example 26-56. Example 26-56 lspprc command output dscli lspprc -l -cfg $DSCLI/profile/DS-01.profile 6500-6501 Date/Time: June 14, 2005 3:28:08 PM EDT IBM DSCLI Version: 5.0.3.134 DS: IBM.
Example 26-58 lssession command (volumes not in session) dscli lssession 65 -cfg $DSCLI/profile/DS-01.profile Date/Time: June 14, 2005 3:30:10 PM EDT IBM DSCLI Version: 5.0.3.134 DS: IBM.
Example 26-61 showgmir of running session with Master and Subordinate dscli showgmir 65 -cfg $DSCLI/profile/DS-01.profile Date/Time: June 14, 2005 5:00:35 PM EDT IBM DSCLI Version: 5.0.3.134 DS: IBM.2107-7506551 ID IBM.
Current Time CG Time Successful CG Percentage Successful CG Percentage FlashCopy Sequence Number Master ID Subordinate Count Master SSID Subordinate SSID 06/15/2005 11:37:46 EDT 06/15/2005 11:36:01 EDT 100 94 0x42AF463B IBM.2107-7506551 1 0xFF65 0xFF67 26.11.6 Pause Global Mirror session Use the pausegmir command to pause Global Mirror processing. This allows you to temporarily suspend Global Mirror processing attempts to form consistency groups, for the specified session.
dscli resumegmir -lss -cginterval -coordinate -drain -session -cfg : ... One reason you would pause and resume a session is to be able to modify the Global Mirror session tuning parameters.
26.12 Site switch basic operations using the DS CLI In this section we show examples of how to do the steps to effect a site switch, using DS CLI commands. For a detailed discussion regarding site swap considerations see 26.4, “Primary site failure and recovery management with TSO” on page 352. 26.12.1 Perform a Global Copy failover When switching sites, use the failoverpprc command to change a secondary device into a primary suspended device, while leaving the primary device in its current state.
Example 26-69 failback - erasing data on site 1 with data from site 2 dscli failbackpprc -type Date/Time: June 15, 2005 CMUC00197I failbackpprc: CMUC00197I failbackpprc: gcp -tgtread 6500:6500 6501:6501 -cfg $DSCLI/profile/DS-02.profile 12:32:27 PM EDT IBM DSCLI Version: 5.0.3.134 DS: IBM.2107-7573731 Remote Mirror and Copy pair 6500:6500 successfully failed back. Remote Mirror and Copy pair 6501:6501 successfully failed back. To query Global Copy information use the command lspprc as described in 26.11.
To query the newly created FlashCopy relationships use the command lsflash as described in 26.11.3, “Query FlashCopy pairs” on page 396. 26.12.4 Verify FlashCopy status between B and C volumes After a local site failure, you will need to check the status of all FlashCopy pairs that were in the Global Mirror session. Let us see how to verify and eventually take some corrective actions on the Global Mirror FlashCopy pairs at the remote site, when there has been a primary site failure.
4. If some FlashCopy pairs are revertible and at least one is not revertible, but all their sequence numbers are equal, it means that some FlashCopy pairs are running in a Consistency Group process and some have already finished their incremental process. We need to commit data to a target volume to form a consistency between the source and target.
The Global Mirror environment cleanup process can be structured in five consecutive steps: 1. 2. 3. 4. 5. End Global Mirror processing. Remove the Global Mirror volumes and the session from the involved LSSs. Remove FlashCopy pairs. Remove Global Copy pairs. Remove paths: a. Between source and target Global Copy LSSs. b. Within site 1, between the Master and the Subordinate LSSs. This task is not required if you have only one source storage image. 26.13.
Example 26-75 rmsession dscli rmsession -quiet -lss 65 01 -cfg $DSCLI/profile/DS-01.profile Date/Time: June 15, 2005 2:20:05 PM EDT IBM DSCLI Version: 5.0.3.134 DS: IBM.2107-7506551 CMUC00146I rmsession: Session 01 closed successfully. Note: The quiet option turns off the confirmation prompt for this command. 26.13.3 Remove FlashCopy pairs To remove the FlashCopy pairs you use the command rmflash.
dscli>rmpprcpath -dev storage_image_ID -remotedev storage_image_ID -remotewwnn wwnn source_LSS_ID:target_LSS_ID ... You need to repeat this process for each data path between the Global Copy source and target storage images, as well as for the paths that connect the Global Mirror Master with its Subordinates if there was more than one source storage image involved. Example 26-78 rmpprcpath dscli rmpprcpath -quiet 65:65 -cfg $DSCLI/profile/DS-01.
408 IBM System Storage DS6000 Series: Copy Services with IBM System z
Part 7 Part 7 Interoperability In this part we discuss the interoperability of the DS6000 Copy Services functions with other IBM storage disk subsystems. © Copyright IBM Corp. 2006. All rights reserved.
410 IBM System Storage DS6000 Series: Copy Services with IBM System z
27 Chapter 27. Combining Copy Service functions In this chapter we discuss the interoperability of Global Copy, Metro Mirror, and FlashCopy. Examples and suggestions are also given for this interoperability. © Copyright IBM Corp. 2006. All rights reserved.
27.1 Data migration Combining the Copy Service functions Metro Mirror and Global Mirror into an environment with double cascading makes it possible to maintain data consistency while migrating data. There are two possibilities for accomplishing data consistency during a migration using an environment that has double cascading. The first possibility is to shut down all applications at the local site and let the out-of-sync drain completely.
consistent with the secondary. The local during this process is still being updated and is changing. The remote will have a snapshot of the time since the freeze was issued.
414 IBM System Storage DS6000 Series: Copy Services with IBM System z
28 Chapter 28. Interoperability between DS6000 and DS8000 In this chapter we show the interoperability between the Copy Services functions in the DS6000 and the DS8000.
28.1 DS6000 and DS8000 Copy Services interoperability Copy Services operations are supported between the DS6000 and the DS8000. This means you can have a mixed Copy Services environment that contains both devices. 28.2 Preparing the environment Before starting Copy Services operations in a mixed DS6000 and DS8000 environment, you must ensure that this environment is set up correctly. 28.2.
– In a mixed environment, one system is the source (local) and one is the target. If you want to be able to manage the PPRC paths and pairs from either machine, then you need network connectivity between the DS6000 SMC and the DS8000 HMC. If you use DS CLI, the SMC and HMC do have to be able to communicate with each other, but the DS CLI machine must be able to communicate with both the HMC and the SMC.
Putting the password in the profile is not very secure (because it is stored in a plain text file), but it might prove more convenient. A password file can be created using managepwfile and is a better way to manage this. After creating the password file (which by default is called security.dat), you can remove the password from the profile and instead specify the pwfile file.
Figure 28-1 Add DS6000 complex to DS8000 7. Enter the IP address of the DS6000 SMC into the Management console 1 IP field. If you have a second SMC, select Define a second Management console and enter the second SMC address into the Management console 2 IP field. Click OK. 8. When the add Storage Complex operation is complete, the Storage Complex panel shows the DS6000 SMC as an additional Storage Complex. 9.
Figure 28-2 Add DS8000 complex to DS6000 6. Enter the IP address of the DS8000 HMC in the Management console 1 IPfield. If you have a second DS8000 HMC, select Define a second Management console and enter the second HMC address in the Management console 2 IP field. Click OK. 7. When the add Storage Complex operation completes, the Storage Complex panel shows the DS8000 HMC as an additional Storage Complex.
Even when you have successfully added one complex to another, new user IDs created in one complex do not mirror to the other complex. They must be manually created and managed in each complex. Storage management You cannot use the DS8000 DS GUI to perform storage configuration on a DS6000. Likewise, you cannot use the DS6000 DS GUI to perform storage configuration on a DS8000. You can only perform Copy Services management tasks on the alternative device.
For DS8000, port IDs look like I0000 or I0123, which actually breaks out as IEECP. The EE is the enclosure number (minus 1), C is the card slot number (minus 1), and P is the port number on the card. So I0123 is enclosure 2 (1+1), slot 3 (2+1), port 3. For DS6000, port IDs range from I0000 to I0001 and I0100 to I0103, which actually breaks out as IEECP. The EE is the controller number (00 or 01), C is the card slot number (always zero), and P is the port number on the card (0 to 3).
Name ID Storage Unit Model WWNN State ESSNet ============================================================================ IBM.1750-1300247 IBM.1750-1300247 511 500507630EFFFE16 Online Enabled Determining the DS8000 WWNN with the DS GUI To determine the DS8000 WWNN with the DS GUI: 1. Click Real-time manager. 2. Click Manage hardware. 3. Click Storage images (not the Storage Unit). 4. Click in the Select column for the DS8000 Storage Image, and from the Select Action menu, select Properties. 5.
Example 28-7 Using DS CLI to establish PPRC paths dscli> mkpprcpath -remotedev IBM.1750-1300247 -remotewwnn 500507630EFFFE16 -srclss 14 -tgtlss 18 I0001:I0103 Date/Time: 3 November 2005 20:37:09 IBM DSCLI Version: 5.1.0.204 DS: IBM.2107-7503461 CMUC00149I mkpprcpath: Remote Mirror and Copy path 14:18 successfully established. dscli> lspprcpath 14 Date/Time: 3 November 2005 20:37:14 IBM DSCLI Version: 5.1.0.204 DS: IBM.
The command breaks out as follows: Device that command is issued to:E200 Primary LCU SSID: 6000 Primary DS6000 WWNN:500507630EFFFE16 Primary DS6000 LSS ID:06 Secondary DS8000 LCU SSID:2105 Secondary DS8000 WWNN:5005076303FFC08F Secondary DS8000 LSS ID:05 First path established: I0003 to I0003 Second path established:I0103 to I0103 28.4 Managing Metro Mirror or Global Copy pairs Having established paths, you can now establish volume pairs. 28.4.
In Example 28-9 on page 425, we connected to the DS8000 HMC with the DS CLI, so the DS6000 is the remote device. To establish pairs where the DS6000 is the source device, we must connect to the DS6000 with DS CLI. This makes the DS8000 the remote device. Important: If you specify the wrong -remotedev or you are using a profile where a remote device that is not the remote device that you want to use is specified, you might get an error message, CMUN03057, that you are specifying an invalid subsystem ID.
Example 28-11 Establishing Global Mirror using DS CLI dscli> mkpprc -remotedev IBM.1750-1300247 -type gcp -tgtread -mode full 0801:0601 Date/Time: 9 November 2005 2:45:53 IBM DSCLI Version: 5.1.0.204 DS: IBM.2107-7503461 CMUC00153I mkpprc: Remote Mirror and Copy volume pair relationship 0801:0601 successfully created. dscli> lspprc 0801 Date/Time: 9 November 2005 2:46:04 IBM DSCLI Version: 5.1.0.204 DS: IBM.
the DS8000 to volume 0601 on the DS6000. We then created a remote FlashCopy on the DS6000 between DS6000 volumes 0601 and 0602. We then removed the remote FlashCopy. Example 28-12 Creating a remote FlashCopy where the DS6000 is the remote target dscli> lspprcpath 08 Date/Time: 9 November 2005 3:01:10 IBM DSCLI Version: 5.1.0.204 DS: IBM.
Performance mismatch (running applications) Suppose a disaster or failure occurs and applications failover to the secondary (or recovery) site and are running using the secondary Storage Units. If the secondary Storage Unit, the DS6000, is less capable (in performance) than the primary Storage Unit, it is likely that you will not be able to complete primary business applications in the required or expected time frame.
430 IBM System Storage DS6000 Series: Copy Services with IBM System z
Part 8 Part 8 Solutions In this part we discuss solutions offered by IBM to assist you in the management, automation, and control of your Copy Services implementation on the DS6000. We provide an overview of the solutions, discuss the options available, and give some examples of using the solutions. © Copyright IBM Corp. 2006. All rights reserved.
432 IBM System Storage DS6000 Series: Copy Services with IBM System z
29 Chapter 29. Interoperability between DS6000 and ESS 800 In this chapter we show the interoperability between the Copy Services functions in the DS8000 and the ESS 800.
29.1 DS6000 and ESS 800 Copy Services interoperability Copy Services operations are supported between the DS6000 and the ESS 800 and Model 750. For the rest of this chapter, all references to the ESS 800 also apply to the ESS 750. On the ESS 800, RMC is called PPRC. All references to PPRC are interchangeable with RMC. 29.2 Preparing the environment Before starting Copy Services operations in a mixed DS6000 and ESS 800 environment, you must ensure that this environment is set up correctly. 29.2.
If he ESS 800 is going to be purely a remote target for PPRC, and you do not plan to use it as a source server or use DS GUI to manage the pairs and paths, then you do not need to have network connectivity to the ESS 800 Copy Services servers. This is because all path and pair establishment is done by connecting to the source server (which would be the DS6000). This setup is not recommended because it is less flexible. 29.2.
# Username is a user created on the ESS Specialist that matches the userid on the DS6000 username:admin # The password for the admin user id. Placing it here is not very secure. password:passw0rd # The password file is created using the managepwfile and is a better way to manage this. # pwfile:security.dat Putting the password in the profile is not very secure (because it is stored in plain text), but it can be more convenient.
Figure 29-1 Add 2105 Copy Services Domain 6. Enter the IP address of the ESS Copy Services server A in the Server 1 IP field. If you have a Server B, select Define a second Copy Services server and enter the Server B IP address in the Server 2 IP address. However, if Server B is running on a 2105-F20, you should not define it. Having added the ESS 800 Copy Services domain to the DS GUI, you are now able to use the DS GUI to create paths and RMC pairs, where the ESS 800 is the source device.
if you use these volumes in a PPRC relationship, their partner volumes should also be 3339 cylinders. Example 29-3 CKD volume size in DS CLI dscli> mkckdvol -extpool P2 -cap 3339 -name av_6K_CKD_#h 0600-0601 Date/Time: 8 November 2005 22:58:31 IBM DSCLI Version: 5.1.0.204 DS: IBM.1750-1300247 CMUC00021I mkckdvol: CKD Volume 0600 successfully created. CMUC00021I mkckdvol: CKD Volume 0601 successfully created. dscli> lsckdvol -lcu 06 Date/Time: 9 November 2005 3:58:42 IBM DSCLI Version: 5.1.0.204 DS: IBM.
Example 29-4 Using ESS CLI to list CKD volumes C:\Program Files\ibm\ESScli>esscli -u copy -p serv1ces -s 9.155.51.237 list volume Wed Nov 09 04:07:52 EST 2005 IBM ESSCLI 2.4.0 Volume Cap Units VolType LSS VS Serial Label ------ ----- ----- ------- --- ---- -------- ----0500 3339 Cyls 3390 05 vs6 *** *** 0501 3339 Cyls 3390 05 vs6 *** *** 0502 3339 Cyls 3390 05 vs6 *** *** 0503 3339 Cyls 3390 05 vs6 *** *** Note: You cannot use the DS CLI to check volume sizes on an ESS 800.
00 04 08 Slot 1 Slot 2 Slot 3 0C Slot 4 20 24 28 Slot 1 Slot 2 Slot 3 Host Bay 1 Host Bay 2 2C Slot 4 80 84 88 Slot 1 Slot 2 Slot 3 Host Bay 3 8C Slot 4 A0 A4 A8 Slot 1 Slot 2 Slot 3 AC Slot 4 Host Bay 4 Figure 29-3 ESS 800 I/O ports decoded 29.3.2 Creating paths with the DS GUI In this example, we show how to establish two PPRC paths from DS6000 LSS 06 to ESS LSS 05 with the DS Storage Manager. Tip: CKD LCUs on an ESS 800 are always LSS 00 to LSS 0F.
Figure 29-5 Select source LSS 7. To specify the target LSS, select the Storage Complex for the ESS from the Storage Complex menu. Then, from the Storage Unit menu, select the appropriate Storage Unit and the LSS from the Storage Unit. When you are finished, click Next to continue. In Figure 29-6 the target LSS on the ESS is LSS 05. Figure 29-6 Select target LSS 8. The next panel shows you which Fibre Channel ports are available for establishing Metro Mirror and Copy paths (see Figure 29-7 on page 442).
Figure 29-7 Select source I/O ports 9. In the next panel (see Figure 29-8), you select, for each source I/O port from the DS6000, a target I/O port on the ESS. When you are finished, click Next. Figure 29-8 Select target I/O ports 10.In the next two panels, you are asked whether you want built a Consistency Group and to verify the information that you entered during the process. Verify the information and click Finish to establish the PPRC paths.
Figure 29-9 Path panel Paths in the reverse direction The previous example showed how to create a path from the DS6000 to the ESS 800. In many cases, it is likely that you will need paths from ESS 800 to the DS6000. Because you have established paths in one direction, you can now establish paths in the opposite direction. Fibre Channel allows bidirectional mirroring over the same physical path. Adding or deleting paths You can add additional paths to an LSS pair by simply creating more paths.
Determining DS6000 WWNN with DS CLI In Example 29-5, we show how to display the WWNN of a Storage Image using the lssi command. Example 29-5 Determine the WWNN of a DS6000 dscli> lssi Date/Time: 3 November 2005 1:05:50 IBM DSCLI Version: 5.1.0.204 Name ID Storage Unit Model WWNN State ESSNet ============================================================================ IBM.1750-1300247 IBM.
Listing the available ports using DS CLI Having determined WWNN of the remote devices, we can now display the ports that are available for establishing PPRC paths. In Example 29-7, we are logged on to the DS6000 using DS CLI, so the remote device is the ESS 800. Note: When using the lsavailpprcport command, you must specify LSSs that actually exist. Example 29-7 Displaying available ports for PPRC path establishment dscli> lsavailpprcport -remotedev IBM.
========================================================= 06 05 Success 2105 I0003 I00AC 5005076300C09517 06 05 Success 2105 I0103 I000C 5005076300C09517 To establish paths where the ESS 800 is the source, you should connect to the ESS 800 using the DS CLI and follow the same process, but specify the DS6000 as the remote device. Important: When you connect using DS CLI to the DS6000 SMC, the DS6000 is the local device and the ESS 800 is the remove device.
Metro Mirror pairs. Volumes 0600 and 0601 from the DS6000 are the source volumes, and the target volumes are 0500 and 0501 from the ESS 800. We followed the same method to set up a Global Copy pair, except that Global Copy is selected (Figure 29-16 on page 449): 1. Click Real-time manager. 2. Click Copy services. 3. Click Metro Mirror. 4. Select the Storage complex, Storage unit, and Storage image for the Metro Mirror source volumes. 5. Select Create (Figure 29-11).
Figure 29-13 Select source volume 8. In the next panel, select the Storage Complex and Storage Unit for the LSS with the target volumes. Then, select the target volume for the first source volume. When you are finished, click Next as shown in Figure 29-14. To expand your choices, you must select the small blue boxes. Figure 29-14 First Metro Mirror target volume 9. Next, select the target volume for the second source volume as shown in Figure 29-15 on page 449.
Figure 29-15 Second Metro Mirror target volume 10.IIn the next panel, you can specify various copy options as shown in Figure 29-16. In this example, we selected Metro Mirror under Define relationship type and Perform initial copy. If you wished to instead use Global Copy, this is where you would select it. Figure 29-16 Copy options 11.The Verify panel opens. Verify the information that you entered, and if everything is correct, click Finish to establish the Metro Mirror volume pairs.
Figure 29-17 Managing Metro Mirror pairs 29.4.2 Managing Metro Mirror pairs with the DS CLI In this example, we show how you can establish Metro Mirror volume pairs between a DS6000 and an ESS 800 with the DS CLI. We created two Metro Mirror pairs. Volumes 0600 and 0601 from the DS6000 are the source volumes, and the target volumes on the ESS are 0500 and 0501. Example 29-11, we created the pairs with the mkpprc command.
29.4.3 Creating Metro Mirror pairs with TSO In Example 29-12, we used the same paths that were created in 29.3.4, “Creating paths using TSO” on page 446. We established a volume pair with Metro Mirror from the DS6000 to the ESS 800.
device. Then, we created session 20 for LSS 05. Finally, we created a Global Mirror using session 20 and LSS 05. Example 29-14 Establishing Global Mirror using DS CLI dscli> mkpprc -remotedev IBM.1750-1300247 -type gcp 0500:0600 Date/Time: 9 November 2005 1:50:05 IBM DSCLI Version: 5.1.0.204 DS: IBM.2105-22399 CMUC00153I mkpprc: Remote Mirror and Copy volume pair relationship 0500:0600 successfully created. dscli> mkremoteflash -dev IBM.1750-1300247 -conduit IBM.
29.6.1 Creating an ESS 800 FlashCopy with the DS GUI The steps to create an ESS 800 FlashCopy using the DS GUI are: 1. Click Real-time manager. 2. Click Copy Services. 3. Click FlashCopy. 4. From the Storage Complex menu, choose the ESS 800 Copy Services Domain. 5. From the Storage Units menu, select the ESS that you wish to perform the FlashCopy on 6. Make selections from the Resource type and Specify LSS pull-downs. 7. From the Select Action menu, select Create, as shown in Figure 29-18.
12.The verification panel opens. Review your selections and click Finish. 29.6.2 Creating an ESS 800 FlashCopy with the DS CLI You must start the DS CLI and connect to the ESS 800. Then issue DS CLI commands as usual. An example is shown in Example 29-15. Example 29-15 Using DS CLI to create an ESS 800 FlashCopy dscli> mkflash -nocp -record -persist 0500:0501 Date/Time: 2 November 2005 20:04:33 IBM DSCLI Version: 5.1.0.204 DS: IBM.
Note: Commands that work with remote FlashCopies use the -dev parameter to define the machine on which the FlashCopy is to be performed. Other commands, such as mkpprc commands, refer to this device as the remote device, using -remotedev. However, because a Flashcopy must be sent to the remote site and then performed locally there, the use of the -dev parameter to refer to the remote machine is correct. Chapter 29.
456 IBM System Storage DS6000 Series: Copy Services with IBM System z
30 Chapter 30. IIBM TotalStorage Rapid Data Recovery IBM TotalStorage Rapid Data Recovery is a service offering solution from IBM Global Services based on enterprise Remote Copy Management Facility (eRCMF). eRCMF is a standalone tool that provides a management layer for ESS800, DS8000, and DS6000 Copy Services in two-site or three-site topologies. It is a common management tool for both z/OS and open system environments.
30.1 Introduction eRCMF is a scalable and flexible solution that protects business data and maintains consistent data. It can be used to manage copy relationships for: Planned outages such as hardware and software upgrades, or disaster practice Unplanned outages such as an actual disaster The solution provides basic commands through the FO/FB Session Manager Proxy, which utilizes the ESS Open API functions to accomplish either a planned or unplanned FO/FB sequence.
solution. However, in a real disaster (fire, explosion, or earthquake) you can never expect the components of your complex to fail all at the same moment. Failures will be intermittent and gradual, and the disaster will occur over many seconds or even minutes. This is known as a Rolling Disaster. The architecture of a viable disk mirroring disaster recovery solution must avoid data corruption caused during a Rolling Disaster.
Primary Primary Secondary B C L M X Y B C L M X Y OK 1. Log Update Database Application OK OK 3. Mark Log Complete 2. Database Update Left side disk data does not match right side. Is this the problem? Yes Figure 30-2 Sample Rolling Disaster So, the issue is that the disk subsystem can't do it all and we must avoid this Rolling Disaster system problem. For this reason, it is strongly recommended that, at a minimum, Consistency Groups be put in place for any mirroring solution.
Consistency Group control function Database Application SNMP trap B C L M X Y B C L M X Y 1. Mirroring failure 2. ESS suspends affected primary and holds application I/O (SCSI Queue full condition) 3. ESS sends a notification about the failure 4. A program like eRCMF can receive the SNMP trap and issue the freeze to all LSSs in the Consistency Group 5.
The goal of the TotalStorage Rapid Data Recovery for UNIX and Windows solution, based on the combination of ESS or DS6000 or DS8000 Copy Services with enterprise Remote Copy Management Facility (eRCMF), is to protect your data from being a mirror of a dying scenario. Using Remote Copy Consistency Group, this solution freezes your environment at a known point instead of mirroring literally hundreds of time-offset failures in a short amount of time.
second PCM in the form of logs from the Master Process. These logs are then used to update state information in case the Backup Process must take control of the configuration, as well as for documentation purposes if there is an alternate site failure.
The Master Process continuously monitors and manages the entire environment at two levels: The Enterprise level The VolumeSet level The distinction between these levels is primarily the scope of actions performed at each level. The Enterprise level represents the entire monitored environment, which consists of two or three data processing locations containing storage managed by eRCMF.
Global Mirror – Storage Server periodically forms Consistency Groups. Testing has shown that given enough bandwidth, Storage Servers can form these groups on an average of every 3 to 5 seconds. – When recovering to the recovery copy, software such as eRCMF examines the states of the Global Mirror relationships and directs the Storage Server through the steps necessary to recover that data to the last committed consistent copy.
466 IBM System Storage DS6000 Series: Copy Services with IBM System z
31 Chapter 31. IBM TotalStorage Productivity Center for Replication The IBM TotalStorage Productivity Center for Replication is an automated solution to provide a management front end to Copy Services. This chapter describes IBM’s strategic solution to manage disaster recovery solutions based on copy service functions in DS6000 and DS8000 storage servers. For details refer to the redbook and product manuals listed below.
31.1 IBM TotalStorage Productivity Center The IBM TotalStorage Productivity Center or TPC is a suite of software products. It is designed to support customers in monitoring and managing their storage environments. Design and development emphasis for TPC is on scalability and standards. The approach based on open standards allows TPC to manage any equipment or solution implementation which follow the same open standards.
31.2 Where we are coming from Since the advent of copy services functions with IBM storage servers, a framework is required to handle and manage disk storage environments and the various combination of software, firmware, and hardware used for replication. To ensure and guarantee data consistency at the remote or backup site, it is even more important to provide a framework around copy services functions that helps achieve that data consistency and ease of management.
Note that TPC for Replication does not support Global Copy. Figure 31-2 is a screen capture from the TPC for Replication GUI, showing the different session types, or Copy Services functions, supported. Figure 31-2 Management of Copy Services functions supported by TPC for Replication TPC for Replication is designed to simplify management of Copy Services by: Automating administration and configuration of Copy Services functions with wizard-based session and copy set definitions.
31.4.2 Metro Mirror Metro Mirror (MM) is a synchronous data replication mechanism that was previously called Peer-to-Peer Remote copy or PPRC. I/O completion is signaled when the data is secured on both the local storage server and the remote storage server. Metro Mirror is popular for two site disaster recovery solutions. Metro mirror is available in any combination between DS6000, DS8000, and ESS800.
Local DS6000 / DS8000 Remote DS6000 / DS8000 Suspends due to planned or unplanned event A A P A A S Replication direction Primary Primary Primary Primary Primary Primary Primary Primary Primary Secondary A A A A Primary Primary Primary Primary Primary Primary Primary Primary 1. Failover command P P 2.
31.5 TPC for Replication terminology TPC for Replication manages and integrates not only the DS6000 and DS8000 but also the SAN Volume Controller. In search of a common terminology and to describe the functions for the different disk storage servers in a common way, new terms are introduced here that are different from those normally used in the context of Copy Services with the ESS 800, DS6000, and DS8000. 31.5.
Global Copy A S2 A P2 FlashCopy Primary Primary Primary Primary Primary Secondary Primary Primary Copy Set J2 Tertiary PPRC links Global Copy A P1 Primary Primary Primary A S1 Primary Primary Secondary FlashCopy Primary Primary Copy Set J1 Tertiary Local site Remote Site Figure 31-5 TPC for Replication - Global Mirror Copy Sets 31.5.2 TPC for Replication session TPC for Replication uses a session concept that is similar to what Global Mirror for System z (XRC) uses.
Local DS6000 / DS8000 Session 1 A P3 Primary Primary Primary A P2 Primary Primary Prim ary Remote DS6000 / DS8000 PPRC path PPRC path A P1 A PB Primary Primary PPRC path A PA Copy Set A S1 Primary Primary Primary Copy Set Secondary PPRC path Primary Primary Primary S2 Remote DS6000 / DS8000 Primary Session 2 Copy Set Secondary Local DS6000 / DS8000 Primary Primary S3 Secondary A SB Primary Primary Primary Copy Set Secondary PPRC path Primary Local site A SA Primary Pri
31.6 TPC for Replication session types There are two TPC for Replication editions, which include different levels of Copy Services functions. 31.6.1 TPC for Replication Basic Edition The Basic Edition also includes Metro Mirror and Global Mirror, besides FlashCopy. However, Metro Mirror and Global Mirror are only configured to replicate data in a unidirectional fashion. FlashCopy FlashCopy includes all functional flavors of FlashCopy.
31.7 TPC for Replication session states Again, a session contains a group of Copy Sets (that is, RMC (PPRC) volume pairs or FlashCopy pairs) that belong to a certain application. You may also consider it a collection of volumes that belong to a certain application or system with the requirement for consistency. Such a session may be in one of the following states: Defined A session is defined and may already contain Copy Sets or have no Copy Sets assigned yet. However, a defined session is not yet started.
Terminology is slightly different from what you may be used to. For example, Metro Mirror uses a primary or source volume at the sending site and a secondary or target volume at the receiving end. Such a pair is now called a Copy Set. FlashCopy usually used to mention source and target volume to identify a FlashCopy pair—in contrast to RMC, which designates volumes in that pair as primary and secondary volumes. Again, such a FlashCopy volume pair is now a Copy Set.
31.9 TPC for Replication and scalability From an architecture point of view there is no limit to the number of Copy Sets that may belong to a single session. The implementation approach of managing Copy Sets and sessions provides the scalability that very large installations require.
Figure 31-8 shows how the TPC for Replication server connects to the storage servers which are going to be managed by TPC for Replication server(s). It does not show the connectivity of the TPC for Replication server to the network through which the users connect with their browsers to the server itself.
SMC Replication Manager Server IP network Ethernet ports server0 System p server1 PowerPC PowerPC server0 server1 DS6000 System p DS8000 Figure 31-9 Replication Manager server connectivity to DS6000 and DS8000 The actual connectivity between the TPC for Replication server and the storage servers is based on Ethernet networks and connects to particular Ethernet ports in the System p™ in the DS8000.
You may configure these new ports either through the GUI or the DSCLI. The GUI provides a new panel to configure the required IP addresses. This panel is in the GUI path of the Storage Image. Under Storage Image is a new Configure Network Port panel.
Primary DNS Secondary DNS State Server Speed Type Location 9.64.163.21 9.64.162.21 Online 00 1 Gb/sec Ethernet-Copper U7879.001.DQD04X5-P1-C1-T1 TPC for Replication server connectivity to the DS6000 For the DS6000, the Replication Manager server connects to the network that connects to the PowerPC® servers in the DS6000. For the DS6000 this is the same network that the Storage Management console connects to because the DS6000 controller or server card contains only a single Ethernet port.
Replication Manager Server 2 1 FREEZE 3 LSS LSS LSS LSS LSS LSS LSS LSS LSS LSS LSS LSS Primaries Session Secondaries Figure 31-10 TPC for Replication server freeze The TPC for Replication server listens for incidents from the storage servers and takes action when being notified of a replication error from the concerned storage server. Figure 31-10 implies a replication error in an LSS that belongs to the session.
Replication Manager Server Ethernet ports 1 FCP ports 2 FREEZE LSS LSS LSS PPRC FCP links LSS LSS Primaries LSS Session Secondaries Figure 31-11 LSS heartbeat triggers freeze when connectivity to server fails Figure 31-11 implies a failing connectivity between the RM server and a primary storage server.
Figure 31-12 Windows 2003 software level SUSE Linux Enterprise Server 9 SP2 Note: SUSE Linux does not support the Two-Site BC configuration. Red Hat Enterprise Linux RHEL4 AS 2.1 For Replication Two-Site BC: Red Hat Enterprise Linux RHEL4 Update 1 SLES9 SP2 AIX 5.3 ML3 Note: For a TPC for Replication Two-Site BC configuration that involves two servers it is possible to run TPC for Replication on two different operating system platforms. For the latest software requirements, refer to: http://www.ibm.
You may refer to the following Web site for the latest information on hardware requirements: http://www-03.ibm.com/servers/storage/support/software/tpcrep/installing.html 31.15 TPC for Replication GUI This section describes the graphical user interface (GUI) that is used to communicate with the the Replication server. Additional details can be found in the documents mentioned at the beginning of this chapter.
Login Configure sessions Session States User machine Global Mirror settings Browser / GUI Advanced tools LAN Database Monitor Server Replication Manager Server Hardware Lavers IP network Series p Series p PowerPC DS8000 PowerPC DS6000 Figure 31-14 TPC for Replication GUI The Web-based client that runs on the user’s machine contains the GUI client, a presentation component. This software piece on the user’s machine is highly optimized to minimize data traffic over the LAN to the RM server.
URL of GUI (RM server) user name password Figure 31-15 Launch the Replication Manager GUI You specify a user ID as a text string on the UserID field and a password in a hidden text field. User IDs are defined and set up in the RM server system. 31.15.2 Health Overview panel After a successful login into the Replication Manager server, the Health Overview panel is displayed, as shown in Figure 31-16 on page 490. Chapter 31.
Figure 31-16 Health Overview panel This health panel displays an overall summary of the Replication Manager system status. It actually shows information very similar to what is also shown in the small box at the left lower corner of this panel. This small health overview box at the lower left corner is always present.
Management Servers This link leads you to the application that manages the RM server configuration. Advanced Tools Here you may trigger to collect diagnostic information or set the refresh cycle for the displayed data. Console This link opens a log that contains all activities of the user and its results. 31.15.3 Sessions panel This is the panel of all sessions within the RM server.
(2) Select action (1) Select session Figure 31-18 About to create Copy Set(s) to sessions Figure 31-19 displays a next possible step to perform an action against a session. After selecting the session GM_Session1 shown in Figure 31-17 on page 491, you may select any action from the action list shown in Figure 31-19. Figure 31-19 Actions against a session This may be an action against the entire session, such as suspending all volumes within the session.
Panel heading Hyper links Interface to functional tasks RM summary Figure 31-20 GUI basic layout Using the Add Subsystem button you can define another storage subsystem to the RM server. The panel used to add a new server is shown in Figure 31-22 on page 494. Figure 31-21 displays the available action list. From this list you select, for instance, the View/Modify Details action and apply it to the previously selected storage server.
The selected storage subsystem connectivity details are now displayed, as shown in Figure 31-22. Figure 31-22 Storage subsystem details You usually use the storage subsystem application only to connect a storage server to the RM server. Figure 31-22 also shows the standard port used by the RM server to communicate with the storage server. All the other fields are self-explanatory. 31.15.5 Path Management panel Figure 31-23 displays the entry panel to manage PPRC paths.
Clicking the Manage Paths button triggers the path wizard to help you define a new PPRC path or remove existing PPRC paths. Clicking on a storage subsystem gives you a list of all the existing paths currently defined for the selected storage subsystem. Figure 31-24 displays all the defined PPRC paths for the selected LSS and the ports used for these PPRC paths.
Figure 31-25 Replication Management servers Figure 31-25 shows only one server named Stops, which is an active server. When it exists, a second row will show the potential second server, which is in the status Standby. A panel with two servers has a slightly different appearance than what is shown in Figure 31-25. You use this panel for basic operations such as defining a server as Standby or to take over in the event of a disaster. In the case of two servers, each server manages its own DB2 database.
Figure 31-26 Advanced tools option Specific tasks in this context are tasks to create a diagnostic package or to change the automatic refresh rate of the GUI. A third task is to enable or disable the heartbeat that happens between the Replication Manager server and the connected storage servers; see also 31.12, “TPC for Replication heartbeat” on page 484. The diagnostic package contains all RM logs. Its location on the RM server is shown on the screen.
Figure 31-27 Console log Besides the commands this log also shows whether the command succeeded, and a message number functions at the same time as a hyperlink to more detailed text about the result of the concerned command execution. 31.16 Command Line Interface to TPC for Replication Besides the GUI you may also manage TPC for Replication through a command line interface (CSMCLI).
Example 31-5 Interactive CLI commands ... start csmcli ... csmcli> lsdevice -devtype ds csmcli> lsdevice -devtype ess The third mode is the script mode, running commands out of a file. Example 31-6 Script mode to execute CLI commands ... start csmcli ... csmcli -script ~/rm/scrtips/devreport In contrast to DSCLI for the DS storage servers, the CSMCLI currently does not use a -profile option. Chapter 31.
500 IBM System Storage DS6000 Series: Copy Services with IBM System z
32 Chapter 32. GDPS overview Today’s on demand data centers must be resilient enough to handle planned and unplanned system outages without impacting the business of the enterprise. Most solutions that help provide continuous IT service availability are based on cluster server software, mirrored storage, and redundant networks.
32.1 GDPS solution offerings GDPS is a family of offerings, for single site or multi-site application availability solution, with the capability to manage the remote copy configuration and storage subsystems, automate System z operational tasks, manage and automate planned reconfigurations, and do failure recovery from a single point of control. Software Automation for SAP, DB2,Siebel,MS SQL Server, Exchange etc. 3 Automation for server z/OS, UNIX, Linux, Windows and 2 1 heterogeneous environments.
Tier 7, Tier 6 integrated solutions Site A Automation via GDPS Site B zSeries zSeries Tape VTS High bandwidth connections VTS Primary FlashCopy Peer to Peer VTS FlashCopy Secondary Copy Servcices Figure 32-2 GDPS integrates many components The GDPS family of System z Business Continuity solutions consists of two major offering categories, and each category has several subofferings.
Near transparent D/R solution Recovery Time Objective (RTO) less than an hour Recovery Point Objective (RPO) of zero (optional) Protects against localized area disasters with a typical distance between sites limited to up to 100 km fiber distance The GDPS/PPRC solution offering combines System z Parallel Sysplex capability and ESS, DS6000, and/or DS8000 Metro Mirror disk mirroring technology to provide a Business Continuity solution for IT infrastructures that have System z at the core.
the Freeze function for GDPS/PPRC, FlashCopy automation, or the unplanned outage functions available through the other versions of GDPS. RCMF/PPRC is positioned as a remote copy management control tool, designed to make the task for operators to stop and start remote copy sessions much easier.
User-initiated status collection and exception monitoring NetView application, does not require System Automation for z/OS RCMF/XRC provides panels and code that execute under NetView, and an operator interface for easier management of a remote copy z/OS Global Mirror configuration in setup, initialization, and any planned outage operational mode. This provides benefits for businesses looking to improve their management of XRC for normal running circumstances.
SDM System z System z Application systems 5 Recovery system 6 4 1 TotalStorage TotalStorage TotalStorage 2 3 Primary Metro Mirror Secondary Tertiary Figure 32-3 GDPS 3-site configuration with GDPS/PPRC and GDPS/XRC The same primary volume for PPRC and XRC can be supported by two different GDPSs, a GDPS/PPRC for metropolitan distance and Business Continuity, and a GDPS/XRC for regional distance and Disaster Recovery.
IGS specialists present a number of planned and unplanned GDPS reconfiguration scenarios with recommendations on how GDPS can assist you in achieving your objectives. At the conclusion of the workshop the following items are developed: acceptance criteria for both the test and production phases, a high-level task list, a services list, and project summary.
A Appendix A. Concurrent Copy In this appendix we describe the Concurrent Copy function on the DS6000 series. © Copyright IBM Corp. 2006. All rights reserved.
Concurrent Copy This appendix describes the characteristics and operation of Concurrent Copy. We also discuss the considerations involved when planning to use Concurrent Copy. The information presented in this chapter can be complemented with the following publications: z/OS DFSMS Advanced Copy Services, SC35-0428 z/OS DFSMSdss Storage Administration Reference, SC35-0424 z/OS V1R3.
Session A session is a logical concept that represents a single invocation of Concurrent Copy (a single DFSMSdss DUMP/COPY command). A session can include one or more data sets or volumes on the same DS6000 or across different DS6000s. An individual DS6000 can support up to 16 simultaneous Concurrent Copy sessions per volume, with a maximum of 64 simultaneous sessions per LSS. Session ID The system assigns a unique session ID to each Concurrent Copy session.
Concurrent Copy, application processing is interrupted for only a minimum time while the system initializes the Concurrent Copy session. Once Concurrent Copy is active, your applications continue to process the data, while it is being backed up using Concurrent Copy. Concurrent Copy provides point-in-time data consistency. The system serializes access to the data being dumped or copied just long enough for the Concurrent Copy session to be initialized.
With the target within the same logical subsystem or DS6000 as the source and FlashCopy installed, DFSMSdss will start a FlashCopy copy process instead of Concurrent Copy. So, you get a FlashCopy invocation with a DFSMSdss COPY FULL command even if the CONCurrent (or CC) parameter is coded in the command. The FASTREPLICATION parameter of the COPY command does not affect Concurrent Copy; it only applies for FlashCopy and SnapShot invocation.
Concurrent Copy throughput Your workload flow and your hardware configuration determines how these factors affect your ability to use Concurrent Copy. When planning for the production use of Concurrent Copy, you should also address: When to schedule Concurrent Copy operations Where to use Concurrent Copy Number of simultaneous sessions (z/OS Global Mirror and Concurrent Copy) that you can run Let us look at these considerations in more detail.
Concurrent Copy coexistence with z/OS Global Mirror The DS6000 is not supported as a source for z/OS Global Mirror, and so there should be no concerns with coexistence. Simultaneous Concurrent Copy sessions As said before, each Concurrent Copy session generates additional channel load and increases utilization of the storage paths within the DS6000.
DFSMSdss: Dump and copy examples Example A-3 shows a DFSMSdss full volume dump using Concurrent Copy. No special action is required to perform a restore operation afterwards. Example: A-3 DFSMSdss full volume dump with Concurrent Copy //DSSJOB JOB ... //DUMPSTEP EXEC PGM=ADRDSSU //SYSPRINT DD SYSOUT=* //DASD DD UNIT=SYSDA,VOL=SER=(SSDASD),DISP=OLD //TAPE DD UNIT=TAPE,VOL=SER=(TAPE01,TAPE02,TAPE03),LABEL=(1,SL), // DISP=(NEW,KEEP),DSN=USER.
//SYSIN DD * //* COPY FULL INDYNAM ((CP11S3)) OUTDYNAM ((TP11S3)) COPYVOLID CC ADR101I (R/I)-RI01 (01), TASKID 001 HAS BEEN ASSIGNED TO COMMAND 'COPY ' ADR109I (R/I)-RI01 (01), 2000.107 16:09:59 INITIAL SCAN OF USER CONTROL STATEMENTS COMPLETED. ADR016I (001)-RI01 (01), RACF LOGGING OPTION IN EFFECT FOR THIS TASK ADR006I (001)-STEND(01), 2000.
Installation options exit Usage of the Concurrent Copy function can also be controlled through the installation options exit, a product-sensitive programming interface intended for users. Refer to Options Installation Exit Routine (ADRUIXIT) described in z/OS DFSMS Installation Exits, SC26-7396, for more information.
B Appendix B. SNMP notifications This appendix describes SNMP traps that are sent out in a Remote Copy environment. It repeats some of the SNMP trap information that is available in IBM System Storage DS6000 Series: Architecture and Implementation, SG24-6781. © Copyright IBM Corp. 2006. All rights reserved.
SNMP overview The DS8000 sends out SNMP traps when a state change in a remote copy services environment occurs. Therefore, 13 traps are implemented. The traps 1xx are sent out for a state change of a physical link connection, and the 2xx traps are sent out for state changes in the logical copy services setup. The DS HMC can be set up to send SNMP traps to up to 2 defined IP addresses. eRCMF (see Chapter 30, “IIBM TotalStorage Rapid Data Recovery” on page 457) is listening to SNMP traps for the DS6000.
1: 2: FIBRE 0003 XXXXXX 0003 XXXXXX OK FIBRE 0103 XXXXXX 0002 XXXXXX OK Table B-1 PPRC path reason codes Reason Code Description 00 No path. 01 ESCON path established. 02 Initialization failed. ESCON link reject threshold exceeded when attempting to send ELP or RID frames. 03 Time out. No reason available. 04 No resources available at primary for the logical path establishment. 05 No resources available at secondary for the logical path establishment.
Remote copy events If you have configured Consistency Groups and a volume in this Consistency Group is suspended because of a write error to the secondary device, trap 200, as shown in Example B-4, is sent. One trap is sent for each LSS that is configured with the Consistency Group option. This trap could be handled by automation software like eRCMF to freeze this Consistency Group.
Trap 212, as shown in Example B-8, is sent when a Consistency Group cannot be created in a Global Mirror Copy relation. Some of the reasons could be: Volumes have been taken out of a copy session. The remote copy link bandwidth might not be sufficient. The FC link between the PPRC primary and secondary system are not available.
Trap 217, as shown in Example B-13, is sent if a Global Mirror Copy Environment was suspended by the DS CLI command pausegmir or the corresponding GUI function.
C Appendix C. Licensing In this appendix we describe how the licensing functions for Copy Services for the DS6000 Series are arranged. © Copyright IBM Corp. 2006. All rights reserved.
Licenses All DS6000 Series machines must have an Operating Environment License or OEL for the total storage installed, as calculated in decimal TB. Licenses are also required for use of Copy Services functions. Each function is enabled for a DS6000 system by acquiring licences for specific feature numbers, as listed in Table C-1.
It should be noted that the RMC License provides the functions of Metro Mirror, Global Copy, and Global Mirror, depending on how it is configured during installation. These functions were previously known as PPRC and its variants. Note also that, for the DS6000 and ESS, the previous Extended Remote Copy (XRC) copy service is now known as Remote Mirror and Copy for z/OS, or RMZ. You cannot order an RMZ license for DS6000. Authorized level All Copy Services functions require licensing to be activated.
528 IBM System Storage DS6000 Series: Copy Services with IBM System z
D Appendix D. CLI migration This appendix discusses the ways that you can migrate Copy Services tasks on the ESS environment to the DS Copy Services environment. The Copy Services functions described here cover the GUI and the CLI. Excluded are the Copy Services functions that are executed directly by z/OS. © Copyright IBM Corp. 2006. All rights reserved.
Migrating ESS CLI to DS CLI With the introduction of the IBM DS6000 Storage Unit, a new Copy Services application is also introduced. The Copy Services functions can be issued with the GUI or the DS CLI. The advanced Copy Services functions that are available in the ESS 800 are also available on the DS6000 (except for Global Mirror). Although the functions are still available, there are also some differences that need to be considered when replacing your ESS CLI with a DS CLI.
Figure D-1 ESS Copy Services GUI tasks panel Highlight the task and click the information panel. See Figure D-2. Figure D-2 ESS task information Review also the specific server scripts (depending on the operating system), that perform tasks that set up and execute saved ESS CLI saved tasks. You might have to edit or translate these scripts to run your DS CLI scripts. Convert the individual tasks Choose the ESS CLI tasks that you need to translate to the DS CLI.
Task parameter ESS CLI parameter DS CLI conversion Description TargetServer 2105.23953 N/A used only once in DS CLI Source and Target vols. 1004 1005 1004:1005 separated by a colon in DS CLI DS CLI commands Example D-3 shows the translation to a DS CLI command. Example: D-3 DS CLI mkflash dscli> mkflash -nocp -dev IBM.2105-23953 1004:1005 CMUC00137I mkflash: FlashCopy pair 1004:1005 successfully created. dscli> lsflash -dev IBM.
Table D-2 ESS and DS CLI commands and parameters comparison ESS CLI DS CLI Comments list server lsserver Like the 2105, the 1750 storage facility image contains one pair of servers.
ESS CLI DS CLI Comments list featurecode lsda, lshba, lsioencl, lsuser, mkuser, rmuser, chuser, lsstgencl The 1750 CLI commands can display feature codes when the appropriate parameters are used with the commands. list task NA show task NA list pprcpaths lspprcpath Unlike the 2105, the 1750 CLI Copy Services functions are not task oriented. The 1750 CLI provides a complete set of FlashCopy and PPRC make, change, remove, list, and show commands.
For 2105, a Rank is either assigned to server 0 or server 1, depending on the Array Site location. A 2105 Rank is assigned to one of 32 possible LSS IDs, depending on device adapter pair location and storage type configuration. For 1750, an Extent Pool is assigned to server 0 or server 1. A Rank that is configured from any Array Site can be assigned to a server 0 or 1 Extent Pool. Array Site position and device adapter pairs are not factors for the Rank-to-Extent Pool assignment.
536 IBM System Storage DS6000 Series: Copy Services with IBM System z
Related publications The publications listed in this section are considered particularly suitable for a more detailed discussion of the topics covered in this redbook. IBM Redbooks For information about ordering these publications, see “How to get IBM Redbooks” on page 539. Note that some of the documents referenced here may be available in softcopy only.
Online resources These Web sites are also relevant as further information sources: Documentation for DS6800: http://www.ibm.com/servers/storage/support/disk/ds6800/ SDD and Host Attachment scripts at: http://www.ibm.com/support/ IBM Disk Storage Feature Activation (DSFA) Web site at: http://www.ibm.com/storage/dsfa The PSP information can be found at: http://www-1.ibm.com/servers/resourcelink/svc03100.nsf?OpenDatabase Documentation for the DS6000 at: http://www.ibm.
How to get IBM Redbooks You can search for, view, or download Redbooks, Redpapers, Hints and Tips, draft publications and Additional materials, as well as order hardcopy Redbooks or CD-ROMs, at this Web site: ibm.com/redbooks Help from IBM IBM Support and downloads ibm.com/support IBM Global Services ibm.
540 IBM System Storage DS6000 Series: Copy Services with IBM System z
Index Numerics 1750 DS6000 11 2105 ESS 800 12 A activating 65 activating FlashCopy machine signature 67 model 67 serial number of the DS6000 67 ANTRQST enhanced to manage Global Mirror 327 ANTRQST API 179 architecture Copy Services 7 asynchronous data replication 250 attributes consistent asynchronous remote copy solution 253 synchronous data replication 250 automation and management 132 B B volumes consistent data 289 bandwidth 145 basic concepts 45 basic concepts of Global Mirror 253 C catch-up transit
CSUSPEND 157, 228 D data backup system 44 data consistency 131, 136–137, 412 Consistency Group 136 data migration 412 data mining system 44 Data set FlashCopy 51, 59, 101 define Global Mirror session 258 defining PPRC volume pairs CESTPAIR command 224 deleting PPRC pairs CDELPAIR command 225 deleting PPRC paths CDELPATH command 225 dependent writes 246, 250 DFSMSdss 97 FASTREPLICATION parameter 97 Disaster Recovery practice 402 display out-of-sync tracks 209 distance 146 distance considerations 216 double
logical subsystem 226 ESS 800 interoperability with DS8000 433 establish Metro Mirror pairs 175 establishing Metro Mirror paths 172 establishing PPRC volume pairs CESTPAIR command 224 example failover/failback 296 examples Global Mirror 342 F failback Global Copy 401 Metro Mirror 134 failbackpprc 169, 231 failover B to A 283 Global Copy 401 Metro Mirror 134 failover and failback 134 failover and failback using TSO CESTPAIR with FAILBACK action 190 CESTPAIR with FAILOVER 187 high level failback process 189
one time test system 122 using a target volume to restore its contents back to the source 125 FlashCopy V2 FASTREPLICATION DFSMSdss parameter 97 flow 70 formation Consistency Group 260 freeze PPRC logical subsystem operation 226 freeze command 140 freezepprc 168, 198, 232 full duplex 203 full volume 48 full volume FlashCopy 97 fuzzy copy 511 G GDPS IBM Global Services offerings 507 PPRC and HyperSwap 504 solution offerings 502 three site solution overview 506 GDPS HyperSwap Manager 134 GDPS/PPRC 503 attrib
307 PPRC paths for multiple primary storage servers 304 PPRC paths through TSO commands 303 PPRC paths with ICKDSF 314 session 258 session through ICKDSF 318–319 delete Global Copy pairs with ICKDSF 327 dependent writes 246, 250 Disaster Recovery practice 402 DS CLI example to establish FlashCopy between B and C 300 DS CLI profile files 390 DS CLI to manage volumes in z/OS 313 DS SM for management 328 establish environment through DS SM 373 environment with ICKDSF 313 environment with TSO commands 303 Flash
H hardware requirements 213 I ICKDSF 157, 196 define PPRC paths 314 delete Global Copy pairs 327, 370 delete session 370 establish FlashCopy 317 establish Global Copy pairs 316 establish Global Mirror environment 313 establish Global Mirror session 318–319 example to establish FlashCopy between B and C 301 Metro Mirror job usage 158 planned outage 366 query active Global Mirror session 322 query device information 372 remove all PPRC paths with ICKDSF 327 remove Global Copy paths 372 remove Global Mirror e
channel connection address 152 command overview 152 Consistency Group 142 CQUERY 155 CRECOVER 156 critical attribute 141 critical mode combination 142 CSUSPEND 157 data consistency 131 display Fibre Channel Connection Information Table 159 distance 146 DS CLI 165, 198 DS CLI command examples 165 DS CLI freezepprc command 198 DS CLI supported environments 165 DS SM panel 234 DS SM usage 172 DS SMC GUI 172 establishing pairs 175 establishing paths 172 failback 134 failbackpprc 169 failover 134 failover and fa
multiple primary storage servers 304 remove 406 remove all with ICKDSF 327 PPRC see RMC PPRC volume pairs CDELPAIR command 225 CESTPAIR command 224 PPRCOPY DELPAIR 159 PPRCOPY DELPATH 160 PPRCOPY ESTPAIR 161 PPRCOPY ESTPATH 160 PPRCOPY FREEZE 161 PPRCOPY QUERY 161 PPRCOPY RECOVER 164 PPRCOPY RUN 164 PPRCOPY SUSPEND 164 primary site failure 282 primary storage servers define PPRC paths 304 production backup system 44 PTC 4 Q query a Global Mirror environment 395 query a Global Mirror session 342 querying PP
suspended 203 suspending PPRC pairs CSUSPEND command 228 symmetrical configuration 146 synchronous data replication 246 T terminate 511 terminating FlashCopy pairs 47 terminating PPRC pairs CDELPAIR command 225 terminating PPRC paths CDELPATH command 225 terminology 268, 510 test system 44 multiple setup example 122 topology 273 TSO 151 example 241 Global Mirror recovery 352 TSO commands 186, 224 batch execution for Global Copy 228 cleaning a Global Mirror session 360 command overview 224 create Global Cop
550 IBM System Storage DS6000 Series: Copy Services with IBM System z
IBM System Storage DS6000 Series: Copy Services with IBM System z IBM System Storage DS6000 Series: Copy Services with IBM System z IBM System Storage DS6000 Series: Copy Services with IBM System z IBM System Storage DS6000 Series: Copy Services with IBM System z (1.0” spine) 0.875”<->1.
IBM System Storage DS6000 Series: Copy Services with IBM System z IBM System Storage DS6000 Series: Copy Services with IBM System z
Back cover ® IBM System Storage DS6000 Series: Copy Services with IBM System z Plan, install and configure DS6000 Copy Services with System z This IBM Redbook will help you plan, install, and configure the IBM System Storage DS6000 Copy Services functions in System z environments, and provides the details you need to implement and manage these functions. The book includes hints and tips.
