CentreVu Call Management System Release 3 Version 8 High Availability User Guide 585-210-931 Comcode 108502204 Issue 1 June 2000
Copyright© 2000 Lucent Technologies All Rights Reserved Printed in U.S.A. Notice Every effort was made to ensure that the information in this document was complete and accurate at the time of printing. However, information is subject to change.
CentreVu CMS R3V8 High Availability User Guide iii CentreVu® Call Management System Release 3 Version 8 High Availability Connectivity, Upgrade and Administration Table of Contents Preface . . . . . . . . . . . . . . . . . . . . . . P-1 Overview . . . . . . . . . . . . . . . . . . . . . P-1 Scope . . . . . . Organization . . . Conventions . . . Related Documents If you have a problem Provide information . . . . . . . . . . . . . . . . . . . . . . . . . . .
CentreVu CMS R3V8 High Availability User Guide iv 3: User Scenarios . . . . . . . . . . . . . . . . . . . . . . 3-1 Agent Trace - Modification . . . . . . . . . . . . . 3-1 Base Load Upgrades with High Availability . . . . . . . . 3-1 Call Work Codes . . . . . . . . . . . . . . . . . 3-2 Change Agent Skills . . . . . . . . . . . . . . . . 3-2 Custom Reports . . . . . . . . . . . . . . . . .
CentreVu CMS R3V8 High Availability User Guide v A: Backup & Restore Procedures . . . . . . . . . . . . . . . . . A-1 CMS Backup Strategy. . . . . . . . . . . . . . . . . A-1 Labeling the backup volume . . . . . Backup information format . . . . How to interpret backup information . . . . . . . . . . . . . . . . . . . . . . . . . A-1 A-2 A-2 B: Items excluded from a CMSADM backup . . . . . . . . . . .
CentreVu CMS R3V8 High Availability User Guide vi
Preface CentreVu CMS R3V8 High Availability User Guide Overview P-1 Preface Overview 0 This document is written for customers who purchase the High Availability feature of the CentreVu® Call Management System (CMS). Scope Organization 0 0 This document contains a variety of procedures to help you maintain your CMS High Availability system. It assumes a minimum level of technical knowledge on the part of its readers to complete the procedures.
Preface CentreVu CMS R3V8 High Availability User Guide Overview P-2 Conventions 0 Related Documents 0 If you have a problem 0 The following conventions are used in this document: ● Unless otherwise specified, all information and procedures in this document apply to the Sun Enterprise 3000, Sun Enterprise 3500, and Ultra5 computers. They will be referred to as the “CMS server.” ● Commands you enter from the console are shown in courier font.
Preface CentreVu CMS R3V8 High Availability User Guide Overview P-3 Provide information 0 When you call the Helpline, be sure to identify yourself as a CentreVu® CMS High Availability customer and be prepared to give the following information: ● Your full name, your organization, and a phone number where a Lucent Technologies representative can contact you about the problem ● The installation location (IL) number The IL number is a 10-digit number from a Lucent Technologies database that helps iden
Preface CentreVu CMS R3V8 High Availability User Guide Overview P-4
CentreVu CMS R3V8 High Availability User Guide Introduction 1-1 Overview Chapter 1: Introduction Overview 1 The primary purpose of the CMS High Availability offer is to ensure an uninterrupted data stream between the DEFINITY ECS and the CMS system, which is achieved by connecting two CMS servers at one site to one DEFINITY® system, thereby eliminating the traditional single point of failure between the CMS and the DEFINITY system.
Introduction CentreVu CMS R3V8 High Availability User Guide 1-2 Overview The CMS HA option allows the following server switch-over options: 1. No switch-over If you do not require continuous access to your CMS data, you can elect not to switch-over to the secondary server after the primary server experiences a major failure event.
CentreVu CMS R3V8 High Availability User Guide Introduction 1-3 Overview Dual ACD Links 1 Duplicate hardware is a key component of the High Availability system. The function of the duplicate hardware is to eliminate a single point of failure in order to prevent data loss due to hardware failures. The dual ACD link feature addresses ACD link failures and builds on the increased ACD link reliability provided by TCP/IP. The C-LAN card provides TCP/IP connectivity between the DEFINITY® and the CMS server.
CentreVu CMS R3V8 High Availability User Guide Introduction 1-4 Overview Hardware Platforms CMS HA is supported on the following platform combinations: ● Sun Ultra* 5 - Ultra 5 ● Sun Enterprise† 3000 - Enterprise 3000 ● Sun Enterprise 3500 - Enterprise 3000 ● Sun Enterprise 3500 - Enterprise 3500 1 Note: R3V8 Feature Enhancements 1 ● For HA systems in which Enterprise 3000 and 3500 servers are combined, the 3500 server should be designated as the primary HA server ● for HA systems in whi
CentreVu CMS R3V8 High Availability User Guide Introduction 1-5 Overview b. non-disruptive R3 migration - non-disruptive R3 migration is the ability to perform any of the R3 migrations with data collection turned on during the entire migration process. R3 migration is critical to synchronizing data during the upgrade process (e.g., R3V6 to R3V8 or R3V8.1 to R3V8.2 when database schema changes are required). 3.
Introduction Overview CentreVu CMS R3V8 High Availability User Guide 1-6 3. Power failures: The Primary and Secondary servers should be separately connected to individual Uninterruptible Power Supplies (UPS) on separate protected power circuits. This configuration ensures that both servers will not be simultaneously disabled due to a localized power failure.
Primary and Secondary CMS Servers CentreVu CMS R3V8 High Availability User Guide 2-1 Primary versus Secondary CMS servers Chapter 2: Primary and Secondary CMS Servers Primary versus Secondary CMS servers When the CMS High Availability offer is installed at your location, you designate one CMS server to be the “Primary” and the other as the “Secondary”.
Primary and Secondary CMS Servers CentreVu CMS R3V8 High Availability User Guide Primary versus Secondary CMS servers Caution 2-2 It is strongly recommended that no users are logged into the Secondary CMS server while the Primary CMS server is operational.
Primary and Secondary CMS Servers CentreVu CMS R3V8 High Availability User Guide 2-3 Primary versus Secondary CMS servers CMS Recovery Kit Purpose Recovery kit contents Recovery kit software components 2 2 The recovery kit consists of the backup media and original software that the Lucent Service organization needs to restore service to your system when problems occur. Store this kit in a secure location to minimize the time your system is out of service.
Primary and Secondary CMS Servers CentreVu CMS R3V8 High Availability User Guide 2-4 Primary versus Secondary CMS servers HA Server Platform Required/ Optional Open Database Connectivity (ODBC) software All Optional Visual Vectors server software All Optional Software Component *Informix is a registered trademark of Informix Software, Inc. †Informix is a registered trademark of Informix Software, Inc. ‡Informix is a registered trademark of Informix Software, Inc.
Primary and Secondary CMS Servers CentreVu CMS R3V8 High Availability User Guide Primary versus Secondary CMS servers Admin operations that are autosynchronized by the switch 2-5 Some of the CMS administration changes made on either of the HA servers will be automatically synchronized on the other server via the Definity switch.
CentreVu CMS R3V8 High Availability User Guide Primary and Secondary CMS Servers 2-6 Primary versus Secondary CMS servers Scripting & Timetables ● Create Supervisor scripts (from a supervisor login) ● Scheduling of Time Tables Note on timetable scheduling: The Timetable window includes the following run options: This timetable will run on this or another CMS server < > Run only on this CMS server* < > Run on this or another CMS server* In some cases, running timetables on both servers is not desi
Primary and Secondary CMS Servers CentreVu CMS R3V8 High Availability User Guide 2-7 Primary versus Secondary CMS servers Admin operations synchronized by backups and restores The following CMS administration operations can be synchronized between the two HA servers by backing up the CMS server on which the operation was performed and restoring the backup to the other server.
Primary and Secondary CMS Servers CentreVu CMS R3V8 High Availability User Guide 2-8 Primary versus Secondary CMS servers Operations that require data collection to be turned off 2 ● Shortcuts – additions or modifications to existing ● User Permissions ● ACD Access ● Feature Access ● Main Menu Addition Access ● Split/Skill Access ● Trunk Group Access ● User Data ● Vector Access ● VDN Access The ability of the CMS High Availability offer to back up, restore, and migrate data with data
CentreVu CMS R3V8 High Availability User Guide User Scenarios 3-1 Chapter 3: User Scenarios The following user scenarios refer to the CMS servers as “Primary” and “Secondary”. You should perform your day-to-day administrative functions on the Primary CMS server and use the Secondary CMS server only when the Primary is down. These user scenarios describe how to perform normal CMS tasks in your High Availability configuration so that the CMS servers are kept synchronized.
CentreVu CMS R3V8 High Availability User Guide User Scenarios 3-2 Call Work Codes 3 Call Work Code changes are specific to a CMS server, so any changes made on the Primary CMS server must be duplicated on the Secondary. To update Call Work Code items, do the following: 1. Perform the Call Work Code changes you require on the Primary CMS server. 2. Perform the Call Work Code changes on the Secondary CMS server. Change Agent Skills To change agent skills: 3 1.
CentreVu CMS R3V8 High Availability User Guide User Scenarios 3-3 Designer Reports R3V8 CMS High Availability requires that Designer reports must exist on each CMS server in order to be run on each CMS server. 3 Method 1: 1. Back up CMS System Administration data on the Primary CMS server. 2. Put the Secondary CMS server in single-user mode. 3. Restore CMS System Administration data onto the Secondary CMS server. 4. Put the Secondary CMS server in multi-user mode. Method 2: 1.
CentreVu CMS R3V8 High Availability User Guide User Scenarios 3-4 Dictionary Changes 3 Dictionary changes are specific to a CMS, so that any changes that are made on the Primary CMS server must be duplicated on the Secondary CMS server. Method 1: Synchronizing dictionary changes by back up and restore of ACD-specific administration data Note that this procedure is for dictionary operations made on a single ACD.
User Scenarios CentreVu CMS R3V8 High Availability User Guide 3-5 Method 2: Synchronizing dictionary changes by backup and restore of specific tables: Dictionary synonyms and Dictionary agent groups can also be duplicated using the specific table backup and restore process shown below. The specific table backup and restore process takes less time than using the ACD Specific Administration data backup described above.
CentreVu CMS R3V8 High Availability User Guide User Scenarios 3-6 4. If the Visual Vectors server software is installed on the system, stop and re-start Visual Vectors on the server software in order to activate the new synonym(s) in Visual Vectors. To stop and restart the Visual Vectors software on the server, perform the following steps a. At the command prompt, enter: setupaas b. Select the run_vvs option from the displayed menu. c.
CentreVu CMS R3V8 High Availability User Guide User Scenarios 3-7 Exceptions 3 Exceptions must be administered individually on each HA server. Note: There are three basic types of Exceptions: call-based, intervalbased or CMS execution-based. Call-based and interval-based exceptions are counted at the switch, so the Primary and Secondary servers are automatically synchronized for these excetion types.
CentreVu CMS R3V8 High Availability User Guide User Scenarios 3-8 External Call History - Turning on and off 3 R3V8 CMS High Availability helps reduce the potential loss of ECH data sent to the External Call History server because if the Primary CMS server becomes inactive (e.g., CMS is down), you can start ECH on the Secondary CMS and continue to collect data. If you do not use any customized CMS reporting solutions developed by Lucent PSO, ECH data should be active on only one CMS server at a time.
CentreVu CMS R3V8 High Availability User Guide User Scenarios 3-9 Forecast Data Storage Allocation Administration R3V8 CMS High Availability permits data collection to remain on during forecasting data storage allocation. Method 1: 3 1. Change the Forecast Data Storage Allocation on the Primary CMS server. 2. Change the Forecast Data Storage Allocation on the Secondary CMS server.
CentreVu CMS R3V8 High Availability User Guide User Scenarios 3-10 Main Menu Additions To synchronize Main Menu Additions, do the following: 3 1. Create Main Menu Additions on the Primary CMS server. 2. Create Main Menu Additions on the Secondary CMS server. NOTE: If you attempt to synchronize the Main Menu Additions by backing up from the Primary CMS server and restoring on the Secondary, the Main Menu Additions will appear on the Secondary CMS server but the associated files will not.
CentreVu CMS R3V8 High Availability User Guide User Scenarios 3-11 Split/Skill Call Profile Setup 3 Split/skill call profile changes are specific to each CMS server, so any changes made on the Primary CMS server must be duplicated on the Secondary. Note: Within the interval in which split/skill call profile changes are made, all data from the time of the profile change and extending back to the beginning of that archive interval are lost.
CentreVu CMS R3V8 High Availability User Guide User Scenarios 3-12 Synchronizing the CMS servers after turning Data Collection On/Off Some CMS administrative actions require CMS data collection to be turned off in order to make the required system changes.
User Scenarios CentreVu CMS R3V8 High Availability User Guide 3-13 Synchronizing the CMS servers after turning Data Collection On/Off ✓ 7. Wait until the most recent archive interval has completed. Verify that the interval has been archived on the Secondary CMS server by doing the following: Using Maintenance: Archiving Status, run the report for interval archiving for all ACDs. Verify from the report that the interval archive for the interval ending at time B has run. 8.
CentreVu CMS R3V8 High Availability User Guide User Scenarios 3-14 Synchronizing the CMS servers after turning Data Collection On/Off ✓ 17. At Time C’ (see graphic), perform an incremental backup of all ACDs on the Primary CMS server. Note: If a Daily/Weekly/Monthly archive occurred before you synchronize data at time B’ or time C’, then after you synchronize the data (at time B or C) you must run the appropriate Daily/Weekly/Monthly archive.
User Scenarios CentreVu CMS R3V8 High Availability User Guide 3-15 Synchronizing CMS Servers after Data Collection Is Turned On/Off Secondary CMS server Primary CMS server Time A (Interval A begins) Link up Turn off data collection Link up Link down (perform admin) Turn on data collection Link back up synchronize Time B (Interval B begins) Time B’ Turn off data collection Link down (perform admin) Turn on data collection Link back up synchronize Time C (Interval C begins) Time C’ Time D (Interval
CentreVu CMS R3V8 High Availability User Guide User Scenarios 3-16 Timetables – Running only on the Primary server In most cases, you will want to run a timetable from only the Primary CMS server. To do so, perform the following procedures: 1. Create a timetable on the Primary CMS server. 3 2. Enter the timetable screen on the Primary CMS server.
CentreVu CMS R3V8 High Availability User Guide User Scenarios 3-17 Timetables – Running on both Primary and Secondary servers There may be instances where you want to run a Timetable from both the Primary and Secondary CMS servers. For example, since the Maintenance error log report is specific to a CMS server, you may want the timetable to run and produce a Maintenance error log report for each CMS server.
CentreVu CMS R3V8 High Availability User Guide User Scenarios 3-18 Timetables – Global edits to change server ownership Use this procedure if the Primary CMS server fails and you would like to globally edit timetables to ensure that they will all run on the Secondary server.
User Scenarios CentreVu CMS R3V8 High Availability User Guide 3-19 7. When the Primary server is returned to service, choose between the following options: a.
CentreVu CMS R3V8 High Availability User Guide User Scenarios 3-20 Users - Adding or modifying 3 To administer a new user on the CMS High Availability system, add the new user on the Primary CMS server. Then restore this new data to the Secondary CMS server. 1. Add user(s) via User Data on the Primary CMS server (for details see Assigning User Data, Chapter 9 in: CentreVu® CMS Administration, 585-210-910). 2.
CentreVu CMS R3V8 High Availability User Guide User Scenarios 3-21 Users - Setting User Passwords Use this procedure to administer CMS user passwords on an HA server. The passwords must be administered separately on each server. 3 1. Log in to CMS. The CMS main menu is displayed. 2. At the CMS main menu, press F3 to select the COMMANDS option. The commands options window is displayed 3. Use the cursor keys to select the Unix(r) system option and press Enter. A terminal window is displayed. 4.
CentreVu CMS R3V8 High Availability User Guide User Scenarios 3-22 Also, when ACD-specific Administration data from the Primary server is restored to the Secondary server, data in the archive interval in which the restore is performed will also be lost on the Secondary server.
High Availability Backup & Restore Strategy CentreVu CMS R3V8 High Availability User Guide 4-1 High Availability Backup Strategy Chapter 4: High Availability Backup & Restore Strategy High Availability Backup Strategy With your High Availability configuration you will essentially follow a traditional backup routine for two CMS servers instead of just one.
High Availability Backup & Restore Strategy CentreVu CMS R3V8 High Availability User Guide 4-2 High Availability Backup Strategy Synchronizing after an unscheduled outage of the Secondary CMS server If you encounter an unscheduled outage of the Secondary CMS server, perform the procedures below to resynchronize it with the data in the Primary CMS server. 1. After the Secondary CMS server is back up and running, do a full maintenance backup of the Primary CMS server. 4 2.
Backup & Restore Procedures CentreVu CMS R3V8 High Availability User Guide A-1 CMS Backup Strategy Appendix A: Backup & Restore Procedures Essentially the procedure to back up and restore the CMS servers in a High Availability configuration is the same as it was with just a single CMS server, except that now you are working with two independent servers instead of one. The backup and restore procedures described in this Appendix are identical to those used for non-High Availability configurations.
CentreVu CMS R3V8 High Availability User Guide Backup & Restore Procedures A-2 CMS Backup Strategy You should have the appropriate number of tapes for the backup. When you run a manual backup (not from a timetable), you get an acknowledgment in the Back Up Data window that tells you the number of tapes needed for a full backup. (Incremental backups should fit on 1 tape so no estimate is needed.
Items excluded from a CMSADM backup CentreVu CMS R3V8 High Availability User Guide B-1 Appendix B: Items excluded from a CMSADM backup A CMSADM backup copies all system directories and files, with the exception of the following: ● any swap devices (such as those displayed with "swap -l") ● /proc ● /cdrom ● /n ● /tmp ● /core ● /vol ● /floppy ● /xfn ● /usr/lib/cms/Aname ● /usr/lib/cms/Pname ● /usr/lib/cms/Sname ● /cms/cmstables ● /cms/db/inf/cms.
Items excluded from a CMSADM backup CentreVu CMS R3V8 High Availability User Guide B-2 ● /etc/mnttab ● /etc/initpipe ● /etc/syslog.pid ● /var/spool/lp/temp ● /var/spool/lp/tmp ● /var/spool/lp/requests ● /etc/nologin ● /usr/dbtemp ● /etc/.
Items backed up during a Full Maintenance backup CentreVu CMS R3V8 High Availability User Guide C-1 Appendix C: Items backed up during a Full Maintenance backup Note that a pathname with one or more slashes (“/”) indicates a UNIX file or directory. A pathname with no slashes indicates an INFORMIX table.
Items backed up during a Full Maintenance backup CentreVu CMS R3V8 High Availability User Guide C-2 ● /cms/cow/reports/designer ● /cms/db/journal/shortcut ● /cms/db/journal/timetable ● ttsched ● ttsctasks ● ttsc ACD Administration data: ●aar_agents ● acd_shifts ● acds ● ag_ex_adm ● agroups ● arch_stat ● dbstatus ● f_cdayconf (forecasting) ● f_chpap (forecasting) ● f_chprof (forecasting) ● f_cstap (forecasting) ● f_cstprof (forecasting) ● f_dataarch (forecasting) ● f_spda
Items backed up during a Full Maintenance backup CentreVu CMS R3V8 High Availability User Guide C-3 ● vec_x_adm ● vectors Historical data: ●ag_actv ● agex ● call_rec ● haglog ● linkex ● mctex ● spex ● tgex ● vdnex ● vecex ● d_secs ● dagent ● dcwc ● dsplit ● dtkgrp ● dtrunk ● dvdn ● dvector ● f_cday (forecasting) ● f_cdayrep (forecasting) ● f_dsplit (forecasting) ● f_dtkgrp (forecasting) ● f_ispday (forecasting) ● f_isplit (forecasting) ● f_itkgrp (forecasti
Items backed up during a Full Maintenance backup CentreVu CMS R3V8 High Availability User Guide C-4 ● htrunk ● hvdn ● hvector ● m_secs ● magent ● mcwc ● msplit ● mtkgrp ● mtrunk ● mvdn ● mvector ● w_secs ● wagent ● wcwc ● wsplit ● wtkgrp ● wtrunk ● wvdn ● wvector
Restore Characteristics of Different Data Types CentreVu CMS R3V8 High Availability User Guide D-1 Appendix D: Restore Characteristics of Different Data Types Local System Administration Data: This is data specific to the particular CMS server on which it was administered. This data can only be restored onto the server from which it was copied.
Restore Characteristics of Different Data Types CentreVu CMS R3V8 High Availability User Guide D-2 After the restore range is calculated by the program, any existing data rows in the current table which fall within the calculated restore range are deleted. The restore program then copies in the new data to the table, which replaces all of the previously deleted rows, as well as any new data rows which may have been included in the actual restore range.
What to do if a CMS Server Fails CentreVu CMS R3V8 High Availability User Guide E-1 Appendix E: What to do if a CMS Server Fails Primary CMS server: If one or more links to the Primary CMS server goes down: 1. Log into your Secondary CMS server and verify status of the link(s). 2. If the links are up on the Secondary CMS server, inform your users that they should log off of the Primary and log onto the Secondary. 3.
What to do if a CMS Server Fails CentreVu CMS R3V8 High Availability User Guide E-2 Both CMS servers: If the links to both CMS servers are down: 1. Contact your CMS technical support organization/representative and inform them you are a High Availability configuration and tell them links to both CMS servers are down. If there is an unscheduled outage of both servers: 1. Contact your CMS technical support representative.
Frequently Asked Questions CentreVu CMS R3V8 High Availability User Guide F-1 Appendix F: Frequently Asked Questions What is the purpose of the CMS High Availability offer? The purpose of the CMS High Availability offer is to ensure data availability between the DEFINITY ECS and the CMS system by connecting two CMS servers at one site to one DEFINITY® system, thereby eliminating the traditional single point of failure between the CMS and the DEFINITY system.
Frequently Asked Questions CentreVu CMS R3V8 High Availability User Guide F-2
CMS Base Load Upgrade Procedure for High Availability Systems CentreVu CMS R3V8 High Availability User Guide G-1 Appendix G: CMS Base Load Upgrade Procedure for High Availability Systems When a CMS base load upgrade is performed on High Availability (HA) systems, the upgrade procedure can be performed in a manner that avoids system downtime and synchronizes data between the two HA servers.
CMS Base Load Upgrade Procedure for High Availability Systems CentreVu CMS R3V8 High Availability User Guide G-2 IConceptual depiction of the base load upgrade process for a CMS HA system.
CMS Base Load Upgrade Procedure for High Availability Systems CentreVu CMS R3V8 High Availability User Guide G-3 Base Load Upgrade Procedure 7 Complete steps 1 through 5 not more than 24 hours before the base load upgrade: CMS R3V8 HA Baseload Upgrade Procedure ✓ 1. Verify the existing CMS version and load on the Primary and Secondary CMS servers. To do this, open a terminal window and enter: pkginfo -x cms The output will indicate the CMS version and base load number currently running on the system.
CMS Base Load Upgrade Procedure for High Availability Systems CentreVu CMS R3V8 High Availability User Guide G-4 CMS R3V8 HA Baseload Upgrade Procedure ✓ 13. Install the CMS R3V8.x load on the Secondary CMS server (see “Installing a New CMS Base Load” in Chapter 3, CMS Upgrades and Migrations). 14. Install CMS patches (as applicable) on the Secondary CMS server (see “Installing CMS Patches” in Chapter 3, CMS Upgrades and Migrations). 15.
CMS Base Load Upgrade Procedure for High Availability Systems CentreVu CMS R3V8 High Availability User Guide G-5 CMS R3V8 HA Baseload Upgrade Procedure 29. Complete a full maintenance backup on the Primary CMS server. (Refer to CMS Upgrades & Migrations, p. 3-6). 30. On the Secondary CMS server, restore the historical data from the Primary CMS server using the time at the beginning of the interval during which the interruption occurred (for e.g.
CMS Base Load Upgrade Procedure for High Availability Systems CentreVu CMS R3V8 High Availability User Guide G-6
CentreVu CMS R3V8 High Availability User Guide IN-1 Index A ACD administration data . . . . Call Processing software. dual link . . . . . . . . . link failures . . . . . . . . Agent Groups . . . . . . . . Agent Skills, changing . . . Agent Trace, modifying . . . Automatic Scripts . . . . . . D . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . C-2 . 1-3 . F-1 . 1-5 . 3-6 . 3-2 . 3-1 .
CentreVu CMS R3V8 High Availability User Guide IN-2 P Platforms, supported . Prerequisites . . . . . Primary CMS Server . failure . . . . . . . unscheduled outage Publications Center . . . . . . . . . . . . . . . . . . . . T . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-4, F-1 . . P-1 . . .2-1 . . E-1 . . .4-1 . . P-2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
