Advanced ColdFusion Administration ColdFusion® 5 Macromedia® Incorporated
Copyright Notice © 1999–2001 Macromedia Inc. All rights reserved. This manual, as well as the software described in it, is furnished under license and may be used or copied only in accordance with the terms of such license. The content of this manual is furnished for informational use only, is subject to change without notice, and should not be construed as a commitment by Macromedia, Inc. Macromedia Inc. assumes no responsibility or liability for any errors or inaccuracies that may appear in this book.
Contents About This Book ............................... xiii Intended Audience . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xiv New Features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xiv Developer Resources . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xv About ColdFusion Documentation . . . . . . . . . . . . . . .
iv Contents Connecting to dBASE/FoxPro Databases. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21 Configuring dBASE/FoxPro options (Windows) . . . . . . . . . . . . . . . . . . . . . . 21 Configuring dBASE/FoxPro Driver options (UNIX) . . . . . . . . . . . . . . . . . . . 23 Connecting to Excel Databases . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24 ODBC: Microsoft Excel Driver options . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Contents v Chapter 4 Configuring Basic Security ............. 71 About Basic Security . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72 Installation defaults . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72 Configuring Remote Development Security (RDS) . . . . . . . . . . . . . . . . . . . . . . . . . 73 Securing data sources . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
vi Contents An Example of ColdFusion Studio Security . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Enabling Advanced Security . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Specifying a User Directory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Defining a security context . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Specifying resources to protect . . . . . . . . . . . . . . . . . . . . . . .
Contents vii Error Messages. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Generic error codes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Usage error codes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Runtime error codes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Data error codes . . . . . . . . .
viii Contents Logging Options . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 178 Maintenance Options. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 180 Setting MIME Types . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Syntax restrictions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Contents ix Using the Verity didump Utility . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Viewing the word list with didump . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Viewing the zone list with didump . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Viewing the zone attribute list with didump . . . . . . . . . . . . . . . . . . . . . . . . 206 206 207 208 Using the Verity browse Utility . . . . . . . . . . . . . . . . . . . . .
x Contents Chapter 12 Configuring ColdFusion Clusters ...... Introduction to ClusterCATS Administration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ClusterCATS Server . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ClusterCATS Explorer (Windows only) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ClusterCATS Web Explorer (UNIX only) . . . . . . . . . . . . . . . . . . . . . . . . . . . . ClusterCATS Server Administrator . . . . . . . .
Contents xi Chapter 13 Maintaining Cluster Members ......... 307 Understanding ClusterCATS Server Modes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 308 Changing Active/Passive Settings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 309 Changing active/passive settings in Windows . . . . . . . . . . . . . . . . . . . . . . . 309 Changing active/passive settings in UNIX . . . . . . . . . . . . . . . . . . . . . . . . . .
xii Contents Configuring Load-Balancing Metrics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Overview of metrics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Load types . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Output variables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Troubleshooting the load-balancing metrics .
About This Book Advanced ColdFusion Administration is intended for anyone who needs to configure databases for the ColdFusion server. Contents • Intended Audience................................................................................................... xiv • New Features ............................................................................................................ xiv • Developer Resources...............................................................................................
xiv About This Book Intended Audience Advanced ColdFusion Administration is intended for anyone who needs to perform ColdFusion server management tasks, such as configuring advanced security or managing clustered servers. New Features The following table lists the new features in ColdFusion 5: Benefit Feature Description Breakthrough productivity User-defined functions Create reusable functions to accelerate development.
Developer Resources xv Benefit Feature Description Easy managment Application deployment services Effortlessly and reliably deploy, archive, or restore entire applications using ColdFusion archive files. Enhanced application monitoring Keep track of server performance and availability with customizable alerts and recovery. SNMP support Monitor ColdFusion applications from enterprise management systems. Expanded Linux support Deploy on additional Linux distributions, including SuSE and Cobalt.
xvi About This Book Resource Description URL Installation Support Support for installation-related issues for all Macromedia products www.coldfusion.com/support/ installation/ Professional Education Information about classes, on-site training, and online courses offered by Macromedia www.coldfusion.com/developer/ training.cfm Developer Community www.coldfusion.
Getting Answers xvii Book Description CFML Reference The online-only ColdFusion Reference provides descriptions, syntax, usage, and code examples for all ColdFusion tags, functions, and variables CFML Quick Reference A brief guide that shows the syntax of ColdFusion tags, functions, and variables Viewing online documentation All ColdFusion documentation is available online in HTML and Adobe Acrobat PDF formats.
xviii About This Book Contacting Macromedia Corporate headquarters Macromedia, Inc. 600 Townsend Street San Francisco, CA 94103 Tel: 415.252.2000 Fax: 415.626.0554 Web: www.macromedia.com Technical support Macromedia offers a range of telephone and Web-based support options. Go to http://www.coldfusion.com/ support/ for a complete description of technical support services. You can make postings to the ColdFusion Support Forum (http://forums.coldfusion.com/DevConf/index.cfm) at any time.
Part I Data Sources and Tools This part describes data source management and introduces the ColdFusion Administrator tools. The following chapters are included: Advanced Data Source Management ..................................................3 Administrator Tools.............................................................................
Chapter 1 Advanced Data Source Management This chapter describes how to create and configure ColdFusion data sources for several databases using ODBC, OLE DB, and native drivers. It also describes how to use ColdFusion to create a database file in a cfquery and how to use connection string options. For basic information on data sources and for information on how to connect to SQL Server, Access, and Oracle databases, see Installing and Configuring ColdFusion Server.
4 Chapter 1 Advanced Data Source Management About ColdFusion database drivers ColdFusion uses ODBC, OLE DB, and native database drivers. For detailed information about ODBC drivers, see Installing and Configuring ColdFusion Server. About OLE DB OLE DB is a Microsoft specification for a set of interfaces designed to access data.
About ColdFusion database drivers 5 Installing the OLE DB provider Before you configure an OLE DB data source, you must have installed a recent version of the Microsoft Data Access Components (MDAC). MDAC includes two OLE DB providers—SQLOLEDB and MSDASQL. For Access databases, Microsoft makes available a Jet provider. For SQL Server, Microsoft offers MSDASQL and SQLOLEDB providers. During its installation process, ColdFusion attempts to detect the MDAC version on your computer.
6 Chapter 1 Advanced Data Source Management The following procedure describes how to configure an OLE DB data source to a Microsoft SQL Server database on Windows NT, using SQLOLEDB as the provider. To configure an OLE DB data source: 1 Open the ColdFusion Administrator. 2 Under Data Sources, click OLE DB.
About ColdFusion database drivers 6 7 Enter the following connection information: • If SQLOLEDB is the provider Enter SQLOLEDB as the Provider, specify the Server that hosts the database, and specify the name of the Default Database. Note For the Server field, if the database is a local SQL Server database, enclose the word local in parentheses: (local). • If Microsoft Jet is the provider Enter Microsoft.Jet.versionnumber as the Provider (such as Microsoft.Jet.OLEDB.4.
8 Chapter 1 Advanced Data Source Management 7 Click CF Settings and specify any ColdFusion-specific settings. For example, enter a username and password if required for the data source. Note The omission of required username and password information is a common reason why a data source fails to verify. 8 Click Create to create the new data source. ColdFusion automatically verifies that it can connect to the data source. If ColdFusion cannot verify the data source, the Status displays as Failed.
About ColdFusion database drivers 9 If you are creating a UNIX data source, you might need to set environment variables for your database client library by editing the ColdFusion start script in /coldfusion/bin. For detailed information about editing the ColdFusion start script for your particular database, see the section about your database.
10 Chapter 1 Advanced Data Source Management Using ColdFusion to Create a Data Source (UNIX only) The MERANT ODBC drivers that ship with all UNIX versions of ColdFusion include a FoxPro 2.5/dBASE driver. You can use the FoxPro 2.5/dBASE driver to create a database file in a cfquery with standard SQL syntax even if you do not have an Oracle, Informix, Sybase, or DB2 database. Note See the MERANT DataDirect ODBC Reference for details about SQL statements used for flat-file drivers.
Using ColdFusion to Create a Data Source (UNIX only) Date date,
Descript char(254)) INSERT INTO Beans1 VALUES ( 1, ’Kenya’, ’33’, {ts ’1999-08-01 00:00:00.000000’}, ’Round, rich roast’) INSERT INTO Beans1 VALUES ( 2, ’Sumatra’, ’21’, {ts ’1999-08-01 00:00:00.12 Chapter 1 Advanced Data Source Management Using Connection String Options ColdFusion 5 allows you to specify a connection string for ODBC data sources. You can do this programmatically or in the ColdFusion Administrator. About the connection string You can use the connection string to do the following tasks: • Specify connection attributes that cannot be defined in the odbc.ini settings. • Override odbc.ini settings. • Make ODBC connections dynamically when there is no data source defined in the odbc.
Using Connection String Options 13 Limiting DSN definitions Another use of the connect string feature is to limit data source name (DSN) definitions. For example, if you are connecting to a server that has multiple databases defined, you might not want to define a ColdFusion DSN for each database. Instead, you can now use the connection string to supply the database name for the single DSN that you defined for that server.
14 Chapter 1 Advanced Data Source Management Example The following code is a dynamic connection. There is no data source definition in the odbc.ini settings. SELECT * FROM authors For dynamic connections, the ColdFusion Administrator Maintain Connect default value is enabled.
Connecting to DB2 Databases 15 Connecting to DB2 Databases On Windows and UNIX, ColdFusion lets you access DB2 databases using ODBC and native drivers. Configuring DB2 options (Windows) If you install ColdFusion on a Windows server, you can configure a DB2 database as a ColdFusion data source using ODBC, OLE DB, or a native driver. For information about using OLE DB with ColdFusion data sources, see “About OLE DB” on page 4. Native driver: DB2 Universal Database 5.2/6.
16 Chapter 1 Advanced Data Source Management ODBC: DB2/6000 options (Solaris) The following table describes ColdFusion options for the MERANT IBM DB2/6000 ODBC driver: Option Description Data Source Name A name for your ODBC data source. Description Descriptive information about the data source. Database Name The name of the DB2/6000 database. Cursors Preserve cursors at the end of each transaction.
Connecting to DB2 Databases 17 You perform the following steps: • Set environment variables. • Catalog a TCP/IP node. • Catalog the database. • Test the connection. You should be familiar with DB2 to successfully complete this process.
18 Chapter 1 Advanced Data Source Management you create a database, it is automatically cataloged on the server with the database alias (database_alias) the same as the database name (database_name). The client uses the information in the database directory, along with the information in the node directory, to establish a connection to the remote database. To add an entry to the client’s database node directory: 1 Run the db2 command line utility db2.
Connecting to DB2 Databases 19 Data source settings for the ColdFusion DB2 native driver The data source setting for the native driver must point to the database name and include a valid DB2 login name and password. The catalog procedures described in the previous section make the connection through the DB2 Client Enabler software. DB2 binding and privileges for ODBC (UNIX) Access to DB2 requires that you bind and grant privileges to the MERANT bind files.
20 Chapter 1 Advanced Data Source Management 3 Place the dll file generated in step 2 into the appropriate directory on the server. For example, put the file on a server called DB2SERVER into the C:\sqllib\function\ folder. You could also put it into the C:\sqllib\function\unfenced\ folder. 4 Run a CREATE PROCEDURE statement to register your stored procedure. • • • 5 The CREATE PROCEDURE statement creates a row in the database catalog (syscat.
Connecting to dBASE/FoxPro Databases 21 Connecting to dBASE/FoxPro Databases On Windows and UNIX, ColdFusion lets you access dBASE/FoxPro databases using ODBC drivers. Note Because dBASE and FoxPro databases are configured identically in the ColdFusion Administrator, they are discussed together in this section. For information on connecting to Visual FoxPro databases, see “Connecting to Visual FoxPro Databases” on page 37.
22 Chapter 1 Advanced Data Source Management ODBC: MERANT dBASE/FoxPro Driver options (Windows) The following table describes the ColdFusion ODBC options for MERANT dBASE/ FoxPro on Windows. You set these options when you configure a ColdFusion data source. Option Description Data Source Name A name for your ODBC data source. Description A short description of the data source.
Connecting to dBASE/FoxPro Databases 23 Configuring dBASE/FoxPro Driver options (UNIX) If you install ColdFusion Server on a UNIX server, you can configure dBASE/FoxPro as a ColdFusion data source using the MERANT ODBC driver. The following table describes the ColdFusion ODBC options for dBASE/FoxPro (Solaris). You set these options when you configure a ColdFusion data source. Option Description Data Source Name A name for your ODBC data source. Description A short description of the data source.
24 Chapter 1 Advanced Data Source Management Connecting to Excel Databases On Windows, ColdFusion lets you access Microsoft Excel using ODBC or OLE DB. For information about using OLE DB with ColdFusion data sources, see “About OLE DB” on page 4. ODBC: Microsoft Excel Driver options The following table describes ColdFusion ODBC options for Microsoft Excel data sources. You set these options when you configure a ColdFusion data source.
Connecting to Excel Databases 25 ODBC: MERANT Excel Workbook Driver options The following table describes ColdFusion ODBC options for data sources created with the MERANT Excel Workbook driver: Option Description Data Source Name A name for your data source. Description Descriptive information about the data source. Database Workbook A name that identifies the workbook file containing the Excel database.
26 Chapter 1 Advanced Data Source Management Connecting to Informix Databases On Windows and UNIX, ColdFusion lets you access Informix databases using ODBC and native drivers. ColdFusion 5 supports Informix 7.3 and later, including Informix Dynamic Server. If you install ColdFusion on a Windows server, you can configure an Informix database as a ColdFusion data source using ODBC, OLE DB, or a native driver. For information about using OLE DB with ColdFusion data sources, see “About OLE DB” on page 4.
Connecting to Informix Databases 27 Configuring Informix using the native driver The configuration options for ColdFusion native drivers are the same for Windows NT and UNIX. The following table describes ColdFusion options for the Informix native driver. You set these options when you configure a ColdFusion data source. Option Description Data Source Name A name for your data source. Description Descriptive information about the data source.
28 Chapter 1 Advanced Data Source Management 2 You must uncompress and/or untar this file into a separate subdirectory on your server; for example: /opt/isdk. This is the directory that you point to in the start script as INFORMIXDIR. 3 Run the script installclientsdk to install the client SDK. 4 Before you continue, verify that you can connect to the Informix server from a client other than ColdFusion or with a utility such as iconnect.
Connecting to Informix Databases 29 Editing the $INFORMIXDIR/etc/onconfig file Edit the $INFORMIXDIR/etc/onconfig file so that it contains the following lines: # System Configuration SERVERNUM 0 # Unique id corresponding to an OnLine instance DBSERVERNAME alldev # Name of default database server DBSERVERALIASES alldevtli # List of alternate dbservernames DEADLOCK_TIMEOUT 60 # Max time to wait for lock in distributed env.
30 Chapter 1 Advanced Data Source Management Configuring Informix SETNET32 settings After you install the client software, you must configure your workstation to connect to the Informix databases. The following example assumes that the demo database that ships with Informix is installed on the Informix server and the name of the demo database is “stores7.” Using the Start button in the Windows taskbar, go to Programs/ Informix-CLI 32 and select Informix Setnet 32.
Connecting to Informix Databases 31 Protocol Type: olsoctcp Yield Proc: 1 - None Cursor Behavior: 0 - Close Enable Scrollable Cursors: 0 - Disabled Get DB List From Informix: 1 - Yes Now you have an Informix ODBC data source. You can use this in a ColdFusion application. It is important to note that you must provide a username and password in the ColdFusion cfquery tag.
32 Chapter 1 Advanced Data Source Management Connecting to Sybase Databases On Windows and UNIX, ColdFusion lets you access Sybase databases using ODBC and native drivers. ColdFusion 5 supports Sybase 11 and later. If you install ColdFusion on a Windows server, you can configure a Sybase database as a ColdFusion data source using ODBC, OLE DB, or a native driver. For information about using OLE DB with ColdFusion data sources, see “About OLE DB” on page 4.
Connecting to Sybase Databases 33 Native: Sybase 11 Driver options To connect to Sybase System 11 databases on Windows NT and UNIX, you must first install the Sybase client software, Sybase Open Client version 11.1.0 with Update 11.1.1 applied. To use the native driver: 1 Install the Sybase Open Client version 11.1.0 (with Update 11.1.1 applied) client software. 2 Verify the connection to the database using a tool like Sybase SQL Advantage.
34 Chapter 1 Advanced Data Source Management Note If the Sybase database is on the same server as ColdFusion, make sure the $SYBASE environment variable that you set up in the ColdFusion start script is pointing to the Sybase client directory and not the Sybase server directory. Both of these directories contain an interfaces file. The /opt/coldfusion/bin/start script #!/bin/sh # start - setup environment and run Cold Fusion servers # This script should be run as root.
Connecting to Text Databases 35 Connecting to Text Databases On Windows and UNIX, ColdFusion lets you access text databases using ODBC drivers. ODBC: Microsoft Text Driver options (Windows) The following table describes ColdFusion ODBC options for Microsoft Text data sources. You set these options when you configure a ColdFusion data source. Option Description Data Source Name A name for your ODBC data source. Description Descriptive information about the data source.
36 Chapter 1 Advanced Data Source Management Option Description Table Type Select the default type of text file. ColdFusion supports comma-separated, tab-separated, character-separated, fixed length, and stream table types. The default type is used when creating a new table and opening an undefined table. • Column Names in First Line Select this check box to use the first row of data in the text file as column names.
Connecting to Visual FoxPro Databases 37 Connecting to Visual FoxPro Databases On Windows, ColdFusion lets you access Microsoft Visual FoxPro databases using ODBC or OLE DB. For information about using OLE DB with ColdFusion data sources, see “About OLE DB” on page 4. The following table describes ColdFusion ODBC options for Visual FoxPro data sources. You set these options when you configure a ColdFusion data source. Option Description Data Source Name A name for your ODBC data source.
38 Chapter 1 Advanced Data Source Management
Chapter 2 Administrator Tools The tools provided with ColdFusion Administrator make it easy for you to share Web site files, analyze log files, and monitor Web site performance. This chapter introduces the Administrator Tools included with ColdFusion Server 5 and their benefits. The ColdFusion Administrator online Help provides additional information about how to use these tools. Contents • Accessing the Administrator Tools...........................................................................
40 Chapter 2 Administrator Tools Accessing the Administrator Tools ColdFusion Server 5 includes a series of administrative tools. To access these tools, open the ColdFusion Administrator and click the Tools tab. Tools tab On each page, you can click Help to get additional information about the tool settings. Navigation bar The left navigation bar lists the tools provided with ColdFusion Administrator. Note that some of the tools provided are limited to the ColdFusion Server 5 Enterprise Edition.
Features on the Tools Tab 41 Features on the Tools Tab The Tools tab offers several administrative tools that you can use to help manage Web site activities or the components that make up your Web site. All tools on this tab are organized into one of the following tool groups: Logs and Statistics, System Monitoring, and Archive and Deploy. Each tool group is outlined in the following sections.
42 Chapter 2 Administrator Tools On the Logging Settings page, you can accept the defaults or change them as needed. Each time you make a change, you must apply the change by clicking Submit Change. By default, log files are stored in the CFusion\log directory and all log files are saved using the ColdFusion 5 format. To learn more about the log settings and the differences between the log file formats, click Help on the Logging Settings page.
Features on the Tools Tab 43 Server Reports The Server Reports supplied with ColdFusion Server 5 Enterprise Edition provide instantaneous statistics about the performance of your ColdFusion Server. In addition, some of these reports provide information that you can use to track server configuration changes and view current configuration settings. To access the Server Reports in the ColdFusion Administrator, click Tools > Server Reports. The following table provides a brief overview of each report type.
44 Report Type Chapter 2 Administrator Tools Description Performance Reports • Cache Pops Report This report identifies per second the average number of ColdFusion templates that were ejected from cache and the maximum average number of ColdFusion templates that were ejected from cache.
Features on the Tools Tab Report Type 45 Description Settings Summary Report The Settings Summary Report shows the status of all ColdFusion configuration settings in one view. From this view, you can print the current configuration settings, or edit them directly by clicking the setting name shown in the report. Settings Change Report The Settings Change Report helps you track ColdFusion configuration changes as they occur.
46 Chapter 2 Administrator Tools Web Server Monitoring The Web Server Configuration page in the ColdFusion Administrator enables you to easily determine the operating status of your Web servers and configured monitoring device(s). Use this page to monitor the operating status of each monitoring device, view and manage incoming server traffic, and to place a Web server in maintenance mode for necessary repairs. To access this page in the ColdFusion Administrator, click Tools > Web Servers.
Features on the Tools Tab 47 Server Probes The Server Probes tool in the ColdFusion Administrator enables you to actively test the health and operation of your local Web sites. Specifically, ColdFusion offers two probes for monitoring your Web site environment: • Default probes The default probes let you test the availability of the ColdFusion Server or a specific URL. • Custom probes The custom probes let you specify a test program to run as a probe.
48 Chapter 2 Administrator Tools The tabular form on the Server Probes page identifies the names and status of each probe configured in ColdFusion along with the name of the Web server that the probe is monitoring. The probe management controls let you suspend the operation of a configured probe and/or create, edit, and remove probe configurations. The Server Probe Setup page lets you configure the settings required to set up a default or custom probe in ColdFusion.
Features on the Tools Tab 49 Load Balancing Integration The Load Balancing Integration page in the ColdFusion Administrator lets you configure ColdFusion with the Cisco Local Director. The Cisco Local Director is a network device with a secure, real-time, embedded operating system that intelligently load balances IP traffic across multiple servers. You can configure ColdFusion to provide availability and load information to the Local Director using the Cisco Dynamic Feedback Protocol (DFP).
50 Chapter 2 Administrator Tools The Archive and Deploy tools group in the ColdFusion Administrator includes the following features: Archive Settings, Create Archive, Deploy Archive, and Archive Security. A description of each of these features follows. Archive Settings The Archive Settings page in the ColdFusion Administrator lets you configure various archive system settings that apply to all archive and deploy operations.
Features on the Tools Tab 51 The following table provides a brief description of the features presented on the Archive Settings and Variable Definition page: Feature Description Archive working The archive working directory text box lets you specify the directory directory where all archive and restore temporary files and log files are written. By default the archive temporary files and log files are written to Cfusion\cfam\car\temp directory.
52 Chapter 2 Administrator Tools Create Archive The Create Archive page in ColdFusion Administrator lets you create and edit archive definitions and build archive files. To access the Create Archive page in ColdFusion, click Tools > Create Archive. Help button Controls for defining archive definitions. Build archive control Navigation bar to specify the items to archive. Use the controls on the Create ColdFusion Archive page to add, edit, and view archive definitions.
Features on the Tools Tab 53 All archive definitions are defined and edited using the Archive Definition page. Use the navigation bar on the Archive Definition page to define the items you want to archive and restore. Each time you make a change in the Archive Definition page you must click Apply. You can remove items in the archive definition by clicking Delete. After you create your archive definition, you can click Build Archive on the Create ColdFusion Archive page.
54 Chapter 2 Administrator Tools retrieval method you can click Browse Server to specify the archive file’s location on your system. After you specified the retrieval method and location of the archive file you can then click Next on this page to specify the location to restore the file. To learn more about how to deploy archive files in ColdFusion, click Help on the Archive Deploy page. Archive Security The Archive Security page lets you digitally sign and/or encrypt your ColdFusion archive files.
Features on the Tools Tab 55 Click the names of the settings in the navigation bar to import a security certificate, sign an archive file, verify the signature of an archive file, encrypt an archive file, or decrypt an archive file. Note Certificates are required to digitally sign a ColdFusion archive file or to verify the signature of an archive file. You can obtain a certificate from a Certificate Authority such as VeriSign, Inc.
56 Chapter 2 Administrator Tools
Part II ColdFusion Security This part describes security features and configuration in ColdFusion Server. The following chapters are included: ColdFusion Security ...........................................................................59 Configuring Basic Security .................................................................71 Configuring Advanced Security..........................................................
Chapter 3 ColdFusion Security This chapter introduces ColdFusion Server Basic and Advanced security features that allow you to protect a wide variety of ColdFusion resources. Contents • Why Is ColdFusion Security Important?.................................................................. 60 • Choosing a Level of ColdFusion Security ................................................................ 62 • To Learn More About Security............................................................................
60 Chapter 3 ColdFusion Security Why Is ColdFusion Security Important? Today’s Web applications offer unique opportunities from e-commerce to global communication and collaboration. Today, developers and administrators alike must concern themselves with issues of security. The nature of the Web—global access, ease of connectivity and interaction, and lack of any real control over clients— creates an environment where application misuse or abuse can flourish.
Why Is ColdFusion Security Important? 61 Types of ColdFusion Security ColdFusion Server provides two mutually exclusive security frameworks called Basic security and Advanced security. You can use either type of security to secure ColdFusion application development and deployment.
62 Chapter 3 ColdFusion Security If your Web server connections are encrypted with SSL, all communications, including ColdFusion transmissions, are automatically encrypted. You do not have to do anything from within ColdFusion to activate data encryption. Choosing a Level of ColdFusion Security The rest of this chapter is designed to help you decide which type of ColdFusion security is right for your particular development needs. Basic and Advanced security are mutually exclusive ColdFusion features.
Choosing a Level of ColdFusion Security 63 Basic security covers all phases of application development and deployment. Basic security is a good solution for trusted users because it offers them a single access level—complete control. Consider implementing Basic security if you have legacy systems or other security models in place.
64 Chapter 3 ColdFusion Security Basic security is a good choice to protect ColdFusion resources if your company consists of a single development group or several small groups all physically located at the same site. Because these developers can be considered highly-trusted users, Basic security can still make sense when they are away from the office and are using RDS to develop applications remotely.
Choosing a Level of ColdFusion Security 65 Deploying applications with Basic security Basic security lets you disable execution of CFML tags that could prevent security hazards if they were used in a ColdFusion application, because they could be used to upload, delete, or otherwise manipulate files on the ColdFusion server. ColdFusion displays an error when it encounters a disabled tag in an application.
66 Chapter 3 ColdFusion Security Securing the ColdFusion Administrator The ColdFusion Administrator is a powerful tool that lets you perform administrative tasks like managing server performance, adding and configuring ColdFusion data sources, scheduling pages, and managing log files. You can secure the Administrator with either Basic or Advanced Security. Just as with application development and deployment, the level of security that controls administrative access depends on the level of trust.
To Learn More About Security 67 To Learn More About Security Security at the speed of the Web changes more frequently and over a broader spectrum than can be covered here. Allaire is dedicated to educating its customers about new security information as it becomes available. Visit the Allaire Security Zone (http://www.allaire.com/developer/securityzone/) to read Allaire’s latest security bulletins and technical briefs that provide information about issues Allaire believes are significant.
68 Chapter 3 ColdFusion Security
To Learn More About Security 69
70 Chapter 3 ColdFusion Security
Chapter 4 Configuring Basic Security Basic ColdFusion security allows you to secure a number of ColdFusion Server resources with password access. This chapter describes configuration options for basic ColdFusion security. Contents • About Basic Security ................................................................................................. 72 • Configuring Remote Development Security (RDS) ................................................ 73 • ColdFusion Remote Development Services (RDS)........
72 Chapter 4 Configuring Basic Security About Basic Security ColdFusion Server offers two levels of security: Basic and Advanced. Basic security allows you to impose the following types of control on the ColdFusion development environment: • You can secure the ColdFusion Administrator with a password. Refer to “Securing the ColdFusion Administrator” on page 66 for more information. • You can secure access from ColdFusion Studio to data sources and files with a password.
Configuring Remote Development Security (RDS) 73 Configuring Remote Development Security (RDS) Restricting access to your application page directories is the most important step you can take in making your site secure. You can do this using ColdFusion Basic security. However, you may find it necessary to provide broader access to these directories if, for example, you have several geographically dispersed participants in a development project.
74 Chapter 4 Configuring Basic Security ColdFusion Remote Development Services (RDS) ColdFusion RDS is a component of ColdFusion Server used by the ColdFusion Administrator and ColdFusion Studio to provide remote HTTP-based access to files and databases. You can use RDS to manage ColdFusion Studio access to files and databases on a server hosting ColdFusion. RDS provides both Basic and Advanced security services for ColdFusion, allowing you to configure the level of security you need for your situation.
ColdFusion Remote Development Services (RDS) 75 Securing ColdFusion data sources The following table shows how ColdFusion Basic security can be configured to secure ColdFusion data sources: Method Description Security Model Basic security is Data sources are accessed enabled on the through RDS on the local local workstation. ColdFusion Server. Data sources that are accessible to the user locally are accessible through ColdFusion Studio.
76 Chapter 4 Configuring Basic Security Using a Password to Restrict Access to RDS The Server, Basic Security page of the ColdFusion Administrator is used to configure passwords for securing the Administrator and for preventing unauthorized access to ColdFusion data source and file resources through ColdFusion Studio. Note Password protection is enabled by default at server installation time. If you have not explicitly disabled password access, then security is already configured for your server.
Configuring Basic Runtime Security 77 Configuring Basic Runtime Security Basic security lets you disable execution of seven CFML tags that could present security hazards. You can, however, specify a special directory, called the Unsecured Tags Directory; this is the only directory from which ColdFusion will execute tags you disable with Basic security. Tags you disable with Basic security remain disabled if you switch to Advanced security.
78 Chapter 4 Configuring Basic Security 5 To specify a directory from which otherwise blocked tags can be executed, enter a fully qualified path (using forward slashes) in the Unsecured Tags Directory field. By default, this is the directory in which the ColdFusion Administrator is installed. ColdFusion displays an error message when it encounters a restricted tag in an application. For more information about these tags, see to the CFML Reference.
Chapter 5 Configuring Advanced Security This chapter describes how to set up and configure ColdFusion Server advanced security. Advanced security, which is based on Netegrity SiteMinder v. 4.11, lets you protect a wide variety of ColdFusion resources. Contents • What is Advanced Security?...................................................................................... 80 • Advanced Security Basics .........................................................................................
80 Chapter 5 Configuring Advanced Security What is Advanced Security? ColdFusion Server Professional and Enterprise editions include Advanced security features that provide scalable, granular security for building and deploying your ColdFusion applications: • Application development Control access to files, data sources and administration for each developer on your team. Coordinate team development on shared servers with the assurance that sensitive data and applications are secure.
Advanced Security Basics 81 Advanced Security Basics All types of Advanced Security implement the following four elements: • User directories • Resources • Policies • Security contexts This section introduces these elements and describes how they work together to build your Advanced Security framework. For detailed, hands-on instructions for actually implementing an Advanced Security framework, see “Creating an Advanced Security Framework” on page 88.
82 Chapter 5 Configuring Advanced Security Resource types A ColdFusion resource type that you want to protect is the core of Advanced security. Selecting a resource to protect doesn’t specify how to protect it or which users can access it; you’re simply telling ColdFusion the name and, if applicable, the action of the resource you intend to secure.
Advanced Security Basics 83 Security contexts A security context is a container for logically-related groups of policies. You can create and implement as many security contexts as your application or development environment requires: • You can reuse a single security context, implementing it across several applications. • If you are deploying a more complex application, you may need to create more than one security context for that application alone.
84 Chapter 5 Configuring Advanced Security Advanced Security Implementations The four elements discussed in the previous section—user directories, resources, policies, and security contexts—are the building blocks of every type of security framework you’ll create. You can implement the following types of Advanced Security: • User security Secures functionality in a ColdFusion application.
Advanced Security Implementations 85 Securing resources with RDS security Remote Development Services (RDS) provides a secure connection from ColdFusion Studio to the ColdFusion Server environment and is a prerequisite to accessing data sources, using server-based browsing, and running the interactive debugger.
86 Chapter 5 Configuring Advanced Security accessed or altered by another company’s applications. It also ensures that no applications can tamper with system resources. The access permissions you assign to a directory tree through a security sandbox override any other access permissions users might have for the tree. For example, suppose you designate the directory c:/applications/hr_app as a security sandbox.
Advanced Security Implementations 87 For example, as a ColdFusion Server administrator, you’ll probably want to assign Administrator access to one or two other users, thus ensuring you’ll have backup administrators and your company won’t have to forgo administrative support if you’re away. You might also want to create a class of Privileged access administrators who can manage all aspects of the ColdFusion environment except Basic and Advanced security.
88 Chapter 5 Configuring Advanced Security Creating an Advanced Security Framework No matter which Advanced Security feature you choose to implement—user security, RDS security, a security sandbox, or administrator security—you’ll follow the same basic steps for creating the framework: 1 Set up the security server. See “Setting Up a Security Server” on page 89 for more information. 2 Set up user directories to authenticate against an NT domain, an LDAP directory, or an ODBC data source.
Setting Up a Security Server 89 Setting Up a Security Server The first step to implementing Advanced security is setting up a security server. In a non-clustered environment, the security server is the server hosting ColdFusion, where your ColdFusion programming resources, files, data sources, custom tags, Verity collections and so on, are stored. In a clustered environment, you can define a single security server in the cluster to handle all security authentication and authorization.
90 Chapter 5 Configuring Advanced Security • • ColdFusion Cache Settings The Security Server value is the physical location of the security server. By default, this is the localhost IP# 127.0.0.1. You can supply an IP address or a logical name that can be resolved to a physical address. 4 Enter a Shared Secret, which is part of the encryption key that validates Advanced security transactions.
Caching Advanced Security Information 91 Caching Advanced Security Information Caching Advanced Security information can greatly improve performance within your ColdFusion applications. The ColdFusion Administrator provides the following Advanced security caches: • Security Server Policy Store Cache caches Advanced security information. You can load this cache at startup. By default, it is notified of administrative changes to the policy store once every minute.
92 Chapter 5 Configuring Advanced Security Defining User Directories User and group authentication is carried out against either an existing Windows NT domain, an LDAP directory, or an ODBC data source. When you set up Advanced security, you must specify at least one user directory. You can add as many user directories as you like. Once you define a user directory, it is available for you to use with any security context you define for this security server.
Defining User Directories 93 5 Enter a username and password if the domain, directory, or data source requires one. You can leave these fields blank if ColdFusion Server is running under Administrator access. 6 Select the Secure Connect check box to implement encrypted transmission of authentication information. Secure Connect must be enabled when accessing an LDAP server over Secure Sockets Layer (SSL).
94 Chapter 5 Configuring Advanced Security and point at the SmSampleUsers.mdb file installed in the cfusion\database directory. 2 Use the ColdFusion Administrator Advanced Security page to add a User Directory. Select the ODBC namespace and enter SmSampleUsers in the location form field. See “Defining User Directories” on page 92 for more information. 3 Associate a user or group with a policy in your security context. Example username/passwords are admin/secret and vlander/firewall.
Defining a Security Context 95 Defining a Security Context The Security Context is a logical set of resources grouped together from an administrative perspective. It does not necessarily correspond to a ColdFusion application or resource name. As its name suggests, the security context is used to establish a context in which authentication and authorization actions are carried out. For example, you might create a security context for a particular application development effort.
96 Chapter 5 Configuring Advanced Security Specifying Resources to Protect When you define a security context, you specify the types of resources to protect, for example, files and directories. Now you must specify exactly which resources and which actions to protect. For example, you might limit write access to files at a specific pathname. Once you’ve defined resources, you define a security policy that matches resources to users and groups.
Specifying Resources to Protect 97 You see the Resource View page again, showing the policy you just created. Other available policies appear in a drop-down box at the bottom of the page. 8 Select the check boxes that correspond to the actions you want to protect. Now you can add users to the policy. To add users and groups to a policy: 1 Click the Edit Users button at the bottom of the Resource View page to open the Users page for the current policy. Click the Add/Remove button.
98 Chapter 5 Configuring Advanced Security Implementing ColdFusion RDS Security ColdFusion RDS security provides security services to developers working in ColdFusion Studio. See “Securing resources with RDS security” on page 85 to learn about RDS security concepts. In order to implement RDS security, you must use the ColdFusion Administrator to: 1 Set up the security server. See “Setting Up a Security Server” on page 89 for more information.
Implementing User Security 99 Implementing User Security The user security feature allows ColdFusion developers to authenticate users and match protected resources with authorized users. See “Securing applications with User security” on page 84 to learn about user security concepts. In order to implement user security you must use the ColdFusion Administrator to: 1 Set up the security server. See “Setting Up a Security Server” on page 89 for more information.
100 Chapter 5 Configuring Advanced Security Implementing Server Sandbox Security ColdFusion Server Enterprise edition supports server sandbox security for hosted sites. This security feature, controlled by the ColdFusion administrator of a hosted site, offers runtime security based on directory access at a hosted site. See “Securing applications with a security sandbox” on page 85 to learn about security sandbox concepts.
Implementing Server Sandbox Security • 101 If you chose Security Context in step 7, select an existing security context from the Security Context drop-down. 10 Enter the username and password for the user whose privileges you want applied to the sandbox. This user must be a member of the security context or NT Domain you selected in step 9. 11 Click Apply to register the sandbox.
102 Chapter 5 Configuring Advanced Security Securing the ColdFusion Administrator With ColdFusion Server, you can decentralize administrative responsibility by creating multiple administrators. Overall security is maintained because these additional administrators can control only the resources and policies for which you’ve given them explicit responsibility.
Viewing a Map of your Security Framework 103 Viewing a Map of your Security Framework ColdFusion lets you display and print a map that details all the components of your Advanced security framework. To view a map of your currently defined security framework: 1 Open the ColdFusion Administrator and click the Advanced Security link. You see the Advanced Server Security page. 2 Make sure the Advanced Security check box is selected. 3 Click the Map button at the bottom of the page.
104 Chapter 5 Configuring Advanced Security An Example of ColdFusion Studio Security This example shows you how to limit ColdFusion Studio access to a specific set of files and/or data sources on a remote server based on username/password authentication. For this example, assume you are responsible for two development groups, Mars and Venus. Each group needs separate access rules for source files and data sources its current projects. To provide this access, you will: 1 Enable Advanced Security.
An Example of ColdFusion Studio Security 2 105 Enter the server name or a TCP/IP address for the LDAP option. If you specify an LDAP directory you can fill out the Lookup Start field with uid= and the Lookup End field with ,ou=ou_name,o=org_name. If you leave the Lookup fields blank then the ColdFusion Studio User will have to enter their entire distinguished name rather than just their user name.
106 Chapter 5 Configuring Advanced Security You see the Add Resource dialog. 2 Enter c:\ to protect all files on the C:\ drive and click OK. 3 Repeat steps 1 and 2 to protect the following directories: c:\development c:\development\mars\* c:\development\venus\* Now that you’ve explicitly protected all the directories and sub directories and files of interest, move on to defining policies.
An Example of ColdFusion Studio Security 107 • C_R_FILE • C_W_FILE • C_DEVELOPMENT_R_FILE • C_DEVELOPMENT_W_FILE. Now the MARS policy has access rights to the mars_dsn and all files in the c:\development\mars directory and sub directories. 3 For VENUS we want to add the following rules: • VENUS_DSN • VENUS_R_DIRECTORY • VENUS_W_DIRECTORY • VENUS_R_FILES • VENUS_W_FILES • C_R_FILE • C_W_FILE • C_DEVELOPMENT_R_FILE • C_DEVELOPMENT_W_FILE.
108 Chapter 5 Configuring Advanced Security Enable ColdFusion Studio Security The last step is to actually enable Studio Security in the Administrator so that users trying to access ColdFusion Server resources from Studio will be properly authenticated before access is granted. To enable ColdFusion Studio security: 1 On the Advanced Security page click the “Use ColdFusion Studio Authentication” checkbox 2 Select the RDSService security context in the list box.
Advanced Security Single Sign-On 109 Advanced Security Single Sign-On Single sign-on is the ability to authenticate once, even when two servers are involved. For example, if the Microsoft IIS Web server authenticates a user, a ColdFusion page implementing the IsAuthenticated function would not need to re-authenticate that user. In single sign-on, two or more agents trying to authenticate a user will share the same authentication ticket and avoid challenging the user twice for credentials.
110 Chapter 5 Configuring Advanced Security Undocumented Tags and Functions The ColdFusion Administrator makes use of several tags and functions not currently documented in the CFML Language Reference. In the context of the ColdFusion Administrator, access to the functionality provided by these undocumented tags and functions is restricted to people with administrative privileges.
Undocumented Tags and Functions 111 • CFUSION_SETTINGS_REFRESH() Refreshes some ColdFusion settings not requiring a restart • CFUSION_DBCONNECTIONS_FLUSH() Disconnects all currently connected ColdFusion datasources Administrative Tags In addition to standard CFML tags, the ColdFusion 5 Administrator uses the following undocumented tags: • CFINTERNALDEBUG Used for internal ColdFusion debugging by product development and to PCode templates without executing them (used by the CFML Syntax Checker).
112 Chapter 5 Configuring Advanced Security
Part III Advanced Verity Tools This part describes a number of Verity tools and utilities you can use for configuring the Verity K2 Server search engine, as well as creating, managing, and troubleshooting Verity collections. The following chapters are included: Configuring Verity K2 Server............................................................ 115 Indexing XML Documents ................................................................137 Verity Spider ...................................................
Chapter 6 Configuring Verity K2 Server This section provides information about setting up and configuring the Verity K2 server, which is installed with ColdFusion Server. Contents • Overview .................................................................................................................. 116 • About K2 Server ....................................................................................................... 118 • Starting K2 Server ....................................................
116 Chapter 6 Configuring Verity K2 Server Overview ColdFusion Server 5 includes an OEM restricted version of the Verity K2 Server, which incorporates a highly scalable search server architecture. K2 supports simultaneous indexing of distributed enterprise repositories and handles hundreds of concurrent queries and users. You will see considerable performance improvements when using K2 Server to search Verity collections.
Overview 117 Collections that will be used by K2 Server during a search are required to be registered for use by that K2 Server. This is accomplished by editing the K2 Server k2server.ini file. Note that K2 server must be stopped and restarted before this file is read and the K2 collections are ready to be used.
118 Chapter 6 Configuring Verity K2 Server About K2 Server K2 Server is a high-performance search engine designed to process searches quickly in a high performance, distributed system. The K2 search system has a client/server model. K2 client applications, such as ColdFusion applications, provide users access to document indexes stored in Verity collections.
About K2 Server 119 Note To use the K2 mode, you must edit the server registration file k2server.ini, configure ColdFusion to use K2 Server, and restart the K2 Server executable, k2server.exe. How ColdFusion determines which mode to use ColdFusion determines the Verity Search mode by comparing the collection name specified in the cfsearch tag against the local registry. If the collection name is found, then the normal VDK search will be conducted.
120 Chapter 6 Configuring Verity K2 Server Starting K2 Server The ColdFusion installer places the K2 files into the following directories: • Windows platforms: cfusion\bin • UNIX: opt/coldfusion/verity//bin The K2 Server is started from the command line or from a script in the Unix environment and can be integrated as a service within the Windows NT environment. The server is designed to run with a minimum of intervention.
Starting K2 Server 121 Windows batch file example The Windows batch file installed as cfusion\bin\startk2server.bat looks like this: set K2_MODE=SEARCH k2server -inifile k2server.ini To start K2 Server, open a command window and execute the batch file. Running K2 Server as a Windows service When you use the -ntservice 1 option, K2 Server runs as a Service in Windows. As a service, you can specify startup parameters for K2 Server so that it starts automatically at boot time.
122 Chapter 6 Configuring Verity K2 Server Stopping K2 Server You can run K2 Server either as a Windows service or in a command window, as an ordinary application. Unless you use the -ntService 1 option when starting K2 Server, K2 runs in the command window. Stopping K2 when run as a service To halt K2 Server when it is running as a Windows service, you have two options: • Open the Services Control Panel and stop the K2 Server service.
Stopping K2 Server 123 if [ "$pid" != "" ] ; then kill $pid pidproc $1 if [ "$pid" != "" ] ; then sleep 5 pidproc $1 if [ "$pid" != "" ] ; then kill -9 $pid fi fi fi } # Make sure K2 server goes away killproc k2server exit 0 # give it sometime to die # if it still lives, use -9
124 Chapter 6 Configuring Verity K2 Server Editing the k2server.ini File To enable a collection for searching using K2 Server, you need to first set up the k2server.ini file. On Windows platforms, k2server.ini can be found in: cfusion\bin. On UNIX, k2server.ini can be found in: opt/coldfusion/verity/ /bin. The k2server.ini file consists of a large number of parameters you probably won’t need to change. To get started quickly focus on the following sections in the k2server.
Editing the k2server.ini File 125 k2server.ini file listing Here’s an example of the k2server.ini file for Windows platforms. Line numbers are included for reference. 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 ## This is an example of a K2 Server ini file used with ColdFusion. ## ## This Server section provides keywords that control ## the behavior of the entire server.
126 Chapter 6 Configuring Verity K2 Server 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 ## ## ## ## ## ## ## ## ## ## ## ## ## ## ## Assume there is the collection called "myCollection" created by ColdFusion. The following [coll-0] and [coll-1] collection sections register the collections created by ColdFusion. The "collAlias" entry is the collection alias name which is the collection name used by CFSEARCH CFML tag. (i.e.
k2server.ini Parameter Reference 127 k2server.ini Parameter Reference The K2 Server configuration file k2server.ini is composed of a series of sections. The first section, [Server], provides keywords that control the behavior of the entire server. Each subsequent section, (in the form [Coll-1], [Coll-2], and so forth) controls each collection and search service configured for the server.
128 Chapter 6 Configuring Verity K2 Server Parameter broker(n) Description Brokers to ping on startup. Multiple brokers may be specified. For example: broker(1)=machinea:9900 broker(2)=machineb:9901 maxColSize The maximum width of the fields to return to the results list, in bytes. Default is 2048 bytes. Search thread keywords Keyword Description vdkHome Directory containing Verity resources. vdkSortingFlag A flag indicating whether the Verity engine will sort at the collection level.
k2server.ini Parameter Reference 129 Keyword Description resultCacheTimeout Timeout in milliseconds for the result cache. Timeout occurs after 60 seconds or when the cache overflows based on resultCacheQuota. resultCacheQuota The number of slots per segment for the result cache. The result cache is composed of 16 segments, each of which has a number of slots for caching items in: K2SearchNew, K2SearchRecv, K2DocReadBatch. Timeout occurs after resultCacheQuota value * 16.
130 Chapter 6 Configuring Verity K2 Server Keyword Description knowledgeBase The path name to a knowledgebase map file, which identifies numerous topic sets (indexed topics). The value of knowledgeBase identifies the topic sets (multiple) to make available to clients at start-up for every search service. If not specified, the value of knowledgeBase from the [server] section is used. numThreads The number of concurrent searches for the collection.
Using the rck2 Utility to Search K2 Documents 131 Using the rck2 Utility to Search K2 Documents The rck2 command-line tool allows you to search collections associated with a K2 Server in a K2 Search System.
132 Chapter 6 Configuring Verity K2 Server rck2 Command Description x Set score precision to 8 or 16 bit. By default, 16 bit precision is used. h or ? Display online help for the rck2 command options. Error Messages All K2 Client API functions return an error code, and K2Success is the successful return value. A complete listing of API error codes follows. Generic error codes Error Code No. Description K2Success (0) Operation completed successfully.
Error Messages 133 Error Code No. Description K2Error_ArgTooLarge (-27) Argument too large. K2Error_InvalidSortSpec (-28) Invalid sort specification. K2Error_GatewayNotAvail (-29) Gateway driver not available. K2Error_VersionMismatch (-30) arg or Vdk Object mismatch K2Error_NoInstallDir (-100) Cannot find installation directory. Data error codes Error Code No. Description K2Error_StyleFiles (-31) Invalid style files. K2Error_Permissions (-32) Bad file or directory permission.
134 Chapter 6 Configuring Verity K2 Server Remote Connection error codes Error Code No. Description K2Error_HostNotAvail (-90) Cannot contact remote host. K2Error_NotReEntrant (-91) Not reentrant. K2Error_CallDenied (-92) Call cannot be executed. Error Code No. Description K2Error_BadFile (-140) Corrupt or unreadable file. File Handling error codes K2Error_EmptyFile (-141) Empty file. K2Error_ProtectedFile (-142) Password protected or encrypted.
Error Messages 135 TCP/IP error codes Error Code No. Description K2TcpError_Memory c100 Out of memory. K2TcpError_ConnDrop c200 Connection closed by remote host. K2TcpError_WillBlock c300 Will block on this call. K2TcpError_Call_DNS c600 DNS lookup failed (use IP address). K2TcpError_Call_Send c700 Send failed (maybe connection damaged). K2TcpError_Call_Recv c800 Recv failed (maybe connection damaged). K2TcpError_Call_Ioctl c900 Ioctl failed (Internal error).
136 Chapter 6 Configuring Verity K2 Server
Chapter 7 Indexing XML Documents This chapter provides an overview of the process of configuring Verity for indexing XML files. Contents • Indexing Overview .................................................................................................. 138 • Style Files ................................................................................................................. 139 • Indexing XML Documents......................................................................................
138 Chapter 7 Indexing XML Documents Indexing Overview The addition of Verity K2 to ColdFusion 5 includes the ability to index and search XML documents. To be properly indexed, XML data files must be well-formed XML documents, as specified in the Extensible Markup Language Recommendation http:/ /www.w3.org/TR/REC-xml. Briefly stated, a well-formed XML document contains elements that begin with a start tag and terminate with an end tag.
Style Files 139 Style Files The following style files are required to enable indexing of XML files. Default style files are installed into in the cfusion\verity\common\style directory (Windows) and opt/coldfusion/verity/common/style directory (Linux and UNIX). Style File Description style.uni Invokes the XML filter for indexing XML documents. style.xml Modifies the default behavior of the XML filter. (optional) style.ufl Defines custom fields in XML documents.
140 Chapter 7 Indexing XML Documents ? "ignore" will skip indexing xmltag, yet index contents ? between the beginning and end of this pair of xmltags ?> ?> ?> ?>
Style Files 141 style.xml command syntax Use these commands in the style.xml file to manage how Verity handles individual XML elements. Refer to the style.xml file listing for examples of these commands. Command Description field Indexes the content between the pair of specified XML tags as field values. By default, the field name is the same as the xmltag value, unless otherwise specified by the fieldname attribute.
142 Chapter 7 Indexing XML Documents The following command indexes the content between the start and end tags of the specified xmltag as a field, which is given the same name as xmltag: The following command indexes the content between the start and end tags of the specified xmltag as a field, which is given the name specified in the fieldname attribute: The following command indexes the content between the start and end
Indexing XML Documents 143 Indexing XML Documents To prepare for indexing XML documents: 1 Make sure that the XML filter (flt_xml.dll, flt_xml.sl, flt_xml.so) resides in the bin directory for the installed platform. 2 Make sure that the style.uni contains the directive for invoking the XML filter. 3 If custom fields or zones are required, define them in the style.ufl file. 4 Specify custom fields to be populated in the style.xml file, as appropriate.
144 Chapter 7 Indexing XML Documents
Chapter 8 Verity Spider This chapter contains basic Verity Spider documentation, explaining how to index documents on your Web site. Contents • Overview .................................................................................................................. 146 • Verity Spider Syntax ................................................................................................ 148 • Core Options...................................................................................................
146 Chapter 8 Verity Spider Overview The Verity Spider enables you to index Web-based and file system documents throughout the enterprise. Verity Spider works in conjunction with the Verity KeyView document filtering technology so that more than two hundred of the most popular application document formats can be indexed, including Office2000 and WordPerfect, ASCII text, HTML, SGML, XML and PDF (Adobe Acrobat) documents.
Overview 147 Flow control When indexing Web sites, Verity Spider distributes requests to Web servers in a round-robin manner. This means one URL is fetched from each Web server in turn. With flow control, it is possible that a faster Web site will finish before a slower one. Regardless, the Verity Spider optimizes indexing every Web server. Verity Spider V3.7 adjusts the number of connections per server depending on the download bandwidth.
148 Chapter 8 Verity Spider Verity Spider Syntax The following section shows the syntax for several basic types of Verity Spider indexing tasks. Overview Before you create an indexing task for a new collection, you should make copies of the relevant default style files to ensure that you have a set of template style files in a known, stable state. Keep in mind that running multiple simultaneous Verity Spider jobs on the Information Server host may cause performance problems for searches.
Verity Spider Syntax 149 Using a command file If you want simpler reuse and archiving of your indexing commands, you should take advantage of the abstraction offered by the -cmdfile option. By using an ASCII text file to store a task’s options, you also avoid the pitfall of using special characters in an option’s parameter value. For example, the -processbif option requires the use of "!*" and therefore any task using that option must also use the -cmdfile option.
150 Chapter 8 Verity Spider -refresh Used for updating a collection, specifies that Verity Spider process only those documents which qualify as follows: • They are new documents in the repository, and they qualify for indexing under the criteria. • They exist in the collection and are recorded in the Verity Spider persistent store with a status of done. If Verity Spider determines that these indexed documents have been updated in the repository, then they are retrieved again to be reparsed and reindexed.
Core Options 151 Core Options -cmdfile Specifies that Verity Spider reads command-line syntax from a file in addition to the options passed in the command-line. This option includes the path name to the file containing the command-line syntax. The -cmdfile option circumvents command-line length limits. The syntax for the command-file is: option optional_parameters For better readability, you should put each option and any parameters on a single line.
152 Chapter 8 Verity Spider -jobpath Syntax -jobpath path Specifies the location of the Verity Spider databases and the indexing job-related files and directories. The job-related directories and their contents are: • log All Verity Spider log files. See -loglevel for descriptions of the log files. • bif Bulk insert files. • temp Web pages cached for indexing. You can also specify the temp directory by using the -temp option. • admin Files created by the Information Server Admin Tool.
Processing Options 153 Processing Options -abspath Type: File system only Generates absolute paths for files. Use this option when the document locations are not going to change, but the collection might be moved around. When you index a Web server’s contents through the file system, you should use -prefixmap with -abspath to map the absolute filepaths to URLs. See also -prefixmap. -detectdupfile Type: File system only Details Enables checksum-based detection of duplicates when indexing file systems.
154 Chapter 8 Verity Spider By default, each indexing thread uses as much memory as is available from the system. -maxnumdoc Syntax: -maxnumdoc num_docs Specifies the maximum number of documents to be downloaded or submitted for indexing. The value for num_docs does not necessarily correspond exactly to the number of documents indexed. The following factors affect the actual number. Whether or not the value of num_docs falls within a block of documents dictated by -submitsize.
Processing Options 155 By default, a document checksum is computed based on the CRC-32 algorithm. The checksum combined with the document size is used to determine if the document is a duplicate. See also -followdup. -noindex Specifies that the Verity Spider gathers document locations without indexing them. The document locations are stored in a bulk insert file (BIF), which is then submitted to the collection.
156 Chapter 8 Verity Spider Note You should not run more than one Verity Spider process in persistent mode. As the Verity Spider is a resource intensive process, you should only run it in persistent mode with an interval of less than one day. For time intervals greater than twelve hours, you should use some form of scheduling. Some examples are cron jobs for UNIX, and the AT command for Windows NT Server. -preferred Syntax: -preferred exp_1 [exp_n] ...
Processing Options 157 For example, to map the filepath /usr/pub/docs to http://web/~verity, use the following: vdkvgwkey /usr/pub URL http://web/~verity See also -abspath. -processbif Syntax: -processbif ’command_string !*’ Due to the use of special characters, which represent the bulk insert file (BIF), you must run Verity Spider with a command file using the -cmdfile option. Specifies a command string in which you can call a program or script which operates on BIFs generated by Verity Spider.
158 Chapter 8 Verity Spider -submitsize Syntax: -submitsize num_documents Specifies the number of documents submitted for indexing at one time. The default value is 128. The upper limit is 64,000. Note Although larger values mean more efficient processing by the indexer, smaller values will allow more parallelism on multi-CPU systems. Furthermore, in the event of a halt during indexing, a smaller value means fewer documents will be lost.
Networking Options 159 Networking Options -agentname Syntax: -agentname string Type: Web crawling only. Specifies the value for the agent name field that is part of the HTTP request. Since Web servers can be configured to return different versions of the same page depending on the requesting agent, you can use -agentname to impersonate a browser client. Use double-quotes if the name contains a space.
160 Chapter 8 Verity Spider For example, previous versions of Verity Spider did not support the "Host" header, which is needed for Virtual Host indexing. Also, a "Proxy-authentication" header was needed to pass a username and password to a proxy server. In Verity Spider V3.7, the "Host" header is supported by default, and the -proxyauth option is available for proxy server authentication. Therefore the -header option is maintained only for backwards compatibility and possible future enhancements.
Networking Options 161 On Windows NT, you should include double quotes around the argument to protect the special character ( * ). On UNIX, you should use single quotes. Note that this is only required when you run the indexing job from a command line. Quotes are not necessary within a command file (-cmdfile). Note You must have valid Verity Spider licensing capability to use this option. -proxy Syntax: -proxy proxyhost:port Type: Web crawling only. Specifies host and port for proxy server.
162 Chapter 8 Verity Spider Specifies the time period, in seconds, that the Verity Spider should wait before timing out on a network connection and on accessing data. The data access value is automatically twice the value you specify for the network connection timeout. The default value for the network connection timeout is 30 seconds, and therefore the value for the data access timeout is 60 seconds.
Paths and URLs Options 163 Paths and URLs Options -auth Syntax: -auth path_and_filename Specifies an authorization file to support authentication for secure paths. Note There must be a corresponding "Authfile=" entry in the Information Server configuration file, inetsrch.ini, so that documents can be accessed for viewing. Both -auth and Authfile= must point to the same file. -cgiok Type: Web crawling only. Allows indexing of URLs containing the ? symbol.
164 Chapter 8 Verity Spider -followdup Specifies that Verity Spider follows links within duplicate documents, although only the first instance of any duplicate documents will be indexed. You may find this option useful if you use the same home page on multiple sites. By default, only the first instance of the document is indexed, while subsequent instances are skipped.
Paths and URLs Options 165 -nodocrobo Specifies ROBOT META tag directives are to be ignored. In HTML 3.0 and earlier, robot directives could only be given as the file robots.txt under the root directory of a Web site. In HTML 4.0, every document can have robot directives embedded in the META field. Use this option to ignore them. This option should, of course, be used with discretion. See Also -norobo and http://www.w3c.org/TR/REC-html40/html40.txt.
166 Chapter 8 Verity Spider -pathlen Syntax: -pathlen num_pathsegments Limits indexing to the specified number of path segments in the URL or file system path. The path length is determined as follows: The host name and drive letter are not included. For example, neither www.spider.com:80/ nor C:\ would be included in determining the path length. All elements following the host name are included. The actual filename, if present, is included. For example, /world.
Paths and URLs Options 167 -reparse Type: Web crawling only. Forces parsing of all HTML documents already in the collection. You must specify a starting point with the -start option when you use -reparse. You can use -reparse when you want to include paths and documents which were previously skipped due to exclusion or inclusion criteria. Remember to change the criteria, else there will be little for the Verity Spider to do. This can be easy to overlook when you are using -cmdfile.
168 Chapter 8 Verity Spider Content Options -casesen Details Makes processing case-sensitive by specifying that the spider process separately keys that differ only in case. Use only for indexing UNIX servers. -exclude Syntax: -exclude exp_1 [exp_n] ... Files, paths and URLs matching the specified expression(s) will not be followed. If you use backslashes, you must double them so they are properly escaped.
Content Options 169 On Windows NT, you should include double quotes around the argument to protect the special characters such as (*). On UNIX, you should use single quotes. Note that this is only required when you run the indexing job from a command line. Quotes are not necessary within a command file (-cmdfile). To use regular expressions, also specify the -regexp option. Keep in mind that if your starting points do not contain the specified -include expressions, nothing will be indexed.
170 Chapter 8 Verity Spider Note When specifying an URL, you must use full, absolute paths using the same format as appears in the HTML hyperlink. If the link is relative, you must change it to absolute to use it with -indexclude. See Also -regexp. -indinclude Syntax: -indinclude exp_1 [exp_n] ... Specifies that only those files and paths in URLs which match the expressions be followed and indexed. If you use backslashes, you must double them so they are properly escaped.
Content Options 171 -indmimeexclude Syntax: -indmimeexclude mime_1 [mime_n] ... Specifies that only those MIME types which match the expressions be followed but not indexed. On Windows NT, you should include double quotes around the argument to protect the special characters such as (*). On UNIX, you should use single quotes. Note that this is only required when you run the indexing job from a command line. Quotes are not necessary within a command file (-cmdfile).
172 Chapter 8 Verity Spider -indskip Syntax: -indskip HTML_tag "exp" Type: Web crawling only. Specifies Verity Spider is follow and parse links, but not index, any HTML document which contains the text of exp within the given HTML_tag. For multiple HTML_tag and exp combinations, use multiple instances of the -skip option. You can use wildcard expressions, where the asterisk ( * ) is for text strings and the question mark ( ? ) is for single characters.
Content Options 173 -metafile Syntax: -metafile path_and_filename Type: Web crawling only. Allows you to use a text file to map custom meta tags to valid HTTP header fields. If you use backslashes, you must double them so they are properly escaped. For example: C:\\test\\docs\\path. This means you are able to use your own meta tag, in the document, to replace what is returned by the Web server, or to insert it if nothing is returned.
174 Chapter 8 Verity Spider You cannot use the question mark ( ? ) wildcard, and the -regexp option does not allow you to use regular expressions. Use -indmimeexclude to allow the Verity Spider to follow documents, without indexing them, to gain access to other desirable document types. -mimeinclude Syntax: -mimeinclude mime_1 [mime_n] ... Specifies MIME types to be included. On Windows NT, you should include double quotes around the argument to protect the special characters such as (*).
Content Options 175 If you use backslashes, you must double them so they are properly escaped. For example: C:\\test\\docs\\path To use regular expressions, also specify the -regexp option.
176 Chapter 8 Verity Spider Locale Options -charmap Syntax: -charmap name Specifies the character map to use. Valid values are 8859 or 850. The default value is 8859. -common Specifies path to the Verity home directory, verity/prdname/common, where verity/ prdname is the user-definable portion of the installation directory. Note This option is typically not needed, as long as the PATH environment variable is set correctly. -datefmt Syntax: -datefmt format Specifies the Verity import date format to use.
Locale Options 177 Where verity/prdname is the user-definable portion of the installation directory, and platform represents the platform directory.
178 Chapter 8 Verity Spider Logging Options -loglevel Syntax: -loglevel [nostdout] argument Specifies the types of messages to log. By default, messages are written to standard output and to various log files in the subdirectory named /log beneath the Verity Spider job directory. If you add nostdout to the loglevel argument, messages will not be written to standard output. Log files, however, will still be created.
Logging Options 179 Choose one of the following arguments to determine which message types are logged. Loglevel Arguments Description summary Includes the following message types: information, warning, error, badkey, progress, summary Use this option only if you do not want skip type messages. skip Includes the following message types: information, warning, error, badkey, progress, skip Use this option only if you do not want summary type messages.
180 Chapter 8 Verity Spider Maintenance Options -nooptimize Prevents the Verity Spider from optimizing the collection, thus reducing processing overhead during the indexing job. Use this option sparingly, as it leaves the collection in less than optimum shape. Some examples of when you might want to use this option are: • You want to manually perform custom optimization of the collection, using mkvdk. By default the Verity Spider optimization mimics the mkvdk actions of maxmerge and vdbopt.
Setting MIME Types 181 Setting MIME Types You can use the MIME type criteria options -mimeinclude, -indmimeinclude, -mimeexclude and -indmimeexclude to include or exclude MIME types. Syntax restrictions When you specify MIME type criteria, keep in mind the following restrictions. Using the wildcard character (*) The asterisk (*) wildcard character does not operate as a regular expression for the value of the MIME type criteria.
182 Chapter 8 Verity Spider When you encounter MIME Types being dropped, make sure the Web server you are indexing has the necessary MIME Type information. See the documentation for your Web server for information about specifying MIME Types. You can examine the indexing job’s log files for indications that files are being skipped due to MIME Types. For example, a typical ASCII file you might want indexed is a log file (filename.log). Unless the Web server understands that files with .
Setting MIME Types 183 Furthermore, you should also use inclusion and exclusion criteria to finely control what is indexed. • If your list of file types to index is rather long, use one of the exclusion criteria: (-exclude, -indexclude, -mimeexclude, or -indmimeexclude) to exclude extensions you know you do not want to index. For example: -exclude ’*.exe’ ’*.
184 Chapter 8 Verity Spider
Chapter 9 Managing Verity Collections with the mkvdk Utility mkvdk is a command-line utility installed with ColdFusion that you can use to perform maintenance operations on Verity collections, which are the primary data type for building searching/indexing functionality into your ColdFusion application pages. Contents • Overview of the Verity mkvdk Utility ..................................................................... 186 • Getting Started with the Verity mkvdk Utility ............................
186 Chapter 9 Managing Verity Collections with the mkvdk Utility Overview of the Verity mkvdk Utility The mkvdk utility is an indexing application, provided with other Verity utilities, that can be used in various ways to create and maintain collections. It is a command line utility that can be used within other applications or shell scripts to provide more sophisticated scheduling and other capabilities.
Getting Started with the Verity mkvdk Utility Type 187 Number Status 8 Info 16 Verbose 32 Debug 64 To calculate the numeric parameter, add up the numbers for the message types you want to include. The default for both -outlevel and -loglevel is 15, which selects fatal, error, warning, and status messages (1+2+4+8). Getting Started with the Verity mkvdk Utility The basic mkvdk syntax is as follows: mkvdk -collection path [option] [...] [filespec] [...
188 Chapter 9 Managing Verity Collections with the mkvdk Utility Alternatively, you can set up a collection and insert documents in one mkvdk command, using this syntax: mkvdk -create -collection collectionname -bulk -insert filespec Note The -create option can be used only once to create the collection directory structure. After a collection directory structure has been created, do not to use the -create option to update the collection.
Getting Started with the Verity mkvdk Utility 189 Building the word list The following command builds the word list in the collection residing in the path directory. mkvdk -words -collection path General processing options mkvdk provides a variety of general processing options, described in the following table: Option Description -collection path This option specifies the path of the collection to create or open. This is required to execute mkvdk. -nolock This option turns off file locking.
190 Chapter 9 Managing Verity Collections with the mkvdk Utility Option Description -nooptimize This option prevents optimization by this instance of mkvdk. Using this option turns off the service level VdkServiceType_Optimize. The service types determine what type of work the Verity engine and its self-administration features will execute on a collection. -nohousekeep This option prevents housekeeping by this instance of mkvdk. Housekeeping includes deleting files that are no longer needed.
Getting Started with the Verity mkvdk Utility 191 The following command performs servicing only. Use this command if you only want to index submitted documents and service the collection. mkvdk -collection path Deleting documents from a collection The following command deletes documents from a collection.
192 Chapter 9 Managing Verity Collections with the mkvdk Utility Keyword Description optimize Enable opportunistic collection optimization assist Enable building of word list housekeep Enable housekeeping of unneeded files delete Enable document deletion (see Chapter 3) backup Enable backup purge Enable background purging repair Enable collection repair dataprep Same as search-index-optimize-assist-housekeep index Same as insert-delete Messaging options mkvdk provides a variety of mess
Getting Started with the Verity mkvdk Utility Type 193 Number Info 16 Verbose 32 Debug 64 Document processing options mkvdk provides a variety of document processing options, described in the following table: Option Description -extract This option extracts field values from documents, using the field extraction rules specified in the style.tde file. For more information, refer to Chapter 9. -insert This option adds documents to the collection. This is the default option for mkvdk.
194 Chapter 9 Managing Verity Collections with the mkvdk Utility Bulk Submit Options mkvdk provides a variety of bulk submit options, described below. An overview to using the feature is described earlier under “Using Bulk Insert and Delete.” For complete information about using bulk submit to insert, update, and delete documents, see Chapter 3. Option Description -bulk This option tells mkvdk to interpret filespec as a bulk submit file. The option can be used with -insert, -update, and -delete.
Collection Maintenance Options 195 Collection Maintenance Options mkvdk provides a variety of collection maintenance options, described in the following table: Option Description -backup dir This option backs up the collection into the specified directory. Note that the backup will not include the tde subdirectory. The tde subdirectory is created by and for Topic Document Entry if Topic Document Entry is used to create or maintain the collection.
196 Chapter 9 Managing Verity Collections with the mkvdk Utility Deleting a collection To delete a collection, use the appropriate command for your operating system. For example, to remove the collection directory structure and control files on a UNIX system, use the following command. rm -r -collection_path Purging a collection The following command deletes all documents from a collection, but does not delete the collection itself.
Collection Maintenance Options 197 Keyword Description spanword This keyword creates a spanning word list across all the collection’s partitions. A collection consists of numerous smaller units called partitions each of which includes a word list. Optionally, a spanning word list can be built with an ngram index. ngramindex This keyword builds an ngram index for the collection. An ngram index is designed to improve the search performance for queries with the and/or operators.
198 Chapter 9 Managing Verity Collections with the mkvdk Utility About optimized Verity databases The Verity Database (VDB) is the fundamental storage mechanism responsible for supporting dynamic access to documents in collections. A VDB consists of simple tables with rows and columns that relate to each other by row position. VDB tables are not relational, and their architecture supports quick and efficient searching over textual data. A VDB consists of segments which are packed into a single file.
Chapter 10 Verity Troubleshooting Utilities This chapter provides information about using a variety of Verity utilities for troubleshooting Verity collections. Contents • Overview of Verity Utilities ..................................................................................... 200 • Using the Verity rcvdk Utility.................................................................................. 201 • Attaching to a Collection Using rcvdk ............................................................
200 Chapter 10 Verity Troubleshooting Utilities Overview of Verity Utilities The following command line utilities are included with ColdFusion for performing a variety of operations on Verity collections: • rcvdk Searching collections and displaying documents. See “Using the Verity rcvdk Utility” on page 201. • didump View collection word lists. See “Using the Verity didump Utility” on page 206. • browse Browse documents table and search results. See “Using the Verity browse Utility” on page 209.
Using the Verity rcvdk Utility 201 Using the Verity rcvdk Utility Using rcvdk, you can check the contents of a collection from the command line. rcvdk allows you to write a variety of queries, using words and phrases separated by commas and/or Verity query language. A viewing option allows you to see document contents and highlights in a simple text display.
202 Chapter 10 Verity Troubleshooting Utilities Attaching to a Collection Using rcvdk To search a collection, you first must attach to it using the a command. This command must include the path name to a collection directory as an argument. After you press return, rcvdk reports whether the attach command was successful. RC>a /z/doc1/c/public/Collection/file_walking/collbldg/html Attaching to collection: /z/doc1/c/public/Collection/file_walking/collbldg/html Successfully attached to 1 collection.
Viewing Results of the rcvdk Utility 203 Viewing Results of the rcvdk Utility After you have attached to a collection and issued a search command successfully, you can view the results list and look at the retrieved documents. You can use the options in the following table: Option Description r Displays the results list, starting with the first document. A maximum of 24 documents will be displayed. r n Displays the results list, starting with the nth document.
204 Chapter 10 Verity Troubleshooting Utilities The following table describes each of the default fields: Field Name Description Number The rank of the document in the results list. The document with the highest score is ranked number 1. Score The score assigned to each retrieved document, based on its relevance to the query. For a NULL query, no scores are assigned, so the Score column in the results list is blank. VdkVgwKey The document key used by the Verity engine to manage the document.
Viewing Results of the rcvdk Utility 205 9: Document Filters and Formatting 10: Collection Style Summary 11: Collection Basics 12: Universal Filter Document Types 13: Using the style.dft File 14: Supported Field Types 15: 16: Recognized Document Types 17: Custom Zone Definitions 18: The KeyView Filter Kit RC> How to display multiple fields Multiple fields can be specified with the fields command, as shown below.
206 Chapter 10 Verity Troubleshooting Utilities Using the Verity didump Utility Using the didump utility, you can view key components of the word index per partition. The word list consists of a list of all words indexed by the Verity engine. The zone list is a list of all zones found by the engine. The zone attribute list is a list of the zone attributes found by the engine.
Using the Verity didump Utility 207 To view the occurrences of a specific word or pattern, enter a command using the -pattern option, as in the following example: didump -pattern acronym 00000003.did The didump utility will display information about the number of occurrences of the word “acronym.” You can display the individual occurrences of a word using the verbose (-verbose) option. Viewing the zone list with didump The zone list contains a list of the zones identified by the zone filter.
208 Chapter 10 Verity Troubleshooting Utilities Viewing the zone attribute list with didump The zone attribute list contains a list of the HTML attributes for the zones identified by the HTML zone filter. The zone attributes listed can be searched using the Verity IN operator together with the WHEN operator in a query. To view the contents of the zone attributes list, use didump with the -attributes flag plus the path name to a partition, like this: didump -attributes /z/collbldg/html/parts/00000003.
Using the Verity browse Utility 209 Using the Verity browse Utility A documents table is built for each partition in a collection. The documents table is used for field searching and for sorting search results. The fields within the documents table are defined by the following collection style files: • style.ddd defines fields used internally by the Verity engine, identified by an initial underscore character (_) • style.
210 Chapter 10 Verity Troubleshooting Utilities Displaying fields There are several options that can be used to control the display of field information.
Using the Verity merge Utility 211 Using the Verity merge Utility The merge utility lets you combine multiple collections with identical schemas. This is useful for merging smaller collections built from different sources into one, large collection. Also, you can use the merge utility to break up the collection into smaller collections of a roughly uniform size. Note The Verity merge utility is available only on Windows platforms.
212 Chapter 10 Verity Troubleshooting Utilities The utility reads srcCollection and splits it in roughly equal-sized pieces, using the file names given for newCollection1 and so on.
Verity VDK Error Messages 213 Verity VDK Error Messages All Verity Developer’s Kit API functions return an error code, and VdkSuccess is the successful return value. A complete listing of API error codes follows. Generic error codes Error Code No. Description VdkSuccess (0) Operation completed successfully. VdkFail (-2) A general failure not covered by another API error code. VdkWarn (1) A general warning. No.
214 Chapter 10 Verity Troubleshooting Utilities Error Code No. Description VdkError_InvalidSortSpec (-28) Invalid sort specification. VdkError_GatewayNotAvail (-29) Gateway driver not available. VdkError_VersionMismatch (-30) Argument or object mismatch. VdkError_NoInstallDir (-100) Cannot find installation directory. Error Code No. Description VdkError_StyleFiles (-31) Invalid style files. VdkError_Permissions (-32) Bad file or directory permission.
Verity VDK Error Messages 215 Licensing error codes Error Code No. Description VdkError_Signature (-50) Invalid/missing signature. VdkError_LicenseFile (-51) Invalid license file. VdkError_LicenseColl (-52) Too many collections open. VdkError_LicenseVolume (-53) Too many documents in collection. VdkError_LicenseAdvQuery (-54) No advanced query capability. VdkError_LicenseHetero (-56) No heterogeneous collections. VdkError_LicenseDataPrep (-57) Not licensed to index documents.
216 Chapter 10 Verity Troubleshooting Utilities Error Code No. Description VdkError_Scoreop (-129) No support for Score operators. VdkError_Opmod (-130) No support for query language modifiers. VdkError_LicenseSession (-131) Too many top-level sessions. Error Code No. Description VdkError_InvalidUser (-80) Invalid user/password combination. Security error codes Remote connection error codes Error Code No. VdkError_HostNotAvail (-90) Description Cannot contact remote host.
Verity VDK Error Messages 217 Warnings Error Code No. Description VdkWarning_CollectionDown (10) The collection was down when it was opened. VdkWarning_QueryComplex (11) Too many matching words. VdkWarning_LowMemory (12) Memory is low for indexing. VdkWarning_CollectionReadOnly (13) The collection is read-only. VdkWarning_DriverNotFound (14) Couldn’t locate specified driver. VdkWarning_LargeToken (15) Returned a token greater than maxSize.
218 Chapter 10 Verity Troubleshooting Utilities
Part IV ColdFusion High-Availabilty This part explains the high-availability server clustering technology, known as ClusterCATS, that is available with ColdFusion Server. The following chapters are included: Scalability and Availability Overview ................................................221 Configuring ColdFusion Clusters .....................................................245 Maintaining Cluster Members ..........................................................307 ClusterCATS Utilities .............
Chapter 11 Scalability and Availability Overview This chapter describes the concepts involved in achieving scalable and highly available Web applications. Contents • What is Scalability?.................................................................................................. 222 • Issues Affecting Successful Scalability Implementations .................................... 225 • What is Web Site Availability? .................................................................................
222 Chapter 11 Scalability and Availability Overview What is Scalability? As an administrator, it’s likely that you often hear about the importance of having Web servers that scale well, but what exactly is scalability? Simply, scalability is a Web server’s ability to maintain a site’s availability, reliability, and performance as the amount of simultaneous Web traffic, or load, hitting the Web server increases.
What is Scalability? 223 Linear scalability Perfect scalability—excluding cache initializations—is linear. Linear scalability, relative to load, means that with fixed resources, performance decreases at a constant rate relative to load increases. Linear scalability, relative to resources, means that with a constant load, performance improves at a constant rate relative to additional resources. Caching and resource management overhead affect an application server’s ability to approach linear scalability.
224 Chapter 11 Scalability and Availability Overview Load management Load management refers to the method by which simultaneous user requests are distributed and balanced among multiple servers (Web, ColdFusion, DBMS, file, and search servers). Effectively balancing load across your servers ensures that they do not become overloaded and eventually unavailable.
Issues Affecting Successful Scalability Implementations 225 Issues Affecting Successful Scalability Implementations Achieving scalable Web servers is not a trivial task. There are various solutions to pick from, setup and configuration tasks to understand and perform, and many delicate dependencies between related but heterogeneous technologies. This section describes some of the major issues affecting successful scalability implementations.
226 Chapter 11 Scalability and Availability Overview Another approach to solving the same problem is to store client variables in a back-end common state repository. This approach enables all Web servers comprising the cluster to access variables in a common, shared back-end data store, such as a database. However, you must be aware that this approach can potentially impact your site’s performance.
Issues Affecting Successful Scalability Implementations 227 In this scenario, if the application uses an appropriate database concurrency validation mechanism, then the HR Director would receive a message informing her that she could not access the employee record because it was in use, thereby alerting her that the HR Generalist is trying to change the record.
228 Chapter 11 Scalability and Availability Overview • Databases Database access, while vitally important to your application’s capabilities and feature set, can be costly in terms of performance and scalability if it is not engineered efficiently. When creating data sources for accessing your database, use a native database driver rather than an ODBC driver if possible because it will provide faster access.
Issues Affecting Successful Scalability Implementations 229 • Translate the natural language names to server IP address mappings so that users can find the site. • If you have enabled round-robin distribution for multi-server load balancing, it can distribute the load among each server in a rote, sequential distribution manner.
230 Chapter 11 Scalability and Availability Overview The following figure illustrates these concepts: com edu gov ... Allaire dev ftp ... allaire.com Zone ntserver allaire.com Domain dev.allaire.com Zone DNS servers store information about the domain name space and are referred to as name servers. Name servers typically have one or more zones for which they are responsible. The name server has authority for those zones and is aware of all the other DNS name servers that are in the same domain.
Issues Affecting Successful Scalability Implementations 231 On the Windows platform, you make DNS entries using the Domain Name Service Manager utility. On UNIX platforms, you make these DNS entries in the name.db file, which is read by the DNS server’s Berkeley Internet Name Daemon (BIND).
232 Chapter 11 Scalability and Availability Overview How to load test your Web applications One of the first things you need to do to be able to load test is purchase a load testing software tool and learn how to use it. There are a variety of good load testing software tools on the market, including Segue’s SilkPerformer, Mercury Interactive’s LoadRunner and RSW’s e-LOAD.
Issues Affecting Successful Scalability Implementations 233 • Minimize distributed environment load testing Load testing in a distributed environment can be problematic if the network on which you are performing your load tests becomes congested, resulting in poor response times.
234 Chapter 11 Scalability and Availability Overview What is Web Site Availability? As you’ve already learned from the previous section, it’s critical to design, develop, test, and deploy your Web applications so that they can scale well under heavy and ever-increasing load. However, the reality is that in spite of the best-laid plans and preparations, servers can fail for seemingly unknown reasons, causing your site to become unavailable.
What is Web Site Availability? 235 For ColdFusion Web applications, it is particularly important that the ColdFusion servers remain as highly available and responsive as the Web server and other dependent servers. ColdFusion processes requests that are sent to it from the Web server. Upon successfully processing the application logic, ColdFusion returns the results back to the Web server, which in turn returns an HTML response back to the browser.
236 Chapter 11 Scalability and Availability Overview submit or retrieve information from your database. Or, a mail server can go down, making it impossible for your users to successfully send mail to you. Ensure that your organization’s IT architecture includes network monitoring and notification software that can quickly report on the general health of your network and alert you about any failed servers.
What is Web Site Availability? 237 Failover considerations The ability to fail over servers that have become unavailable to redundant servers is a cornerstone of any mission-critical application, one that ensures an application’s continuous and reliable operation.
238 Chapter 11 Scalability and Availability Overview If you plan to use a parallel model, Allaire recommends that you use many middle range servers rather than fewer high-end ones or lots of inexpensive ones. Servers that provide adequate capacity and are moderately priced can generally accommodate all your needs just as well as expensive ones at a fraction of the cost.
Techniques for Creating Scalable and Highly Available Sites 239 Techniques for Creating Scalable and Highly Available Sites Now that you have a fairly good understanding of scalability and availability, the next step is to familiarize yourself with the techniques you can use to achieve scalable and highly available Web sites.
240 Chapter 11 Scalability and Availability Overview Clustering for failover relies on redundant servers to ensure that business-critical applications remain available if one of the servers in a cluster fails. Intelligent software-based failover solutions can detect when a server has failed and automatically redirect new incoming HTTP requests to the cluster members that are available.
Techniques for Creating Scalable and Highly Available Sites 241 The following figure shows a router distributing requests in round-robin fashion to the available servers in a Web server cluster: Advantages A hardware-based clustering solution, such as a router, is an attractive solution for the following reasons: • Proven technology • Relatively low complexity • No recurrent licensing fees • Semi-intelligent Routers can load balance in a round-robin fashion, detect failures, redirect traffic and remove f
242 Chapter 11 Scalability and Availability Overview Considerations Carefully evaluate the following issues against a router’s attributes: • Expense Hardware devices can be expensive relative to some software solutions, even without yearly licensing fees. • Single point of failure If a problem develops on the load-balancing device itself and it fails, your load balancing and failover strategies are no longer working.
Techniques for Creating Scalable and Highly Available Sites 243 • Optimizing load balancing scheme with application-aware and session-aware load balancing • Automatically detecting failures • Automatically redirecting traffic to available servers • Automatically notifying administrators of problems Advantages The following benefits make a software-based clustering solution attractive: • Relatively low expense Compared to the cost of hardware devices, such as routers or switches, software-based clustering
244 Chapter 11 Scalability and Availability Overview • Platform constraints Determine if the software solution you are considering will be available on your platform or operate with your preferred Web server. If reviewing data sheets and other marketing collateral from vendors, make sure that the robust features you want are available on the platform you need. • Level of complexity Some software-based clustering solutions have relatively low complexity.
Chapter 12 Configuring ColdFusion Clusters Once you have configured your Web site and installed ClusterCATS, use the procedures in this chapter to create and configure your clusters. Contents • Introduction to ClusterCATS Administration ....................................................... 246 • Creating Clusters ..................................................................................................... 252 • Removing Clusters ..................................................................
246 Chapter 12 Configuring ColdFusion Clusters Introduction to ClusterCATS Administration ClusterCATS consists of three components: • ClusterCATS Server • ClusterCATS Explorer and ClusterCATS Web Explorer • ClusterCATS Server Administrator and btadmin The components are described in the sections that follow. All of the components are installed on a machine when you run the ClusterCATS for ColdFusion installation program.
Introduction to ClusterCATS Administration 247 • Configuring e-mail-based alarm notifications • Monitoring clusters Note You can run the ClusterCATS Explorer from any server in the cluster, or you can run it remotely. This flexibility allows administrators in different geographic locations the ability to administer distributed clusters. You can also use ClusterCATS Explorer to administer UNIX clusters from a single Windows machine. Multiple clusters can be viewed from a single Explorer.
248 Chapter 12 Configuring ColdFusion Clusters ClusterCATS Web Explorer (UNIX only) ColdFusion Enterprise includes the ClusterCATS Web Explorer (btweb) for administering UNIX-only clusters. It is a graphical, cross-platform, Web-based utility used to create, configure, and administer ClusterCATS clusters. Note ClusterCATS for ColdFusion only installs ClusterCATS Web Explorer on UNIX servers but you can access it from any computer with an Internet browser.
Introduction to ClusterCATS Administration 249 Apache considerations Make the following changes to the Apache Web server’s httpd.conf file to enable the ClusterCATS Web Explorer (btweb). Replace the IP address specified in the example below (192.168.96.71) and the port (2222) with one appropriate for your system and enable authentication for the virtual directory. ### ### BTWeb Administration ### Listen 192.168.96.71:2222
250 Chapter 12 Configuring ColdFusion Clusters For Apache: http://:/default.html servername or virtual_host is the name of the Web server on which you installed ClusterCATS and is the communication port number that the Web server or virtual host has been configured to listen for HTTP requests. The Enter Network Password dialog box appears: 3 Enter your user name and password in the appropriate fields and click OK. Note The default user name and password is admin.
Introduction to ClusterCATS Administration 251 ClusterCATS Server Administrator The ClusterCATS Server Administrator is a Windows-based utility that lets you perform server-specific maintenance activities for each server in a cluster. Unlike the ClusterCATS Explorer, which let you administer your clusters from a single, central computer, you must run the ClusterCATS Server Administrator from each server in your cluster.
252 Chapter 12 Configuring ColdFusion Clusters btadmin btadmin is a scriptable utility that lets you perform server-specific maintenance activities for each server in a cluster. btadmin is available on both UNIX and Windows servers. Unlike the ClusterCATS Web Explorer, which lets you administer your entire cluster from a single, central computer, you must use btadmin from each server in your cluster.
Creating Clusters 253 To create a server cluster using the Cluster Setup Wizard: 1 Select Start > Programs > ColdFusion > ClusterCATS Explorer. The ClusterCATS Explorer opens: 2 Select Configure > Cluster Setup Wizard. Alternatively, you can click the Cluster Setup Wizard icon that appears in the toolbar.
254 Chapter 12 Configuring ColdFusion Clusters 3 Enter a name for your cluster and GoColdFusion in the License Key field and click Next. Note The License Key field is case-sensitive, so be sure to enter the key exactly as shown in this step. Make your cluster names logically consistent with their purpose. For example, Sales Web, Customer Support Web, and so on. The List of Web Servers dialog box appears: 4 Click Add to add available Web servers to your cluster.
Creating Clusters 255 If you are not configuring this Web server for offline maintenance support, go to step 8. Note You can only set the maintenance support option when creating a cluster or adding a cluster member to a cluster. You cannot configure or modify this option after you have created and added the cluster member to the cluster. Enabling maintenance support for clusters requires that you configure your cluster for ClusterCATS dynamic IP addressing.
256 Chapter 12 Configuring ColdFusion Clusters 10 If you want to use the default load threshold settings, click Next and go to step 13. However, if you do not want to use the defaults, select the server and click Configure to configure new peak and gradual redirect load thresholds for that cluster member. The Load Thresholds dialog box appears: 11 Enter new numerical values (not higher than 100%) in the Peak Load Threshold and Gradual Redirect fields and click OK.
Creating Clusters 257 14 If you want to configure different types of alerts to go to different people, click Details in the Alert Notification dialog box. The Alarm Notification dialog box appears: 15 Select an alert event and enter the e-mail address of the recipient. If you want the same person to receive the majority of alerts, click Propagate to automatically fill each event’s Recipient column with the same e-mail address. You can then manually change the few recipients that are different.
258 Chapter 12 Configuring ColdFusion Clusters 16 If your server cluster supports a site that needs to maintain persistent state on the same Web server during a user session, select Yes to enable session-aware load balancing. Otherwise, select No and click Next. The Load Balancing Device dialog box appears: 17 If you are using a hardware-based load balancing device in addition to ClusterCATS to manage and distribute load, enter the name of the Web site that this device supports (for example, www.
Creating Clusters 259 To manually create clusters: 1 Select Start > Programs > ColdFusion > ClusterCATS Explorer. The ClusterCATS Explorer opens: 2 Select Cluster Manager > New Cluster. Alternatively, you can right-click the Cluster Manager icon and select New Cluster or click the New Cluster button in the toolbar.
260 Chapter 12 Configuring ColdFusion Clusters 3 Add a new cluster using the fields as described in the following table: Field Description Cluster Name Enter a unique name for the cluster. Make your cluster names logically consistent with their purpose. For example, Sales Web, Customer Support Web, and so on. License Key Enter GoColdFusion. This field is case-sensitive, so be sure to enter the key exactly as shown. Web Server Name Enter the fully qualified host name (for example, doc.allaire.
Creating Clusters 261 Creating clusters in UNIX 1 Open the ClusterCATS Web Explorer if it is not already opened. 2 Click the Create New Cluster link.
262 Chapter 12 Configuring ColdFusion Clusters 3 Add a new cluster using the fields as described in the following table: Field Description Cluster Name Enter a unique name for the cluster. Make your cluster names logically consistent with their purpose. For example, Sales Web, Customer Support Web, and so on. Web Server Name Enter the fully qualified host name (for example, doc.allaire.com) for the first server you want to be a member of this cluster.
Removing Clusters 263 Removing Clusters To delete an entire cluster, you must delete each cluster member from the cluster individually, using the procedure described in “Removing Cluster Members” on page 266. Note When deleting cluster members, you must delete the Admin Manager (Windows) or the Admin Agent (UNIX) last. This server is the first server you added to the cluster. When the last cluster member has been removed, the cluster itself is deleted.
264 Chapter 12 Configuring ColdFusion Clusters Adding Cluster Members You can add servers to an existing cluster at any time. This section describes the following: • “Adding cluster members in Windows” on page 264 • “Adding cluster members in UNIX” on page 265 Adding cluster members in Windows Use the ClusterCATS Explorer to add servers to a cluster.
Adding Cluster Members Enabling maintenance support for clusters requires that you configure your cluster for ClusterCATS dynamic IP addressing. For more information, see “ClusterCATS Dynamic IP Addressing (Windows only)” on page 334 . 5 Enter the fully qualified host name of the maintenance address (for example, serv1.yourcompany.com) in the Maintenance Address field. 6 Click OK. 7 Repeat steps 2 through 6 to add additional servers to the cluster manually.
266 Chapter 12 Configuring ColdFusion Clusters Removing Cluster Members You can remove servers from an existing cluster at any time. This section describes the following: • “Removing cluster members in Windows” on page 266 • “Removing cluster members in UNIX” on page 267 Removing cluster members in Windows Use the ClusterCATS Explorer to remove cluster members. To remove a cluster member from a cluster: 1 Open the ClusterCATS Explorer and select a cluster member. 2 Select Server > Delete.
Removing Cluster Members 267 Removing cluster members in UNIX Use the ClusterCATS Web Explorer to remove cluster members. To remove a cluster member from a cluster: 1 Open the ClusterCATS Web Explorer if it is not already open. 2 Click the Delete Server link. The Delete Server page appears: 3 Select the cluster member you want to delete from the Web Server Name drop-down box. A message appears telling you that the selected server has been deleted.
268 Chapter 12 Configuring ColdFusion Clusters Server Load Thresholds ClusterCATS makes certain that your Web applications remain available and running at optimum performance by intelligently managing the amount of HTTP traffic hitting your clustered servers. By setting load thresholds on each server in your cluster, you can control and manage your site’s availability and performance.
Server Load Thresholds 269 The server’s Properties dialog box appears: 3 Select the Load tab. 4 Enter a new numeric value (less than 100%) in the first Load Management field. This is referred to as the Peak load threshold. In the example above, the Peak load threshold is set to 90. 5 Enable the Gradual Redirection check box. 6 Enter a new value in the Gradual Redirection field. This value must be lower than the Peak load threshold. 7 Click OK to apply your new threshold settings.
270 Chapter 12 Configuring ColdFusion Clusters Viewing a cluster’s load status ColdFusion reports its load data directly to ClusterCATS. Consequently, you can view the load on the ColdFusion servers at any time using the Server Load Monitor. To view your cluster’s current load levels: 1 Open the ClusterCATS Explorer and select a cluster. 2 Select Monitor > Load. Alternatively, you can right-click the cluster you have selected and select Monitor > Load.
Server Load Thresholds 271 To configure load threshold settings using the Server Load dialog box: 1 Open the ClusterCATS Explorer and select a server. 2 Select Monitor > Load. Alternatively, you can right-click the server and select Monitor > Load. The Server Load dialog box appears: 3 Use your mouse to drag the Peak load threshold (red) up or down. As you move the line, the Peak load threshold percentage changes. 4 Enable gradual redirection by selecting the Gradual Redirection check box.
272 Chapter 12 Configuring ColdFusion Clusters Configuring load thresholds on UNIX To configure load thresholds for a cluster member: 1 Open the ClusterCATS Web Explorer if it is not already open. 2 Click the Show Cluster link. The Show Cluster page appears: 3 Enter the fully qualified host name of a server in the Web Server Name field.
Server Load Thresholds 4 273 Click OK. The Cluster Member List page appears, as the following figure shows. If you get an "Error: Server could not be found" message, make sure you used the correct, fully-qualified server name and that the server is running.
274 Chapter 12 Configuring ColdFusion Clusters 5 Click the Server Attributes link. The Connect To Server page appears: 6 Select the server you want to connect to from the Web Server Name listbox.
Server Load Thresholds 7 Click OK. The selected server’s Server Properties page appears: 8 Click the Administration link under Server Attributes. The Server Administration page appears for the selected server.
276 Chapter 12 Configuring ColdFusion Clusters 9 To change the Peak load threshold, enter a new numeric value (less than 100%) in the Standard Load Threshold field. 10 Enable the Gradual Redirection check box if it is not already enabled. 11 To change the Gradual Redirection load threshold, enter a new numeric value in the Gradual Load Threshold field. This value must be lower than the Standard Load Threshold. 12 Click OK to apply your new load threshold settings.
Session-Aware Load Balancing Enabling session-aware load balancing on Windows To enable session-aware load balancing: 1 Open the ClusterCATS Explorer and select a cluster. 2 Select Configure > Administration. Alternatively, you can right-click on the cluster and select Configure > Administration. The Cluster Properties dialog box appears: 3 Select the Session State Management check box. 4 Click OK.
278 Chapter 12 Configuring ColdFusion Clusters Enabling session-aware load balancing on UNIX To enable session-aware load balancing: 1 Open ClusterCATS Web Explorer if it is not already open. 2 Click the Show Cluster link. The Show Cluster page appears: 3 Enter the fully qualified host name of the server for which you want to configure session-aware load balancing in the Web Server Name field.
Session-Aware Load Balancing 4 Click OK. The Cluster Member List page appears: 5 Click the Administration link under Cluster Attributes.
280 Chapter 12 Configuring ColdFusion Clusters 6 Select the Enable session-aware load balancing check box. 7 Click OK to enable session-aware load balancing for the selected cluster. Configuring ColdFusion probes in Windows This section describes the following: • “Adding ColdFusion probes” on page 280 • “Removing ColdFusion probes” on page 285 Adding ColdFusion probes ClusterCATS lets you set up one probe monitor for each server in the cluster.
Session-Aware Load Balancing To add a new monitor and ColdFusion probe: 1 Open the ClusterCATS Explorer and select a server. 2 Select Server > New Monitor. Alternatively, you can right-click the server and select New Monitor.
282 Chapter 12 Configuring ColdFusion Clusters 3 Enter a name you want to assign to this probe’s monitor in the Name field on the New Monitor dialog box and click OK. The monitor’s Properties dialog box appears: 4 Click the New Probe button . The ColdFusion Web Application Probe settings dialog box appears: 5 Configure the application probe settings as described in the following table: Field Description Web Server Select the name of the server from the drop-down list.
Session-Aware Load Balancing 283 Field Description Working directory Enter the absolute path to the probe’s working directory. Do not change the default selection unless you installed ColdFusion to a directory other than the default installation directory. Startup Parameters Replace the with the actual URL of the site you want the probe to access, and replace with a text string that appears on apage on the site you are probing. Tips.
284 Chapter 12 Configuring ColdFusion Clusters 6 Click Register to create the probe. 7 Close all open dialog boxes. Icons for the monitor and probe appear under the Monitor Manager in the ClusterCATS Explorer. To add a new probe to an existing probe monitor: 1 Open the ClusterCATS Explorer. 2 Select the cluster_name > Monitor Manager > monitor_name in the left pane. 3 Select Monitor > Properties. The monitor’s Properties dialog box appears: 4 Click the New Probe button .
Session-Aware Load Balancing 6 Click Register to create the probe. 7 Close all open dialog boxes. 285 An icon for the new probe appears under the Monitor Manager in the ClusterCATS Explorer. Removing ColdFusion probes To remove a ColdFusion probe: 1 Open the ClusterCATS Explorer. 2 Select the cluster_name > Monitor Manager > monitor_name > probe_name in the left pane. 3 Select Probe > Delete. Alternatively, you can right-click the probe and select Delete.
286 Chapter 12 Configuring ColdFusion Clusters 8 Click the ColdFusion Probe link.
Session-Aware Load Balancing 9 287 To create a new probe, click New. The ColdFusion Application Probe page appears: If this is the first probe for this server or you clicked New to add another probe, the ColdFusion Application Probe page appears: 10 Configure the application probe settings as described in the following table. Field Description Status This is an informational field. If the probe is not registered, the Status displays Not registered.
288 Chapter 12 Configuring ColdFusion Clusters Field Description Startup Parameters Enter the actual URL of the site you want the probe to access followed by a text string that appears on a page within the site you are probing (cfprobe.cfm in the screen shown in step 9.) Note: Do not modify the RESTART explicit parameter if you want the probe to automatically restart the ColdFusion Server upon detecting a failure.
Session-Aware Load Balancing 4 Click OK. The Cluster Member List page appears. 5 Click the Server Attributes link. The Connect To Server page appears. 6 Select the server that hosts the probe in the Web Server Name listbox. 7 Click OK. The selected server’s Properties page appears. 8 Click the ColdFusion Probe link. The Probe List page appears. 9 Select the probe you want to edit or remove. 289 10 To remove the probe, click Delete. ClusterCATS removes the ColdFusion probe.
290 Chapter 12 Configuring ColdFusion Clusters Load-Balancing Devices You can configure ClusterCATS to work in conjunction with a third-party hardware load balancing device or load balancing software product to provide comprehensive load balancing and failover support for your server clusters.
Load-Balancing Devices 291 • If two or more Web servers on the same system are in clusters using Cisco LocalDirector load balancing, then each cluster must have the same DFP Agent Listen Port number configured. The ClusterCATS DFP agent can only listen on one port. LocalDirector dynamic-feedback command settings Use the LocalDirector dynamic-feedback command options as described in this section to optimize your LocalDirector setup. Note Do not use the dynamic-feedback-pw command.
292 Chapter 12 Configuring ColdFusion Clusters LocalDirector will attempt to reconnect, indefinitely, every 30 seconds. The LocalDirector will close the connection if it is inactive for 60 seconds. For more information on the dynamic-feedback command options, refer to “LocalDirector dynamic-feedback command settings” on page 291. 6 Open the ClusterCATS Explorer and select a cluster. 7 Select Cluster > Properties or select Configure > Administration.
Load-Balancing Devices 293 8 Select the Load Balance tab and choose Cisco LocalDirector from the Load Balancing Product drop-down list. 9 Edit the cluster properties as described in the following table. Field Description Website Alias Enter the name of the virtual server (www.yourcompany.com) you created in step 3. LocalDirector IP Address Enter the IP address of the Cisco LocalDirector.
294 Chapter 12 Configuring ColdFusion Clusters Field Description HTTPS Port Enter the port number on which each cluster member listens for secured HTTP requests. Enter 0 if not applicable. Bind ID Enter the same Bind ID specified for the explicit (real) servers on the LocalDirector in step 4.
Load-Balancing Devices 295 3 Select Configure > Administration. Alternatively, you can right-click the cluster and select Configure > Configure. The Cluster Properties dialog box appears: 4 Select the Load Balance tab. The selection in the Load Balancing Product drop-down list indicates how ClusterCATS will actively load balance HTTP traffic across the cluster. 5 Enter the name of the Web site in the Website Alias field. 6 Click OK to apply your changes.
296 Chapter 12 Configuring ColdFusion Clusters 6 In the Load Balancing Product field, enter the URL of the Web site for which the load balancing product has been set up to manage HTTP traffic. 7 Click OK to apply your changes. Administrator Alarm Notifications The ClusterCATS alarm notification feature provides instant feedback about critical events that take place within a cluster. Once an event triggers an alarm, ClusterCATS notifies one or more people by e-mail.
Administrator Alarm Notifications 297 Configuring administrator alarm notifications on Windows To configure an alarm notification: 1 Open the ClusterCATS Explorer and select a cluster. 2 Select Configure > Alarm Notification. Alternatively, you can right-click the cluster and select Configure > Alarm Notification.
298 Chapter 12 Configuring ColdFusion Clusters 4 Click OK. The Cluster Member List page appears. 5 Click the Alarm Notification link. The Alarm Notification page appears: 6 Enter the e-mail address of the person you want to be notified about the occurrence of an event in that event’s corresponding field. If you want multiple people to receive an e-mail notification about the same event, add more e-mail addresses to the field and separate each e-mail address with a comma.
Administrator E-mail Options 299 Administrator E-mail Options The ClusterCATS administration e-mail support feature reports vital statistics about your cluster to designated e-mail accounts in your organization. You can set up the following types of administration e-mail options: • Report e-mail Lets you know each day how your server clusters are functioning.
300 Chapter 12 Configuring ColdFusion Clusters Configuring administration e-mail options on Windows To configure administration e-mail options: 1 Open the ClusterCATS Explorer and select a cluster. 2 Select Configure > Support. Alternatively, you can right-click the cluster and choose Configure > Support.
Administrator E-mail Options 301 3 Enter the fully qualified host name of a server for which you want to configure administrator e-mail support in the Web Server Name field. 4 Click OK. The Cluster Member List page appears. 5 Click the Support link. The Cluster Support page appears: 6 Edit the e-mail support fields as described in the following table: Field Description SMTP Gateway Enter the name of the server through which outgoing e-mail will be sent.
302 Chapter 12 Configuring ColdFusion Clusters Administrating Security When you enable ClusterCATS administration security for a specific cluster, only authorized users are able to access and administer that cluster using their ClusterCATS Explorer (Windows) or the ClusterCATS Web Explorer (UNIX). ClusterCATS provides three administration security settings for securing your server cluster environment: • Disabled Authentication This is the default setting.
Administrating Security 303 To configure authentication modes for your clusters: 1 Create a user account on each server within your cluster for each administrator that you want to be able to administer the servers using the ClusterCATS Explorer. For Unix, you must be a member of "sys" group. For Windows NT, you must be a member of "admin" group. If your cluster members are NT servers, use the Windows User Manager utility to create your user accounts.
304 Chapter 12 Configuring ColdFusion Clusters Note ClusterCATS requires you to enter a valid user name and password after selecting the type of authentication you are using so that you do not inadvertently lock yourself out of the cluster. 6 Click OK to enable local user authentication for the selected cluster. Only administrators who have accounts on each secured server can access and administer those cluster members using ClusterCATS Explorer.
Administrating Security 305 5 Select the domain from the List Names drop-down box. 6 Select the users you want to add to the group and click Add. 7 Click OK in all open dialog boxes to apply your changes and to close the User Manager for Domains utility. 8 Open the ClusterCATS Explorer and select the cluster for which you want to configure authentication. 9 Select Configure > Administration or select Cluster > Properties. Both menu selections display the Properties dialog box.
306 Chapter 12 Configuring ColdFusion Clusters Configuring authentication on UNIX To configure authentication modes for your clusters: 1 Open ClusterCATS Web Explorer if it is not already open. 2 Click the Show Cluster link. The Show Cluster page appears. 3 Enter the fully qualified host name of the server for which you want to configure administrator authentication in the Web Server Name field. 4 Click OK. The Cluster Member List page appears. 5 Click the Authentication link.
Chapter 13 Maintaining Cluster Members After you have created your clusters, added servers to those clusters, and configured them with load balancing and high availability features, they will likely run inconspicuously in your environment for quite some time. However, at some point you may need to update software and content or perform general maintenance tasks that are beyond the typical cluster creation and configuration activities. Contents • Understanding ClusterCATS Server Modes ....................
308 Chapter 13 Maintaining Cluster Members Understanding ClusterCATS Server Modes ClusterCATS allows you to move cluster members into various modes of operation depending on the tasks you want to perform on that server. These modes allow you to remove servers from clusters to perform maintenance activities without disturbing the current traffic flow among other things.
Changing Active/Passive Settings 309 Changing Active/Passive Settings All cluster members are added to a cluster with the ClusterCATS Server in Active state by default. In Active state, ClusterCATS Servers intercept requests to your Web resources and provide availability and failover services. From time to time, you may want to turn off these load balancing and failover services to help you troubleshoot problems. To do this, change the ClusterCATS Server’s state from Active to Passive.
310 Chapter 13 Maintaining Cluster Members Changing active/passive settings in UNIX To change a cluster member’s state: 1 Open ClusterCATS Web Explorer if it is not already open. 2 Click the Show Cluster link. The Show Cluster page appears. 3 Enter the fully qualified host name of the server in the Web Server Name field. 4 Click OK. The Cluster Member List page appears. 5 Click the Server Attributes link under Other. The Connect To Server page appears.
Changing Restricted/Unrestricted Settings 311 Changing Restricted/Unrestricted Settings ClusterCATS lets you stop a cluster member from receiving any HTTP requests by changing the restricted/unrestricted setting. You may want to restrict a server when performing server maintenance or software updates, verifying load configurations, or as an alternative method to managing load.
312 Chapter 13 Maintaining Cluster Members 6 Click OK. Restricting/unrestricting servers in UNIX To change restriction settings for a cluster member: 1 Open ClusterCATS Web Explorer if it is not already open. 2 Click the Show Cluster link. The Show Cluster page appears: 3 Enter the fully qualified host name of a server in the Web Server Name field. 4 Click OK. The Cluster Member List page appears. 5 Click the Server Attributes link under Other. The Connect To Server page appears.
Using Maintenance Mode (Windows only) 313 10 To allow this server to participate in the cluster as normal, select Unrestricted from the Restriction Status drop-down box. 11 Click OK. Using Maintenance Mode (Windows only) Putting a ClusterCATS Server in Maintenance mode lets you remove a server from an active cluster gracefully so that you can perform necessary updates or maintenance tasks without disrupting your users.
314 Chapter 13 Maintaining Cluster Members To put a cluster member in Maintenance mode: 1 Open the ClusterCATS Explorer and select a cluster member that you want to update. 2 Select Configure > Load. Alternatively, you can right-click the cluster member and select Configure > Load. The Properties dialog box appears for the selected cluster member with the Load tab active. 3 Change the Peak load threshold to 0% so that any additional HTTP requests will be redirected to other servers in the cluster.
Using Maintenance Mode (Windows only) 5 Physically go to the server you selected in step 1 and open the ClusterCATS Server Administrator utility on this server by selecting Start > Programs > ColdFusion 3.0 > ClusterCATS Server Administrator The ClusterCATS Server Administrator appears: 6 315 Click the Service Status window button to display the Manage ClusterCATS Services dialog box.
316 Chapter 13 Maintaining Cluster Members 7 Select the Stopped option to stop the ClusterCATS service and enter a value, in minutes, in the Drain Down Period field. This allows current users to conclude their sessions within the time indicated. 8 Click OK. When the drain-down period expires, the server will fail over to another server in the cluster.
Updating an Existing Cluster Member (Windows only) 317 Updating an Existing Cluster Member (Windows only) Periodically you will need to update software or content that resides on your cluster members. Software updates might include new versions or patches to operating system software, Web server software, new Web applications, ClusterCATS software, or other third-party products.
318 Chapter 13 Maintaining Cluster Members 7 Select Running. ClusterCATS will add the cluster member back into the cluster. 8 To initially limit the amount of HTTP traffic sent to the server, return to the ClusterCATS Explorer and reconfigure the cluster member’s Peak Load threshold to a low value such as 10%. 9 Click OK. 10 Within the ClusterCATS Explorer, right-click the cluster member and select Monitor > Load.
Resetting Cluster Members 319 Resetting Cluster Members ClusterCATS includes a utility for resetting cluster members to their pre-clustered state. You may want to do this for two reasons: • You want to permanently remove a cluster member from a cluster • You want to change a cluster member from one cluster to another cluster To perform both of these tasks, you must first reset each server’s configuration to its original, pre-clustered state.
320 Chapter 13 Maintaining Cluster Members Resetting cluster members on UNIX Enter the following command at the server you want to reset: btadmin -reset
Chapter 14 ClusterCATS Utilities ColdFusion Enterprise ships with a number of scriptable command-line utilities for configuring, administering, and troubleshooting your ClusterCATS clusters. This chapter describes these utilities. Contents • Using btadmin ......................................................................................................... 322 • Using bt-start-server and bt-stop-server (UNIX only) ......................................... 325 • Using btcfgchk .........................
322 Chapter 14 ClusterCATS Utilities Using btadmin btadmin is a scriptable utility installed on each server in cluster. It provides most of the functionality of the Windows-based ClusterCATS Server Administrator so that UNIX and Windows administrators can include calls in automated scripts.
Using btadmin 323 The following table describes the btadmin options for changing the ClusterCATS settings: Option Description enable Enable the specified option for a Web server instance. disable Disable the specified option for a Web server instance. add Add a new Web server instance. delete Delete an existing Web server instance. config Configure a specified option for an instance. btadmin prompts you for additional information when using the config option.
324 Chapter 14 ClusterCATS Utilities [help] Use the help option to get a list of the btadmin utility’s features and syntax. Using btadmin on Windows btadmin is a Windows executable invoked from the command line in the /program directory. The table below describes each of the options and their syntax for btadmin. Option Description btadmin Displays btadmin online help. btadmin -v Displays the current version of Microsoft’s IIS if it is bound to the ClusterCATS Server.
Using bt-start-server and bt-stop-server (UNIX only) 325 Using bt-start-server and bt-stop-server (UNIX only) The bt-start-server and bt-stop-server utilities start and stop the Web server that is bound to the ClusterCATS Server. This command starts or stops either the Netscape Enterprise Server or Apache Web server.
326 Chapter 14 ClusterCATS Utilities btcfgchk DNS errors The btcfgchk utility reports on DNS configuration problems. ClusterCATS requires that your DNS be configured with correct forward and reverse mappings. A forward mapping (AName record) translates the host name to an IP address. Conversely, a reverse mapping (PRT record) translates an IP address to its host name. ClusterCATS expects the mapping to be one-to-one (one host name to one IP Address).
Using btcfgchk 327 Error Description Error looking up by name ClusterCATS could not resolve the given host name to an IP address. Use nslookup to look up the host name in DNS. Host name a round-robin name, or does not map to configured IP address The host name maps to more than one IP address (round-robin DNS) or maps to an IP address not found on this machine.
328 Chapter 14 ClusterCATS Utilities Using hostinfo The hostinfo utility is a network management tool that displays information about a specified domain name. Use it to analyze and troubleshoot problems you are having with DNS mappings to a particular domain. Syntax Invoke hostinfo from the command line in the / program/directory using the following syntax: hostinfo [fully_qualified_host_name] Specifying a fully qualified host name is optional.
Using sniff 329 Using sniff The sniff utility is a network management tool that displays the packets that a specific Network Interface Card (NIC) is hearing.
330 Chapter 14 ClusterCATS Utilities
Using sniff 331
332 Chapter 14 ClusterCATS Utilities
Chapter 15 Optimizing ClusterCATS ColdFusion Enterprise provides some enhanced capabilities that allow you to customize your ClusterCATS implementation. This chapter describes some of these options. Contents • ClusterCATS Dynamic IP Addressing (Windows only) ........................................ 334 • Using Server Failover............................................................................................... 340 • Configuring Load-Balancing Metrics .............................................
334 Chapter 15 Optimizing ClusterCATS ClusterCATS Dynamic IP Addressing (Windows only) This section describes how to enable ClusterCATS dynamic IP addressing on your site. You do not have to configure your system on UNIX for dynamic IP addressing because it is set up by default.
ClusterCATS Dynamic IP Addressing (Windows only) 4 335 Create your clusters. “Creating clusters in Windows” on page 252. Benefits of ClusterCATS dynamic IP addressing There are several benefits to using ClusterCATS dynamic IP addressing: • Using Maintenance mode. With dynamic IP addressing, cluster members put into Maintenance mode on Windows clusters will fail over to another server and then gracefully return when brought out of Maintenance mode.
336 Chapter 15 Optimizing ClusterCATS To set up a maintenance address prior to installing ClusterCATS: 1 Back up your system files. 2 Obtain a new IP address and new computer name. Be sure to configure your DNS so that your new address has both forward and reverse DNS entries. 3 For IIS 4.0 and 5.0: Uninstall any products which are configured as part of IIS, including Allaire ColdFusion. 4 For IIS 4.0: Uninstall the Windows NT 4.
ClusterCATS Dynamic IP Addressing (Windows only) 8 337 Enter a new name for the computer in the Computer Name field. This name corresponds to the new IP address that you just added. Do not change the Domain field on this tab. Note The Computer Name on the Identification tab should only be a NetBIOS name, not a fully-qualified host name (FQHN). For example, support1.allaire.com is a possible FQHN. The first portion of this FQHN (support1) can be a NetBIOS name.
338 Chapter 15 Optimizing ClusterCATS To enable dynamic addressing: 1 Verify that you can access your server via its maintenance address. If not, assign one to the server using the procedure described in “Setting up maintenance IP addresses” on page 335. 2 Configure your Web server to support ClusterCATS dynamic IP addressing.
ClusterCATS Dynamic IP Addressing (Windows only) 339 6 Open the Advanced IP Addressing dialog box by right-clicking Network Neighborhood and select Properties. On the Protocols tab, select TCP/IP Protocol and click Properties and then click Advanced. 7 Unbind the IP addresses from the Web server’s NIC by selecting each IP address in the IP Addresses region and clicking Remove. This removes the IP addresses corresponding to the Web Site. 8 Click OK three times.
340 Chapter 15 Optimizing ClusterCATS Using Server Failover The ability to fail over servers that have become unavailable to redundant servers is a cornerstone of any mission-critical application, one that ensures an application’s continuous and reliable operation. Server failover was an option to select during the installation process. If you did not select it during installation, you must reinstall ClusterCATS and select that option.
Configuring Load-Balancing Metrics 341 Configuring Load-Balancing Metrics ColdFusion Enterprise provides you the option of customizing the load balancing metrics of Web servers clustered with Allaire ClusterCATS software. This section describes how to customize the metrics to your specific Web site implementation. Overview of metrics The ColdFusion server records the time each JSP page and servlet request takes to be processed and can return metrics derived from this timing data upon request.
342 Chapter 15 Optimizing ClusterCATS Load types The probed JSP page is located at /btauxdir/ getsimpleload.jsp. The probe agent responds to output generated by this page and uses it to calculate the overall load based on the weighting of the two available metrics set in the LOADTYPE variable: • AVG_REQ_TIME AVG_REQ_TIME calculates load based on the average service request time. The load is derived by dividing the request time by the maximum acceptable request time.
Configuring Load-Balancing Metrics 343 • CCRTTPercent CCRTTPercent represents the percentage of the calculated average ROUND_TRIP_TIME that the probe agent should apply to the load metric supplied by CCLOADVALUE. CCRTTPercent is the second variable that you might change in getsimpleload.jsp to customize your server’s load metrics. It acts as a tuning knob to determine how much external influence on server performance should be calculated into the server's overall load value.
344 Chapter 15 Optimizing ClusterCATS
Index A A records 230 absolute hyperlinks 276 Access OLE DB providers 5 Active mode described 308 Active/Passive mode changing 309 changing in UNIX 310 changing in Windows 309 adding cluster members UNIX 265 Windows 264 Admin Agent defined 263 Admin Manager defined 263 administering ClusterCATS alarm notifications 296 Apache considerations 249 btadmin 252 ClusterCATS Explorer 246 ClusterCATS Web Explorer 248 e-mail support options 299 introduction 246 Netscape considerations 248 opening the Web Explorer 24
346 btcfgchk DNS Errors 326 sample output 325 syntax 325 bt-start-server usage 325 bt-stop-server usage 325 btweb 248 busy state 313 C cached query connection string 13 CCLOADMAX 342 CCLOADVALUE 342 CCRTTPercent 343 CFAUTHENTICATE 95 CFAUTHENTICATE tag 99 cfcollection 119 cfdocumentation 119 cfquery as diagnostic for unverified data source 8 creating a data source in 10 cfsearch 119 Cisco LocalDirector and DFP Agent Listen Port 291 and dynamic IP addressing 290 and gradual redirection 290 and Passive mode
Index Connecting DB2 data sources 15 dBASE/FoxPro 21 Excel 24 Excel Workbook 25 Informix 26 Informix data sources 27 Informix through ODBC/CLI 29 Sybase 32 text databases 35 Visual FoxPro 37 connection string about 12 connectstring attribute 13 in cached query 13 passing attribute-value pairs 12 viewing information in SQL Server 12 creating clusters 252 in UNIX 261 in Windows 252 manually 258 Windows 252 with hardware solutions 240 with software solutions 242 D data error codes 214 data source creation in
348 Index H hardware planning for failover 237 hardware-based clustering advantages 241 considerations 242 illustrated 241 solutions 240 hostinfo 328 sample output 328 syntax 328 HTTP redirection 268 HTTP server failure alarm notification 296 hyperlinks relative 276 I icon legend 247 Indexing XML documents overview 138 indexing XML documents configuring style files 139 configuring style.xml 139 implementation summary 138 prerequisites 143 searching using rcvdk 143 Style Files 139 style.dft 142 style.
Index maintenance support in ClusterCATS enabling 260 merge, using Verity 211 merge, Verity utility 211 metrics average request time, described 341 configuring 341 last request time, described 341 load-balancing 341 output variables 342 overview 341 troubleshooting 343 Microsoft Data Access Components (MDAC) 5 version detection 5 mkvdk indexing XML documents with 143 mkvdk syntax 186 mkvdk, about optimized databases (VDBs) 198 mkvdk, about squeezing deleted documents 197 mkvdk, accessing online help 188 mk
350 rcvdk, starting 201 rcvdk, using Verity 201 rcvdk, Verity utility 201, 202, 203 RDS Basic security 98 configuring basic security 73 RDS Security 85 rebooting avoiding double-reboot 335 redirecting traffic 268 with Maintenance mode 313 redundancy ensuring corrective actions 238 planning 237 systems monitoring 238 relative hyperlinks 276 relative vs.
Index Server sandbox security 65 server state changing 309 server unreachable alarm notification 296 Service Level Keywords 191 session management 225 session-aware load balancing description 276 enabling on UNIX 278 enabling on Windows 277 relative vs.
352 Sybase client software 9 syntax, mkvdk 186 System and services files 16 systems monitoring for failover 238 T technical support e-mail support 299 testing Web site load 232 text databases connecting 35 third-party load balancing devices 294 using in UNIX 295 using in Windows 294 thresholds 268 gradual redirection 268 training.
Index Verity Spider syntax command file use 149 command-line options -refresh 150 -start 149 overview 148 Verity Spider command 148 Verity utilities, overview 200 Verity utility, browse 209 Verity utility, didump 206 Verity utility, merge 211 Verity utility, rcvdk 201, 202 Verity VDK error messages 213 Verity warnings 217 version detection Microsoft Data Access Components (MDAC) 5 viewing the word list, Verity didump 206 virtual servers hardware-based clustering 240 Visual FoxPro connecting 37 W warnings,
354 Index
