User Guide

Path and URL options 123
-retry
Typ e : Web crawling only
Syntax:
-retry num_retries
Specifies the number of times that Verity Spider should attempt to access a URL. Use the -retry
option when it is likely that an unstable network connection will give false rejections.
The default value is 4.
-timeout
Typ e : Web crawling only
Syntax:
-timeout num_seconds
Specifies the time period, in seconds, that Verity Spider should wait before timing out on a
network connection and on accessing data. The data access value is automatically twice the value
you specify for the network connection timeout.
The default value for the network connection time-out is 30 seconds, and therefore the default
value for the data access time-out is 60 seconds.
Path and URL options
The following sections describe the Verity Spider path and URL options.
-auth
Syntax:
-auth path_and_filename
Specifies an authorization file to support authentication for secure paths.
Use the
-auth option to specify the authorization file. The file contains one record per line. Each
line consists of server, realm, username, and password, separated by whitespace.
The following is a sample authorization file:
# This is the Authorization file for HTTP's Basic Authentication
#server realm username password
doleary MACR my_username my_password
-cgiok
Typ e : Web crawling only
Lets you index URLs containing query strings. That is, a question mark (?) followed by additional
information. This typically means that the URL leads to a CGI or other processing program.
The return document produced by the web server is indexed and parsed for document links,
which are followed and in turn indexed and parsed. However, if the web server does not return a
page, perhaps because the URL is missing parameters that are required for processing in order to
produce a page, nothing happens. There is no page to index and parse.