User Guide

Networking Options 159
Networking Options
-agentname
Syntax: -agentname string
Typ e: Web crawling only.
Specifies the value for the agent name field that is part of the HTTP request. Since
Web servers can be configured to return different versions of the same page
depending on the requesting agent, you can use -agentname to impersonate a
browser client.
Use double-quotes if the name contains a space. Use -cmdfile if the agent name you
want to use contains forbidden characters such as slashes or backslashes.
-connections
Syntax: -connections num_connections
Details Specifies the maximum number of simultaneous socket connections to make
to Web sites for indexing. Each connection implies a separate thread.
The default value is 6.
Note
Verity Spider’s dynamic flow control makes the most use of all available connections
when indexing Web sites. If you are indexing multiple sites, you may want to increase
this number. Note that increasing the number of connections may not always help
because of such dependencies as your network connection and the capabilities of
the remote hosts.
-delay
Syntax: -delay num_milliseconds
Type: Web crawling only.
Details Specifies the minimum time between HTTP requests in milliseconds. The
default value is 0 milliseconds for no delay.
-header
Syntax: -header string
Typ e: Web crawling only
Specifies an HTTP header to be added to the spidering request. For example:
-header "Referer: http://www.verity.com/"
Verity Spider sends some predefined headers, such as Accept and User-Agent among
others, by default. Special headers are sometimes necessary to correctly index a site.