User Guide

156 Chapter 8 Verity Spider

Note

You should not run more than one Verity Spider process in persistent mode. As the

Verity Spider is a resource intensive process, you should only run it in persistent

mode with an interval of less than one day. For time intervals greater than twelve

hours, you should use some form of scheduling. Some examples are cron jobs for

UNIX, and the AT command for Windows NT Server.

-preferred

Syntax: -preferred exp_1 [exp_n] ...

Typ e: Web crawling only

Specifies a list of hosts or domains which are to be preferred when retrieving

documents for viewing. You can use wildcard expressions, where the asterisk ( * ) is

for text strings and the question mark ( ? ) is for single characters. To use regular

expressions, also specify the -regexp option. Use this option when you leave

duplicate detection enabled and do not specify

-nodupdetect.

When indexing, you may encounter a non-preferred host first. In that case,

documents are parsed and followed and stored as candidates. When duplicates are

encountered on another server, which is preferred, the duplicate documents from

the non-preferred server are skipped. When documents are requested for viewing,

they will be retrieved from the preferred server.

On Windows NT, you should include double quotes around the argument to protect

the special characters such as (*). On UNIX, you should use single quotes. Note that

this is only required when you run the indexing job from a command line. Quotes are

not necessary within a command file (

-cmdfile).