Neoview Database Administrator's Guide (R2.2)

8 Using the Neoview Loader
The Neoview Loader provides processes and commands for loading data into your Neoview
platform or extracting data from it. gcmd is the user interface for managing the Neoview Loader.
Using gcmd commands, you can load data from any data source running on a Linux platform.
You will probably need to do some data preparation before the data is suitable for loading. The
Neoview Loader performs a number of functions to transform properly formatted ASCII character
strings, and performs data type conversions to make them suitable for the target database column
(that is, data type, size, scale, and other SQL attributes).
The Neoview Loader accepts data as either fixed width format or delimited format. To take
advantage of the data transformations described above, use delimited format.
Each source data field and its value is mapped to a corresponding destination table column. If
the number of source data fields is different from the number of destination table columns, you
must use a format file. This file directs the Neoview Loader to ignore fields that will not used.
This additional processing overhead may affect load performance. For a description of the format
file, see “Using a Format File” (page 105).
This chapter describes:
• “Performance Considerations” (page 93)
• “Loading (Importing) Data” (page 94)
• “Input Data Considerations For Load Tasks” (page 100)
• “Command-Line Arguments for Load Tasks” (page 108)
• “Logs” (page 115)
• “Recovery” (page 116)
• “Extracting Data” (page 121)
• “Output Considerations for Extract Tasks” (page 125)
• “Command-Line Arguments for Extract Tasks” (page 127)
• “Neoview Loader FAQs” (page 132)
Performance Considerations
Loading performance will vary significantly depending on a number of factors of the system
environment and is primarily dependent upon how many concurrent activities are contending
for Neoview Loader system resources.
In general, a system with directly attached disks with the data already on them can load at about
1MB per second per CPU on a Neoview system, on configurations up to 64P. On larger
configurations a second data loader will be needed to maintain these data rates.
On systems where new data is being acquired from enterprise sources, or where other system
activities are going on while the Neoview Loader is running, performance will vary depending
on the mix of these activities.
Loading performance can also be affected by the type of data being uploaded (ascii decimal
compared to binary), null column support, and row size.
Extract performance will vary depending on a variety of factors, such as the system load, the
number of columns in the table and the data types. The main performance factor is whether the
extract is a serial or parallel extract:
For serial extracts, the extract rate can be expected to vary up to 20 MB per second.
For parallel extracts, the extract rate can be expected to vary up to 2MB per second per CPU on
configurations up to 64P. On larger configurations a second Loader will be required to maintain
these data rates.
Performance Considerations 93