Communicator e3000 MPE/iX Release 6.5 Express 2 (Software Release C.65.02) (30216-90322)

Chapter 4 59
Increases in Limits
Large Size Datasets in TurboIMAGE/iX
Large Size Datasets in TurboIMAGE/iX
B. T. Vikram Kumar
Commercial Systems Division- Databases
Introduction
This enhancement in TurboIMAGE/iX removes the current 80 GB limit on the size of a
dataset so as to allow users to have really large datasets. Now the only limiting factor is
the number of entries in a dataset, which can be up to 2**31 -1.
As the solution involves changes in the internal data structures of IMAGE, a new utility
DBBIGSET provides an easy way for existing databases to migrate to the new structures.
Once migrated, the structural details will be transparent to the users. User interface
changes have been made to DBSCHEMA, DBUTIL and DBINFO.
Old and new format databases can co-exist in the same system.
Dataset Size Limitations
The IMAGE database management system has been around for more than two decades.
When HP first introduced the product, the maximum size of a dataset was limited to the
MPE file size. However, when users felt the need for much larger dataset sizes, the
'JUMBO' feature was introduced. With this feature, a user can create a dataset with a size
greater than the MPE file size of 4GB. Even though the dataset physically spans across
multiple MPE files called chunk control files and chunkset files, the JUMBO feature allows
them to be seen as a single logical file. Even with this feature, users could only define a
dataset up to 40 GB in size, because of the limit of block numbers, whereas the JUMBO
feature per se doesn't have this restriction. Earlier, IMAGE used to transfer data in blocks,
which necessitated addressing of a record through a block number and an offset within the
block. The maximum number of blocks allowed to be addressed in IMAGE was 2**23 -1,
and the maximum block size was 5120 bytes. The total file size with these, turns out to be
42,949,667,840 bytes (approximately 40 GB). A quick solution has been provided in
IMAGE version C.07.10 released with MPE/iX version 6.0, wherein the limit of 40 GB has
been extended to 80 GB. This was done by increasing the maximum number of blocks to
2**24 -1.
The current enhancement lifts the 80 GB restriction by changing the way that records
are addressed internally.
Entryname vs Entrynumber formats
To understand the solution in detail, a brief discussion of the terms entryname and
entrynumber is worthwhile.
As discussed above, in IMAGE, records (entries) were referenced through a 24 bit block
number and an 8 bit offset (slot number) within the block. This format is referred to as
'entryname' format. For example, the 101st record in a dataset, with a blocking factor of 8,
will be referenced as hex c05, represented as the 24-bit blocknumber of 12 (hex c) and a