TurboIMAGE/XL Database Management System Reference Manual (30391-90012)

Chapter 10 487
Internal Structures and Techniques
Large Size Datasets in TurboIMAGE/iX
Large Size Datasets in TurboIMAGE/iX
Dataset Size Limitations
The IMAGE database management system has been around for more than two decades.
When HP first introduced the product, the maximum size of a dataset was limited to the
MPE file size. However, when users felt the need for much larger dataset sizes, the
'JUMBO' feature was introduced. With this feature, a user can create a dataset with a size
greater than the MPE file size of 4GB. Even though the dataset physically spans across
multiple MPE files called chunk control files and chunkset files, the JUMBO feature allows
them to be seen as a single logical file. Even with this feature, users could only define a
dataset up to 40 GB in size, because of the limit of block numbers, whereas the JUMBO
feature per se doesn't have this restriction. Earlier, IMAGE used to transfer data in blocks,
which necessitated addressing of a record through a block number and an offset within the
block. The maximum number of blocks allowed to be addressed in IMAGE was 2**23 -1,
and the maximum block size was 5120 bytes. The total file size with these, turns out to be
42,949,667,840 bytes (approximately 40 GB). A quick solution has been provided in
IMAGE version C.07.10 released with MPE/iX version 6.0, wherein the limit of 40 GB has
been extended to 80 GB. This was done by increasing the maximum number of blocks to
2**24 -1. The current enhancement lifts the 80 GB restriction by changing the way that
records are addressed internally.
Entryname vs Entrynumber formats
To understand the solution in detail, a brief discussion of the terms entryname and
entrynumber is worthwhile.
As discussed in the beginning of this chapter, a pointer used to point to a record. In
IMAGE, records (entries) were referenced through a 24 bit block number and an 8 bit
offset (slot number) within the block. This format is referred to as 'entryname' format. For
example, the 101st record in a dataset, with a blocking factor of 8, will be referenced as hex
c05, represented as the 24-bit blocknumber of 12 (hex c) and a slot within the block of 5.
Now we can use a full 32-bit number to refer a record number. We call this 'entrynumber'
format. So, the 101st record in a dataset will be represented as just 101, irrespective of the
blocking factor used.
All existing databases prior to this release use entryname format. With this solution, a
database can be created either in the entrynumber format or the entryname format.
However, if large datasets greater than 80 GB are to be created, it can be done only if the
database is created in entrynumber format.
Changes in User Interfaces
DBSCHEMA
To protect existing database environments and scripts, the new DBSCHEMA (version
C.09.00 of IMAGE) will create databases in entryname format by default. However, if a