Hub/Switch Installation Guide

Chapter 7 HPSS User Interface Configuration
442 September 2002 HPSS Installation Guide
Release 4.5, Revision 2
Meetingswith TransarcandIBM Austinhave takenplacetodiscusstheissue.The aboverestriction
may be fixed in the future.
7.6.2.7.2 Migration and Purge Algorithms
Currently,theHDMreadsthroughalltheanodesinanaggregatetodeterminemigrationandpurge
candidates. Using empirical data wedeterminedthattheHDMreads approximately70 entriesper
second (this is disk hardware dependent).The optimum number of objects stored on an aggregate
for migration purposes can be determined using the following parameters:
The amount of time it takes to read through all objects on the aggregate.
The frequency of migration runs (purge runs when space is needed).
The amount of time it takes to move files from DFS to HPSS.
Ifmigration runsonce perday,anddatacanbemovedin4hours, thecurrentsuggestion isto allow
approximately500,000 objectson anaggregate.Thismeansthatmigration willtake approximately
6hours(7142seconds=~2hoursforheaderrumble+4hoursfordatamovement).Obviously,these
numbers vary dramatically and are dependent on file size and the number of files that need to be
migrated.Remember, onlynewfilesand fileswhose datahasbeenalteredneed tobemigrated into
HPSS. Filesthat have already been migrated intoHPSS, but have been read needto update XDSM
attributes, but this can be done quickly (approximately 60 per second).
Episode XDSM(DMAPI) changes have been discussed between HPSS and Transarc. It is believed
that with thesechanges, the Episode XDSM(DMAPI)rate to readall objects in an aggregate would
be increased to approximately 400 per second. This would greatly reduce the time taken to
determine migration and purge candidates. Then the suggested limit for the number of objects on
an aggregate could be raised to 2 million objects.
7.6.2.7.3 Fileset Quotas
The Episode fileset quota is currently stored in a 32-bit field (actually only 2^31 – 1 can be stored).
The quota represents 1K allocations. So, the current maximum quota per fileset is about 2TB. The
Episode quota reflects the amount of datastored inthe filesetplus the amount ofanode space.The
quotacan notbe increasedbeyond2TB.Whenthequota limitis reachedtheonly wayto storemore
files is to remove old files.
When theHDMpurges data fromEpisode thequota isnot affected.The quotareflects all dataever
stored through the DFS interface. If a file is written to a mirrored fileset through the HPSS
interfaces, the quota is not altered (except for anode space) until the data is actually read through
the DFSinterface. So, it is possible to writemore than 2TB toa mirrored fileset, but onlyif the data
is written through a non-DFS interface and never read through the DFS interface (This may occur
iflargedatadumpsaremadedirectlytoHPSS,buttheclientwishestouseDFStomanagethename
space).
There are no current plans to change the Episode fileset quota implementation.