Hub/Switch Installation Guide

Chapter 2 HPSS Planning
66 September 2002 HPSS Installation Guide
Release 4.5, Revision 2
storage class reachesthe thresholdconfigured in the purgepolicy forthat storageclass. Remember
thatsimply addingmigrationandpurgepoliciesto astorageclasswillcause MPSto beginrunning
againstthestorageclass,but itisalsocriticalthat thehierarchiestowhichthatstorageclassbelongs
be configured with proper migration targets in order for migration and purge to perform as
expected.
The purpose of disk migration is to make one or more copies of data stored in a disk storage class
tolower levelsin thestorage hierarchy.BFS usesa metadataqueue topass migrationrecordsto MPS.
When a disk fileneeds tobe migrated (because it has been created, modified, or undergone aclass
of service change), BFS places a migration record on this queue. During a disk migration run on a
given storage class, MPS uses the records on this queue to identify files which are migration
candidates.Migrationrecordson thisqueue areorderedby storagehierarchy,filefamily, and record
create time, in that order. This ordering determines the order in which files are migrated.
MPS allows disk storage classes to be used atop multiple hierarchies (to avoid fragmenting disk
resources). To avoid unnecessary tape mounts, it is desirable to migrate all of the files in one
hierarchy before moving on to the next. At the beginning of each run MPS selects a starting
hierarchy. This is stored in the MPS checkpoint metadata between runs. The starting hierarchy
alternatestoensure that, whenerrors areencountered orthemigration target isnot100percent, all
hierarchiesareservedequally.For example,if adiskstorageclassisbeingused inthreehierarchies,
1, 2, and 3, successive runs will migrate the hierarchies in the following order: 1-2-3, 3-1-2, 2-3-1, 1-
2-3, etc. A migration run ends when either the migration target is reached or all of the eligible files
in every hierarchy are migrated. Files are ordered by file family for the same reason, although
families are not checkpoints as hierarchies are. Finally, the record create time is simply the time at
which BFS adds the migration record to the queue, and so files in the same storage class, hierarchy,
andfamilytendtomigrateintheorderwhichtheyarewritten(actuallytheorderinwhichthewrite
completes).
When a migration run for a given storage class starts work on a hierarchy, it sets a pointer in the
migration record queue to the first migration record for the given hierarchy and file family.
Followingthis,migrationattemptstobuildlistsof256migrationcandidates.Eachmigrationrecord
read is evaluated against the values in the migration policy. If the file in question is eligible for
migration its migration record is added to the list. If the file is not eligible, it is skipped and it will
not be considered again until the next migration run. When 256 eligible files are found, MPS stops
readingmigrationrecordsanddoestheactualworktomigratethesefiles.Thiscyclecontinuesuntil
either the migration target is reached or all of the migration records for the hierarchy in question
are exhausted.
The purpose of disk purge is to maintain a given amount of free space in a disk storage class by
removing data of which copies exist at lower levels in the hierarchy. BFS uses another metadata
queue to pass purge records to MPS. A purge record is created for any disk file which may be
removedfromagivenlevelinthehierarchy(becauseit hasbeenmigratedorstaged).Duringa disk
purge run on a given storage class, MPS uses the records on this queue to identify files which are
purge candidates. The order in which purge records are sorted may be configured on the purge
policy, and this determines the order in which files are purged. It should be noted that all of the
options except purge record create time require additional metadata updates and can impose extra
overheadonSFS.Also,unpredictablepurgebehaviormaybeobservedifthepurgerecordordering
is changed with existing purgerecordsin the system until these existing records arecleared.Purge
operates strictly ona storage class basis, andmakes no consideration ofhierarchies or filefamilies.
MPSbuildslistsof32 purgerecords,andeachfileis evaluatedforpurgeat thepointwhenitspurge
recordisread.If afileisdeemed tobeineligible,it will notbe consideredagain untilthenextpurgerun.
A purge run ends when either the supply of purge records is exhausted or the purge target is
reached.