HP-UX Reference (11i v3 07/02) - 5 Miscellaneous Topics (vol 9)

n
numa_policy(5) numa_policy(5)
(Tunable Kernel Parameters)
NAME
numa_policy - physical memory allocation policy on cell-based HP-UX servers
VALUES
Default
0
Allowed values
Minimum: 0
Maximum: 2
DESCRIPTION
Large servers are built from one or more component blocks called cells. Each cell has at least 1 CPU and
normally some amount of memory. These cell-based servers use Non-Uniform Memory Access (NUMA)
protocols. This means that an application thread can experience widely variable memory latency depending
on whether it is accessing memory from the same cell upon which it is executing or from a different cell.
HP-UX implements a number of memory allocation features to optimize performance. Users can divide the
memory on the system into two classes of memory: Interleave and Cell-local.
Interleave memory delivers uniform latency. It makes NUMA hardware look like a single bank of uniform
memory, delivering the same average latency for all applications across all CPUs. This is very useful in
spreading the memory load across the entire memory fabric. By default, all the memory in an HP-UX
server is configured as interleaved.
Cell-local memory delivers low latency (high performance) to a process running in the same cell as the
memory it accesses. Conversely, references from a remote cell will have a much higher latency.
Users can configure cell-local memory using the HP-UX command parmodify(1M).
The numa_policy dynamic kernel tunable allows users to define how interleaved and cell-local memory
are utilized. There are three possible values:
numa_policy = 0
This is the default policy. In a machine configured with a mix of cell-local memory and inter-
leave memory, HP-UX makes intelligent decisions on how to use the memory by distinguishing
between private memory and shared memory. Memory allocated to private objects (stack, heap,
private files, etc.) is likely to come from cell-local memory. Memory allocated to shared objects
(shared files, System V shared memory) is allocated from interleave memory unless the user
specifies otherwise using fadvise(2) for files or uses one of the memory options in shmget(2).
numa_policy = 1
This policy directs HP-UX to use cell-local memory for all memory allocations. It will select the
cell from which the locality is to be allocated by first honoring any cell-locality policies specified
in fadvise() or shmget(). If no cell-locality policies were specified, then it allocate all
memory from the local cell. If there is not sufficient memory in the local cell, then memory from
the closest cell is used, and so on and so forth. Even the interleave is used, but only if the cell-
local memory available is too remote. When using this option, care needs to be exercised to
ensure that applications are scheduled on the same cell where the memory has been allocated.
See mpsched(1) for more information.
numa_policy = 2
This policy directs HP-UX to over-ride any locality policies supplied via fadvise() or
shmget() and allocate all memory from the interleave. If there is not sufficient memory in the
interleave, then cell-local memory is used.
Restrictions on Changing
Changes to this tunable take effect on all memory allocated subsequent to the change. Any memory
created before the change will not be affected.
When Should the Value of This Tunable Be Changed to 1?
The tunable value should be set to 1 only if all performance-sensitive applications on the system have a
high-degree of cell locality. That is, if the great majority of references to shared memory are from
processes running on the same cell where the memory has been allocated, then setting numa_policy =
2
will have a positive effect on performance. The value should normally be changed back to 0 after the
memory allocations have been completed. Changing the value to 1 should be done after sufficient memory
HP-UX 11i Version 3: February 2007 − 1 − Hewlett-Packard Company 339