ccNUMA Overview

Impact on applications
Many applications will not be adversely affected by ccNUMA, but some key types will. Applications that need to
be aware of ccNUMA include:
• Pthreads applications
• SystemV shared memory applications
• Message passing applications. MPI is ccNUMA-aware with version 1.8.4; some applications will need to be
relinked to take advantage of optimizations in MPI for cell local memory
• Applications that use anonymous mmap() to allocate memory
Applications generally not adversely affected include:
• Single-threaded applications
• Parallel applications run in single CPU mode—both shared memory and distributed or message passing
versions
• OpenMP applications in general do not need to be changed, but they may need to be relinked if they are built
with archive versions of libomp. If the application is built with the shared version of libomp, this problem is
avoided.
Memory topology
HP-UX 11i v2 provides several interfaces to determine the memory configuration of a system or a process.
Pstat_getlocality() returns system-wide information, while pstat_getproclocality() returns per-process information.
These calls return a mapping of the memory allocated by an application (or all applications) to the LDOMs of the
system. This will include the interleave region, which is a locality but not a Locality Domain.
For pstat_getlocality(), additional information is provided that describes the processor configuration for each
LDOM.
An application could use pstat_getproclocality() to make sure that memory is actually allocated in the same LDOM
that requested it.
These pstat() utilities should be used in conjunction with the mpctl() calls (described next) to get a complete
topology for the application’s execution domain.
For systems that do not incorporate the cellular architecture of the HP Integrity servers, these interfaces will behave
as if the system is one LDOM, which, in fact, it is.
Process management
Process management and scheduling are enhanced in HP-UX 11i v2 release for ccNUMA systems. They provide
default optimal behavior in application placement and scheduling on HP Integrity servers. They also provide
explicit programming interfaces for applications that want to further exploit the ccNUMA characteristics of the
underlying system. Specifically, the applications can manage their launch policies as well as processor and
locality domain binding.
Launch policies
Launch policies are used by the process manager to best assign where a process or thread starts to execute and
what the physical relationship of the processes or threads are to each other. Every process and thread has an
assigned launch policy. The launch policy for a thread need not match the launch policy for its process. There are
three ways to set launch policies:
• mpctl(2)
• pthread_launch_policy_np(3T)
• mpsched(1)
HP-UX 11i v2 provides several thread-launch policies and process-launch policies for distributing work in various
ways among a system’s locality domains. These launch policies determine how HP-UX selects the locality domains
into which it launches threads or processes.
9