HP-UX Reference (11i v2 07/12) - 7 Device (Special) Files, 9 General Information, Index (vol 10)

g
gang_sched(7) gang_sched(7)
NAME
gang_sched - Gang Scheduler
DESCRIPTION
The gang scheduler permits a set of MPI (Message Passing Interface) processes, or multiple threads from a
single process, to be scheduled concurrently as a group.
Gang scheduling is enabled and disabled by setting the
MP_GANG environment variable to ON or OFF.
The gang scheduling feature can significantly improve parallel application performance in loaded timeshare
environments that are oversubscribed. Oversubscription occurs when the total number of runnable parallel
threads, runnable MPI processes, and other runnable processes exceeds the number of processors in the
system.
Gang scheduling also permits low-latency interactions among threads in shared-memory parallel applica-
tions.
Only applications using the HP-UX V11.0 MPI or pthread libraries can be gang scheduled. Because HP
compiler parallelism is primarily built on the pthread library, programs compiled with HP compilers can
benefit from gang scheduling.
INTERFACE
The HP-UX gang scheduler is enabled and disabled using an environment variable. The variable is defined
as:
MP_GANG [ON | OFF]
Setting MP_GANG to ON enables gang scheduling and setting it to OFF disables it. If MP_GANG
is not set,
or if it is set to an undefined value, no action is taken.
Gang scheduling is a process attribute that is inherited by child processes created by
fork (see fork(2)).
The state of gang scheduling for a process can change only following a call to
exec
(see exec(2)).
BEHAVIOR
After the
MP_GANG environment variable is set to ON, any MPI or pthread application to execute and find
this variable will enable gang scheduling for that process.
Only the pthread and MPI libraries query the MP_GANG variable--the operating system does not.
Gang scheduling is an inherited process attribute. When a process with gang scheduling enabled creates a
child process, the following occurs:
• The child process inherits the gang scheduling attribute.
• A new gang is formed for the child process. The child does not become part of its parent’s gang.
The gang scheduler is engaged only when a gang consists of multiple threads. For a pthread application,
this is when a second thread is created. For an MPI application, it is when a second process is added.
As a process creates threads, the new threads are added to the process’s gang if gang scheduling is enabled
for the process. However, once the size of a gang equals the number of processors in the system, the follow-
ing occurs:
• New threads or processes are not added to the gang.
• The gang remains intact and continues to be gang scheduled.
• The spill-over threads are scheduled with the regular timeshare policies.
• If threads in the gang exit (thus making room available), the spill-over threads are not added into
the gang. However, newly created threads are added into the gang when room is available.
MPI processes are allocated statically at the beginning of execution. When MP_GANG is set to ON, all
processes in an MPI application are made part of the same gang.
Thread and process priorities for gangs are managed identically to timeshare policy. The timeshare priority
scheduler determines when to schedule a gang and adheres to the timeshare policies.
Although it is likely that scheduling a gang will preempt one or more higher priority timeshare threads,
over the long run the gang scheduler policy is generally fair. All threads in a gang will have been highest
priority by the time a gang is scheduled. Because all threads in a gang must execute concurrently, some
threads do not execute when they are highest priority (the threads must wait until all other threads have
HP-UX 11i Version 2: December 2007 Update − 1 − Hewlett-Packard Company 53