HP-UX 11i v3 Memory Management Subsystem

ManualsBrandsHP ManualsSoftwareHP-UX 11i v3 Enterprise OE

The HP-UX 11i v3 Memory

Management Subsystem

Technical white paper

Table of contents

Introduction ......................................................................................................................................... 3

The memory management subsystem .................................................................................................. 3

Tools to examine the memory management subsystem .......................................................................... 3

Top ................................................................................................................................................ 4

Swapinfo ........................................................................................................................................ 5

HP GlancePlus ................................................................................................................................. 5

Sar ................................................................................................................................................. 8

Locinfo to illustrate pstat_getlocality() call ............................................................................................ 9

Ipcs .............................................................................................................................................. 10

Interpreting the output from the tools ................................................................................................. 11

Does my system have enough memory? ........................................................................................ 11

Does my system have enough swap space? ................................................................................... 11

The example system .................................................................................................................... 11

Configuration and tuning opportunities ................................................................................................. 14

Local memory versus interleaved memory .......................................................................................... 14

Ejectable (floating) memory ............................................................................................................. 14

System base page size ................................................................................................................... 15

Configuring swap space ................................................................................................................. 15

Sizing the filecache ........................................................................................................................ 15

Tuning JFS (VxFS) ........................................................................................................................... 16

Considerations for application developers ............................................................................................ 17

Addressing model .......................................................................................................................... 17

Local memory ................................................................................................................................ 17

Large Virtual Pages ........................................................................................................................ 18

For more information .......................................................................................................................... 18

Appendix 1 -- Variations with OS release ............................................................................................. 19

Memory management enhancements since the initial HP-UX 11i v3 release .......................................... 19

Differences between HP-UX 11i v2 and HP-UX 11i v3 ........................................................................ 19

Unified Filecache ........................................................................................................................ 19

Unconditional use of pseudo-swap ............................................................................................... 19

Appendix 2 -- Variations with platform architecture ................................................................................ 19

Appendix 3 – Operation of the pager (vhand) ...................................................................................... 20

Summary of content (26 pages)

PAGE 1
The HP-UX 11i v3 Memory Management Subsystem Technical white paper Table of contents Introduction ......................................................................................................................................... 3 The memory management subsystem .................................................................................................. 3 Tools to examine the memory management subsystem .......................................................................... 3 Top ..........
PAGE 2
Appendix 4 – Management of swap space .......................................................................................... 21 Pseudo-Swap Space ....................................................................................................................... 21 Physical Swap Space ..................................................................................................................... 22 Device Swap Space.................................................................................
PAGE 3
Introduction The purpose of this paper is to explain the aspects of the HP-UX 11i v3 memory management subsystem relevant to obtaining the best possible computing experience. The first section “Tools to ”, describes the tools and utilities that can be used to gain insight into the operation of the subsystem. The second section, “Configuration and tuning opportunities”, explores the configuration and tuning opportunities available to the system administrator.
PAGE 4
Top The top(1) command gives statistics about the top processes on the system, as well as system memory size and utilization. For the example below, the command was invoked as top -h, to hide the information about individual CPUs. System: vmkona1 Load averages: 0.29, 0.31, 0.26 353 processes: 275 sleeping, 78 running Cpu states: (avg) LOAD USER NICE SYS IDLE BLOCK 0.29 9.3% 0.0% 7.8% 82.9% 0.0% Wed Jan SWAIT 0.0% INTR 0.0% 6 10:42:36 2010 SSYS 0.
PAGE 5
The total virtual size is always at least as large as the resident size, because the former accounts for all memory that has been defined by the process, even that which has not been accessed recently, or ever, and so is not in core. Swapinfo The swapinfo(1M) command reveals information about the size and utilization of swap space. For the example below, the command was invoked as swapinfo -mt to display values in megabytes instead of kilobytes, and to provide a row showing the totals.
PAGE 6
screen of interest. To examine the memory characteristics of a specific process, its process identifier (PID) must be specified. The following screen is for the Memory Report, selected by the keystroke m: Glance C.05.00.
PAGE 7
The following screen is for Swap Space, selected by the keystroke w: Glance C.05.00.
PAGE 8
The following screen is for the Process Memory Regions, chosen by the keystroke M and then the PID 8121 to examine an sqlplus process: Glance C.05.00.
PAGE 9
The example for swapping and switching activity was obtained by invoking the command as sar -w 60 1: # sar -w 60 1 HP-UX vmkona1 B.11.31 U ia64 01/06/10 10:51:29 swpin/s bswin/s swpot/s bswot/s pswch/s 10:52:29 0.00 0.0 0.00 0.0 37547 In this example, the 0.0 in the bswin/s column indicates that there are no 512-byte units per second being transferred from swap space into memory and the 0.
PAGE 10
Cell 3 was designated as a floating cell to allow for the possibility of online deactivation. The kernel never places certain data structures in floating memory, but the floating memory is well utilized by applications. Interleaved memory is used for objects that are expected to be referenced globally, such as program text and shared kernel data. The example system needs just over 16 GB of interleaved memory; configuring it with less than that amount would lead to suboptimal performance.
PAGE 11
Interpreting the output from the tools Does my system have enough memory? The tools described above can be used to determine if a system has enough memory for the workload it is running. The simplest way is to check the memory utilization reported by glance at the top of each screen. A good target value for memory utilization is around 85% to 90%. A value lower than 50% indicates that the system is overprovisioned and you're not getting the full value from all the memory that you paid for.
PAGE 12
The output from the top command shows a similar processor load to vmkona1, but there is much less physical memory in the system and much less memory is free: System: vmkona2 Load averages: 0.91, 0.94, 0.94 313 processes: 231 sleeping, 82 running Cpu states: (avg) LOAD USER NICE SYS IDLE BLOCK 0.91 86.3% 0.0% 1.0% 12.7% 0.0% Fri Feb 26 13:52:27 2010 SWAIT 0.0% INTR 0.0% SSYS 0.
PAGE 13
The output from glance shows that memory utilization is 99%. Glance C.04.60.
PAGE 14
Configuration and tuning opportunities HP-UX aims to be simple to use. Its behavior straight out of the box should be completely satisfactory for most uses, with no need to perform manual configuration or tuning. However, both the size of the host servers and the workloads imposed upon them vary widely across the range of scenarios in which HP-UX is deployed. For example, system memory size can be as small as 1.5 GB or as large as 8 TB.
PAGE 15
The HP-UX Virtual Partitions Administrator's Guide contains more detailed recommendations for configuring base and floating memory in vPars scenarios. System base page size A base page is the unit of physical memory management in the HP-UX kernel. It is the smallest block of physical memory that can be allocated for storing data and code and it is the smallest unit of memory protection.
PAGE 16
filecache size of 50%. Two thirds of the systems tuned the filecache_max parameter downward into the range of 5% to 10% of physical memory. A small minority of systems sized the filecache to be more than half of memory. Tuning JFS (VxFS) The Journaled File system, also known as HP OnlineJFS, JFS, or VxFS, is often a major consumer of system memory, in particular for its inode cache. The amount of memory used for the JFS cache is controlled by tunable parameters vx_ninode and vxfs_ifree_timelag.
PAGE 17
Tuning ninode does not affect the JFS inode cache, which is maintained separately from the HFS inode cache. If your only HFS file system is /stand, then ninode can usually be tuned to a low value, for example, 400. By default, the JFS inode cache is dynamic in size: it grows and shrinks as needed. However, since the inodes are freed to the kernel memory allocator’s free object chain, the memory may not be available for use for other reasons (except for other same-sized memory allocations).
PAGE 18
Application programs can use the memory hints, possibly in conjunction with the processor binding facility offered through the mpctl(2) interface, to attempt to gain favorable processor to memory alignment on NUMA platforms.
PAGE 19
Appendix 1 -- Variations with OS release Memory management enhancements since the initial HP-UX 11i v3 release • Floating cells were first supported in the September 2007 Update. • The Kernel Access Infrastructure feature to allow the mapping of user virtual addresses into the kernel was introduced in the March 2008 Update. • The numa_policy kernel tunable parameter to control the default placement of application memory objects was introduced in the March 2008 Update.
PAGE 20
Appendix 3 – Operation of the pager (vhand) Examination of the output from the top command illustrates two key points: • The total amount of virtual memory used by all of the processes running on the system can greatly exceed the total amount of physical memory available to the system. • Any given process runs happily with only a fraction of its total virtual address space loaded into memory.
PAGE 21
When the age hand arrives at a pregion, it ages some constant fraction of pages before moving tothe next pregion (by default 1/16 of the pregion's total pages). The p_agescan tag enables the age hand to move to the location within a pregion where it left off during its previous pass, while the p_ageremain charts how many pages must be aged to fill the 1/16 quota before moving on to the next pregion.
PAGE 22
For systems such as factory-floor controllers, which perform best when the entire application is resident in memory, pseudo-swap space can be used to enhance performance: you can either lock the application in memory or ensure that all processes don't use more than 7/8 of system memory. When the number of processes created approaches capacity, the system might exhibit thrashing and a decrease in system response time.
PAGE 23
Allocation of Pseudo-Swap Space Approximately 7/8 of available system memory is available as pseudo-swap space. Pseudo-swap is tracked in the global pseudo-swap reservation counters swapmem_max (enabled pseudo-swap) and swapmem_cnt (currently available pseudo-swap). If physical swap space is exhausted and no additional file system swap can be acquired, pseudo-swap space is reserved for the process by decrementing swapmem_cnt. For example, on a 4 GB system, swapmem_max and swapmem_cnt track approximately 3.
PAGE 24
Appendix 5 – source code for locinfo program The source code for the locinfo program follows, in the hope that it will prove useful in analyzing the memory distribution on NUMA servers. The code is based on the example that appears in the pstat_getlocality(2) manpage. /* * This program returns system-wide and per-process memory locality information. * To compile the 32-bit version, use -D_PSTAT64. * The 64-bit version does not need any special compiler flags.
PAGE 25
/* Get a maximum of BURST pst_locality structures */ count = pstat_getlocality ( pstl, sizeof(struct pst_locality), BURST, idx ); while ( count > 0 ) { for ( i=0 ; i
PAGE 26
printf ( "%6d%10s%10s%10s%10s\n", i, total_str, shared_str, private_str, weighted_str ); i++; count = pstat_getproclocality ( &ppl, sizeof(struct pst_proc_locality), pid, i ); } if ( count < 0 ) { if ( errno == ESRCH ) { fprintf ( stderr, "Process %d not found\n", pid ); exit(1); } perror ( "pstat_getproclocality" ); exit(1); } if ( i == 1 ) { /* Don't print totals if there's one locality */ printf ( "\n" ); return; } pages_to_str pages_to_str pages_to_str pages_to_str ( ( ( ( total, total_str ); shared,