IBM AIX 7.1, memory_affinity, memplace_shm_anonymous and Oracle SGA

I'm not just an MSSQL.Sasquatch :)

Below is one of my favorite series on Oracle memory behavior on IBM Power systems running AIX.
http://intermediatesql.com/aix/how-oracle-uses-memory-on-aix-part-1-processes/
http://intermediatesql.com/aix/how-oracle-uses-memory-on-aix-part-2-sga/
http://intermediatesql.com/aix/how-oracle-uses-memory-on-aix-part-3-locking-sga/

In these posts, as well as other sources, shmget() is documented as the shared memory allocation function for creating the Oracle SGA.

There's a whole world of fun memory management information waiting at:
vmo -Fa

(The capital F is necessary to see the restricted parameters.)


I believe vmo parameter memplace_shm_anonymous will influence memory allocation for the Oracle SGA.
http://pic.dhe.ibm.com/infocenter/aix/v7r1/index.jsp?topic=%2Fcom.ibm.aix.prftungd%2Fdoc%2Fprftungd%2Fmem_place_vmo.htm

The documentation indicates that, like the other memplace parameters, memplace_shm_anonymous "can either have a value of 1, signifying the first touch scheduling policy, or 2, signifying the round-robin scheduling policy."

Further, the documentation indicates that the default for memplace_shm_anonymous is 2 for round-robin.

But its never really that easy, is it?  :)

All of the Power 7 servers with AIX 7.1 that I've seen have had this parameter set to 0.

This document explains that the value of zero means auto-affinitized, "where the system decides the best placement for the memory."
http://www-01.ibm.com/support/docview.wss?uid=isg1IZ74067

I wish there was more information about how the system decides.  I'm working with and near a LOT of IBM Oracle where "the system has decided" that a significant portion of the Oracle SGA should be in paging space, even though there are quite a few free memory pages.

Disabling 64k pages helps a bit.  On one of the systems memory_affinity was disabled, and that stopped the paging.

I've read a lot about memory_affinity.  Too much maybe.  But I still haven't been able to figure out the most predictable method of memory management on these servers.  Based on the documentation round-up I'll share in a future post, and experiences across several systems, I think I'll be recommending the following for the analytics systems I work with: disable 64k pages, disable memory_affinity, disable enhanced_memory_affinity.  4k only pages while optimizing SGA size and other memory config.  If 16mb pages are desired, only after optimal sizes have been determined with 4k-page-only system.

A lot of sysadmins have just acclimated to Oracle SGA in paging space while there is free server memory.  That's too bad.

But, like me, they may just have run out of solid answers for memory management.