AIX enhanced_memory_affinity

There is more to the memory affinity story, at least on AIX.  In AIX 7.1 and a midstream AIX 6.1 TL, enhanced_memory_affnity was introduced.

Nigel Griffiths is a great source for AIX knowledge.  Ever heard of the nmon utility?  Well... it's "Nigel's monitor" and he is that Nigel. 

This is pretty good stuff for understanding enhanced memory affinity from a developer perspective.

AIXPert Blog > Local, Near & Far Memory
https://www.ibm.com/developerworks/mydeveloperworks/blogs/aixpert/entry/local_near_far_memory_part_1_large_power7_boxes_more_local_memory26?lang=en
https://www.ibm.com/developerworks/mydeveloperworks/blogs/aixpert/entry/local_near_far_memory_part_2_virtual_machine_cpu_memory_lay_out3?lang=en
https://www.ibm.com/developerworks/mydeveloperworks/blogs/aixpert/entry/local_near_far_memory_part_3_scheduling_processes_to_smt_virtual_processors130?lang=en
https://www.ibm.com/developerworks/mydeveloperworks/blogs/aixpert/entry/local_near_far_memory_part_4_aggressive_intelligent_threads46?lang=en
https://www.ibm.com/developerworks/mydeveloperworks/blogs/aixpert/entry/local_nar_far_memory_part_5_low_entitlement_has_a_bad_side_effect4?lang=en
https://www.ibm.com/developerworks/mydeveloperworks/blogs/aixpert/entry/local_nar_far_memory_part_6_too_high_a_virtual_processor_number_has_a_bad_side_effect198?lang=en
https://www.ibm.com/developerworks/mydeveloperworks/blogs/aixpert/entry/local_near_far_memory_part_7_vm_placement_also_needs_ram206?lang=en
https://www.ibm.com/developerworks/mydeveloperworks/blogs/aixpert/entry/local_near_far_memory_part_8_dynamic_lpar_changes_can_mess_up_your_placement149?lang=en
https://www.ibm.com/developerworks/mydeveloperworks/blogs/aixpert/entry/local_near_far_memory_part_9_firmware_updates69?lang=en
https://www.ibm.com/developerworks/mydeveloperworks/blogs/aixpert/entry/local_near_far_part_10_final_of_the_table_by_model5?lang=en
https://www.ibm.com/developerworks/mydeveloperworks/blogs/aixpert/entry/local_near_far_part_11_why_local_far_on_lower_end_machines3?lang=en

But, I'm not satisfied knowing how it works... I want to know how I can bend it to my will :)

Oracle Architecture and Tuning on AIX v2.30
http://www-03.ibm.com/support/techdocs/atsmastr.nsf/WebIndex/WP100883
Section 2.2.2 (AIX free memory) explains the relationship of vmpools, mempools, and framesets. 

Section 2.2.8 (AIX Enhanced Memory Affinity) explains local, near and far memory categories, and provides sample output from 'lssrad -av' and 'mpstat -d'.

Interesting.  But, like memory_affinity, there are signs of possible trouble for enhanced_memory_affinity for large databases.


Power Systems
High performance clustering using the 9125-F2C
Planning and Installation Guide
https://www.ibm.com/developerworks/wikis/download/attachments/162267485/p775_planning_installation_guide.rev1.2.pdf?version=1
page 66 of 130 mentions disabling enhanced_memory_affinity to "quiet the VMM daemon"


Hmmmm...

And, check out the help text for enhanced_memory_attach_limit (which is set to 100% by default, by the way).

http://www-304.ibm.com/support/docview.wss?uid=jpn1J1001970&aid=3
The help text includes the warning 'too much affinitized memory in a vmpool can cause paging and impact overall system performance.'