Thursday, May 9, 2013
Getting your memory savings back on with ESXi and small memory pages
Many other VCDX's have posted all the technical details about this before, so this isn't revolutionary, but back in the day's of 32 bit OS's their memory was stored in small 4k pages on the ESXi host. TPS was great at comparing and colapsing these and generating huge memory savings, typically about 30%. Now with modern 64 bit OS's, this memory is stored in large 2MB pages by default. The reason for the large pages is some performance enhancement, and i'm sure in a highly CPU/Memory latency sensitive environment there is some benefit. However, in every environmnet I've ever worked in, the bottleneck is qty of memory and disk I/O. TPS does begin to break down the large pages into smaller pages when the system is in the last 6% of memory available, before it starts to swap, this is great, but usually, it's too late to really matter. I recommend disabling the large pages, and therefore having ESXi store it in the smaller blocks. You can force this to happen by changing a property on ESXi on the host under advanced "mem" called Mem.AllocGuestLargePage and change it from 1 = large pages enabled to 0 = large pages disabled. After you reboot the host, your VM's should begin to use the smaller pages and almost immediately begin saving your memory with TPS.