Memory usage is incorrectly reported on the web GUI when using an SCB/SSB VMware appliance (4293017)

Our users reported an issue with high memory usage being reported on an lightly loaded SCB on the System monitor, and the Dashboard page of the web GUI.

Unusually high memory usage can be observed on a virtual SCB (or SSB) appliance, even when the virtual appliance is only lightly loaded, or no traffic passes through it at all.

This memory usage can't be explained with the memory usage of the applications and services on the appliance, because the sum of their memory usage is significantly lower than the usage reported by the web GUI.

In some cases the guest VM may even resort to swapping because of the low amount of free memory available.

Execute the following bash one-liner from the boot firmware to verify the issue:

mount -t debugfs nodev /sys/kernel/debug 2>/dev/null && grep -E "^(target|current):.*pages$" /sys/kernel/debug/vmmemctl ; umount /sys/kernel/debug 2>/dev/null

If the two lines returned by the command contain a non-zero number, then the ballooning driver is active.
The line beginning with "current" will give you the number of memory pages consumed by the driver. (One memory page is 4 KB in size.)

VMware's memory ballooning technology is using the unaccounted memory.

The most likely cause is that the VM-host is heavily loaded, and low on free physical memory. It uses the memory ballooning driver in the guest VMs to reclaim free memory, which can then be allocated to other running VMs.

If this situation occurs frequently, then it is a sign of the VM-host having too small amount of physical memory installed.

In order to diagnose this issue, the following steps may be taken:

• verify the amount of phyisical RAM installed into the VM-host
• inspect the memory usage of the guest VMs running in the VM-host
• check the memory reservation values set up for the individual VM-guests

In order to resolve this issue, the following steps may be taken:

• install more physical RAM in the VM-host
• add more nodes to your ESXi cluster, and
• move some of the guest VMs to another node of the ESXi cluster, which still has free memory available
• do not disable page sharing or the balloon driver
• set up memory limits and memory reservation for your mission-critical guest VMs in order to ensure that their memory will not be reclaimed by the VM-host