I ran into an odd issue today on one of my colocated servers. The server is a Core 2 Duo with 8 GB of RAM, running a thinned down version of CentOS. The host is configured to fit all virtual machine memory into RAM, and the host s able to reserve 6400 MB of RAM.
There are 3 VMs on there; VM1 has 3072MB of RAM allocated to it, VM2 and VM3 each had 1536MB allocated. I was adding some memory-intensive software to VM3 so I upped it's RAM allocation to an even 2GB (2048MB).
Shortly after that change the load on the host jumped from ~0.30 to ~1.8. A handful of services on VM3 stopped starting and when I opened the Task Manager (it's a Windows server) the task manager process was pegging the CPU. If you search for "taskmgr.exe 100% cpu" or "taskmgr.exe peg cpu" you'll find that most of the cases are spyware-related. In this case it turns out that the host was starved for memory. 'top' on the host showed that I wasn't swapping any RAM to speak of, but I only had ~256MB free.
Tasks: 89 total, 1 running, 88 sleeping, 0 stopped, 0 zombie Cpu(s): 1.8%us, 58.5%sy, 0.0%ni, 39.5%id, 0.2%wa, 0.0%hi, 0.0%si, 0.0%st Mem: 7457468k total, 7194396k used, 263072k free, 24652k buffers Swap: 2048184k total, 120k used, 2048064k free, 6716580k cached
I reduced VM3's RAM back down to 1.5GB and magically all of my problems went away.
Tasks: 95 total, 1 running, 94 sleeping, 0 stopped, 0 zombie Cpu(s): 2.0%us, 5.1%sy, 0.0%ni, 92.5%id, 0.3%wa, 0.0%hi, 0.0%si, 0.0%st Mem: 7457468k total, 7194588k used, 262880k free, 22956k buffers Swap: 2048184k total, 124k used, 2048060k free, 6687792k cached
Tags: CentOS, Server 2008, VMware, Windows
December 26th, 2009 at 7:55 am
Thank you. I've spent haft a day to find out while the last virtual machine runs so slow.
I have a host machine with 8GB RAM running 4 virtual machines. They take around 7.5GB memory and the last powered on machine runs very slow. Taskmgr.exe ussualy takes 30 to 50 percent of CPU usage.
After reducing memory for the last machine from 2GB to 1.5GB, it runs fast again.
More info: the host machine runs Windows 2008 x64, two of the virtual machines run Windows 2003 x86, and two others run Windows 2008 x64.