I have a memory problem with a Linux server (CentOS). It is a VM. Below is the context.
It has 70Gb of RAM. There are 2 tomcat running on it.
One tomcat is given a heap of 13Gb and the other one a heap of 20Gb.
The one with 20Gb has the JVM option -Xss160k set (to try to solve my problem).
These 2 tomcat can be using up to 48Gb of RAM (virt.)
Linux is going to use the 70Gb and there will be about 35Gb of cached memory. The swap is 1Gb and it uses pretty much none of it (8Mb).
Now the problem is that I get the following error from the tomcat every so often:
OutOfMemoryError: unable to create new native thread
If I am not mistaken, this happens if the OS itself runs out of memory and is therefore unable to create new threads.
Now the question is: how is that possible when it has 35Gb of cached memory? Shouldn't it re-allocate it and use it as needed.
Is there maybe some kernel option I can set to avoid this? For example it could use up to 65Gb, cache 30Gb and keep 5Gb free.