Right after the error was thrown, we checked the available memory in the available memory graph in the ColdBox cache debugging and we had around 170MB free. We have not set a minimum heap size in our JVM args, so it grows throughout the day. The maximum heap is 1 gig. We are guessing that we got the error because the used heap was very close to the currently allocated (total) heap amount. We spent some time this morning looking at cacheManager thresholdChecks and saw that you are comparing freeMemory to totalMemory. We’re a bit confused, we were under the impression that maxMemory would be the method to consider when determining if we are about to run the server out of memory, not totalMemory which is just the currently allocated memory. Won’t the JVM automatically increment the heap upwards when we get close to totalMemory usage?
Consider this scenario:
The Maximum heap size is 1 gig (MaxMemory)
There are 300 MB allocated to the heap (TotalMemory)
ColdFusion is currently using 295MB (TotalMemory - FreeMemory)
That would show 1.6% Free.
Imagine if a request came through at that point in time (before the JVM allocates more to TotalMemory) it would appear as if you were almost out of memory, but in reality there are 700 MB remaining available for allocation. Would this not cause the issue we experienced?
I see your point totally. The decision to go with totalMemory was because that is what has been allocated for object consumptions. “the total amount of memory currently available for current and future objects, measured in bytes.” However, the point to note also, is that the MaxMemory is the max allocated for the JVM, so potentially, the JVM will resize the heap automatically, but it should also increase the TotalMemory also. If there are delays in seeing the response of the TotalMemory increase, then you might see this.
Anyways, I will investigate this further and let you know if it would be best to do the comparison against the MaxMemory instead of the totalMemory.
Does anybody have anymore suggestions on this?
I love doing work on the cache, so please give your input and suggestions of how to make it even better.
The one thing I was thinking was that maybe memory is a red herring.
How much memory can a single instance of this interceptor possibly
take up? Unless it's got a gazillion-element string array inside of
it, I don't see it consuming more than a few K, and under "normal"
conditions, which it seems like Christine is describing, I can't
imagine how this thing would be "overflowing" the memory you've
allowed.
Christine, are you on CF8? If so, can you turn on Debugger and step
through the code where it's trying to put this into the cache? I'd
think that this would be a pretty fast way of seeing why ColdBox
thinks it can't store it in the cache.
another way to rule out memory would be to just add stuff manually to
the cache in code. just pick some eventhandler and put a few calls to
cache.add( createObject("component","SomeCFC") ) (or whatever the call
is... can't remember right now) inside of a 100-element loop or
something. if new objects can go into the cache when you
programmatically put them there, this would *appear* to rule out
memory being the problem.
I think you are right Christine, I have been doing some tests and yes, the memory heap won’t even increase because the threshold is stopping it, even though the heap still has the potential to grow. I will update this to look at the MaxMemory and not the Total Memory.