Hi,
Why does Houdini need such a huge amount of virtual memory?

On my intel Mac even after rebooting the machine and restarting Houdini it still comes out at a wapping 35gigs. Most other software I use doesnt seem to need more that about 3gigs. Could this cause any problems as find running 9.5.170 and any other software ie After Effects, Shake, Monet after a while Houdini locks up.

Regards

Simon

Mac OSX 10.5.4
2.66 dual core Intel
5 gigs ram

A clean houdini session uses almost exactly the same amount of *virtual* memory(making that part clear) as Firefox 3 and my mail program on Linux(separately, not combined), which in my books rates as pretty good. That number shows in top as ‘367m’, which I could see being interpreted as a third of a gigabyte, despite the documentation shown below for top:

VIRT – Virtual Image (kb)
The total amount of virtual memory used by the task. It includes all code, data and shared libraries plus pages that have been swapped out.


Again, a fresh FF session is using 323m, as is my mail program. I'm not sure I see a problem here, based on the definition of virtual. Actual physical memory being used is around 115m, again the same as a web browser.

The instant you start talking about doing real work, well of course that's another issue. Some things swallow up memory, they need it. Running other major apps requires a degree of monitoring, depending of course on your hardware. Bugs and compositing(which is always a memory bear) aside, I've always found Houdini fairly reasonable with memory.

Cheers,

J.C.

Hi
Are you on a Mac or PC?
Also are you saying that with that amount of virtual memory on a Mac I shouldnt experience any problems. Every time I start a clean session without any other software running I see the same amount being used.

Regards

Simon

I'm Linux. What I see there does indeed look wrong, though. I'm unsure if it's an incorrect number being reported, or something more serious. Certainly if that's a fresh session it shouldn't be that high. I'll let one of the SESI Mac guys jump in…

Cheers,

J.C.

Hi Simon,

It is the same on my Mac. Almost 36 GB on the fresh session.

Kamil.

It could have something to do with Houdini being 64bit (none of the other apps in your application list are). Perhaps the Mac system monitor determines the virtual memory size by just subtracting the lowest address from the highest address.

Most programs are set up so that the heap (allocated memory) grows upwards from a base address, while the stack (temporary data for calling functions & allocating local variables) grows down from a different, higher address. They are placed well apart so that they never overlap (until you run out of memory). For a 64b program, it looks like the stack is placed at an extremely high address, and this may be generating the huge virtual memory “size”.

As JColdrick says, it's the resident portion that's important to keep an key on. This is the amount of actual physical memory that a program is currently using.

I've spent a bit of time tracking this down. I worked on hserver because it is statically linked (that means it doesn't link to our gigantic Houdini framework). I tracked the leap in virtual memory to a system routine. For anyone who wants the gory details, I set a break on the mmap function and the leap in memory comes while the dynamic linker (dyld) is loading system libraries.

On a whim, I fired of a new empty Cocoa project in Xcode and turned on 64-bit and garbage collection and compiled and ran the app. Sure enough, it also jumped up to 34.92GB.

So, it appears to be a combination of 64-bit and garbage collection that causes the scary virtual memory size. It has nothing to do with Houdini as I can recreate the problem with a trivial app that uses none of our code.

That being said, it doesn't seem to cause any issues (other than scaring the socks off of people). The OS will only map in a tiny fraction of that 34GB that actually gets used. So, I wouldn't worry about it. It just seems to be the way Apple has implemented garbage collection.

might be good idea to add this to an OSX FAQ…

Increase the RAM. This will allow the laptop to process more information with the memory in the laptop, instead of using the virtual memory. When you use virtual memory, it causes hard drive usage thus reducing power efficiency. This mostly applies to cases where you need to run programs that require a lot of virtual memory usage, since the more RAM you add, the more energy you will use.

ESXi Backup Software [esxibackup.net]