I've already submitted this as a bug report, but I thought I'd mention it here as well.

Houdini Indie 14.0.201.13
Windows 8.1
2 x 20 3.0 GHz Xeons
64 GB of RAM
GTX 580 3 GB

I wanted to test out the whitewater in a small scale scene before trying it out in my larger project, so I created this very basic project file. I cached the base fluid sim using the DOP IO in “Surface_Source_Fluid”, then used the shelf tool to create the whitewater setup, making use of my pre-cached fluid as a base.

With each frame of whitewater cached, my memory use skyrocketed - filling up all 64 GB within the first 10 frames. I got a warning that I was using too much memory, Houdini crashed, and windows automatically logged off or restarted; not sure which. I tried twice more, with same results.

I've been having some occasional RAM issues in other programs over the last couple months (Cinema 4D) that have been very hard to pin down in terms of whether they were software related or hardware related - I even reformatted my workstation twice in the process - so there's a chance that this might be related, though I can't be sure.

If there's any more information I can share that would be helpful, let me know.

Thanks!
Luke

Just wondering, have you looked at the Performance Monitor to see which node is using the most memory?

The “whitewatersolver1” and the “whitewater_emitter” container nodes split the RAM consumption; I've attached a screenshot of each along with their contents.

If someone wants to see a more detailed screenshot that dives farther into the node network, let me know.

Bumping this thread, as a solution hasn't presented itself. It seems to be more on my system end of things rather than the Houdini side. Here's an updated version of the same scene, in which I was able to optimize things enough to slow the memory leak a little bit, but not a whole lot (still fills up most of the 64 GB within 10 frames when trying to cache the whitewater).

Hi Luke,

Sorry, just looked at this (somehow the did bug not show up in the bug database).

When debugging whitewater problems, the first place to look is always Whitewater Source, so you can see where you're actually emitting whitewater. In your case if you look at that node, you'll see the entire FLIP sim is showing up there as emission points!

This is because you're not importing and writing out your surface and vel fields to disk along with your FLIP particles. When Whitewater Source calculates acceleration, it compares the current points to the previous frame's velocity field. If the vel field is missing, it's always comparing the particles' velocity to 0, so there's always acceleration (admittedly we should do a much better job of erroring out if the vel or surface fields are not there.) Anyway, in the current file you're sending in your entire FLIP sim as emitter particles (over 12M points), and it's replicating whitewater points at each particle; not surprisingly you're quickly running out of memory. (Houdini is also particularly memory-hungry under Windows, unfortunately.)

However, even if you fix this, you've got your initial particles situated a fair bit above the bottom of fluid tank for the FLIP sim, so they immediately fall down due to gravity and make a big splash, adding more acceleration. You need to make sure your Volume Limits align with the edges of your initial particle box, or just use the FLIP tank shelf tool.

With those issues fixed your whitewater sim should no longer run out of memory. However for some reason the .hip file preferences have Unit Length 10M = 1M (Preferences | Hip File Options menu item). It appears that was set before you created the whitewater sim but after the FLIP sim, so those two sims are operating at different scales and your gravity, buoyancy, foam depth, etc. on the whitewater sim are all in the wrong units and look strange.

Finally you might consider animating your ship at the Object level and turning off Use Deforming Geometry. Not a really big deal, but it would avoid re-calculating the collision SDF each timestep.

I've attached a file with a few of these fixes that yields a reasonable whitewater sim.

Edit: sorry, forgot one more thing: you're also visualizing the different whitewater particles by type, which is fine, but internally actually creates a copy of all the points with colors attached. So that makes the memory use worse if you're displaying the points while simulating.

For big whitewater sims you're better off writing straight to disk, then if you want to visualize using the various point groups for foam, spray, etc. and coloring them in SOPs for visualization.

THANK YOU. So much. Initial tests on my end are much improved. A couple comments/questions:

Sorry, just looked at this (somehow the did bug not show up in the bug database).

Ah, shoot. I had submitted this in to Side Effects support rather than the bug department; when their systems didn't show the same RAM accumulation, I assumed it was purely on my end (and therefore not a bug).

When debugging whitewater problems, the first place to look is always Whitewater Source, so you can see where you're actually emitting whitewater. In your case if you look at that node, you'll see the entire FLIP sim is showing up there as emission points!

Yeah, I wasn't sure about that - but this was literally my very first test with whitewater, I had changed very little of the defaults - so I wasn't sure what exactly was normal there.

This is because you're not importing and writing out your surface and vel fields to disk along with your FLIP particles. When Whitewater Source calculates acceleration, it compares the current points to the previous frame's velocity field. If the vel field is missing, it's always comparing the particles' velocity to 0, so there's always acceleration (admittedly we should do a much better job of erroring out if the vel or surface fields are not there.) Anyway, in the current file you're sending in your entire FLIP sim as emitter particles (over 12M points), and it's replicating whitewater points at each particle; not surprisingly you're quickly running out of memory. (Houdini is also particularly memory-hungry under Windows, unfortunately.)

Ah, this explains a lot. For some reason I had thought that spray/foam calculations only needed the geometry - so I thought “Why I am a caching what I don't really need?” Optimization FAIL on my part, haha.

With those issues fixed your whitewater sim should no longer run out of memory. However for some reason the .hip file preferences have Unit Length 10M = 1M (Preferences | Hip File Options menu item). It appears that was set before you created the whitewater sim but after the FLIP sim, so those two sims are operating at different scales and your gravity, buoyancy, foam depth, etc. on the whitewater sim are all in the wrong units and look strange.

This is very interesting, as I had previously thought that that particular HIP file option was only in effect at sim time, rather than at scene setup time. How exactly did you go about resolving this issue? Did you simply delete the whitewater setup & redo the shelf tool with the correct HIP file settings - or did you manually go in and fix the gravity, buoyancy, etc?

Finally you might consider animating your ship at the Object level and turning off Use Deforming Geometry. Not a really big deal, but it would avoid re-calculating the collision SDF each timestep.

In terms of “correct Houdini usage” isn't it preferable to perform PSR adjustments in a transform node rather than at the object node level? I've heard differing views on this, and I'm guessing that it's really just a case of “different situations call for different settings”.

Thanks again!
Luke

Luke Letellier
Ah, this explains a lot. For some reason I had thought that spray/foam calculations only needed the geometry - so I thought “Why I am a caching what I don't really need?” Optimization FAIL on my part, haha.

Well, strictly speaking you can send just the points into Whitewater Source and have it rebuild those fields for you (see the Sources tab), but that's more for use with external sims. Rebuilding does the exact same thing as FLIP, so better to just save the FLIP output in the first place. Also, the whitewater sim itself requires these fields: it's how it advects particles along with the original FLIP sim, and figures out depth of particles for things like particle classification, buoyancy, etc.

Luke Letellier
This is very interesting, as I had previously thought that that particular HIP file option was only in effect at sim time, rather than at scene setup time. How exactly did you go about resolving this issue? Did you simply delete the whitewater setup & redo the shelf tool with the correct HIP file settings - or did you manually go in and fix the gravity, buoyancy, etc?

Right, it applies at node creation time, scaling the various parameters that are marked with units. The assumption is that the units setting is more of a studio- or at least project- wide setting that doesn't change very often. It's often used at studios that also work with Maya and have adopted a centimeter-based unit scale.

To fix it I just used Revert to Factory Defaults on the parameter tab then restored just the Volume Source parameter (and turned on Volumetric Foam).

Luke Letellier
In terms of “correct Houdini usage” isn't it preferable to perform PSR adjustments in a transform node rather than at the object node level? I've heard differing views on this, and I'm guessing that it's really just a case of “different situations call for different settings”.

For non-deforming geometry, you're generally better off at the object-level. In that case DOPs imports the geometry and builds a collision SDF only once, then just applies any updated object-level transforms on each timestep. But in the scheme of a large FLIP sim and smaller collision object like this the cost is not really that great either way.

Thanks for all the info. Very helpful in my learning process!

While those techniques seem to have solve the memory leaks in that particular setup, I'm still experiencing leaks & crashes that I assume to be related to the same core issue on my system.

Essentially, I'm trying to figure out why the SESI Windows workstations
didn't experience the issue at all with the original project files. After testing my hardware and reformatting my system twice - with the problem still persisting - well…it's a bit
strange.

Are there any sorts of diagnostic tools that can be run on a system when things are going quirky, with the tool helping to
pinpoint exactly where conflicts (either in the software itself, or with
the system as a whole) are occurring? Maybe a debug console inside Houdini?

Thanks,
Luke

For Windows, it might be memory fragmentation due to TBB Malloc.

For Windows, it might be memory fragmentation due to TBB Malloc.

Just researched the topic for the first time; it might be the case, but it seems as though those issues were resolved [sidefx.com] about a year ago in R13.

Linux seems to be helping a lot of people out, but I'm just not in a position to jump over in that direction. Managing a workflow between two operating systems is hard enough; I really don't want to throw a third one in if I don't absolutely have to.