Hello,

I am on Linux and Houdini 14.0.247 and trying to do a large scale river simulation with some waterfalls. Just finished the flip simulation with which I was very happy. It has about 105.000.000 Particles. I did some testing with different methods of creating whitewater and found, that the whitewater shelf tool gives you really beautiful results with little efford.

The main issue I have now is the extreme memory consumption of the whitewater solver. I turned of bubbles and mist and set the birth rate from 100 to 20. On my 96 GB machine where the base simulation would use about 50gb of ram (which I also think is quite high) the whitewater which has now “only” 50.000.000 particles killed the whole rest of the ram, 16GB of the swap and then crashed Houdini.

It crashes before I reach my initial state frame. Before the crash the size of the .sim file of the base sim is 17 GB, the size of the .sim file of the whitewater sim only 1/3 of the base sim which is about 5 GB.

After dialing down the birth rate and not seeing any improvement I for the sake of testing set the birth rate to 1. Here it solves through the first time. I get a bit more than 3 mil particles which use another ridicules 32 GB of ram on top of the 50 GB for the flip simulation

I think I am doing something fundamentally wrong here. Maybe someone could have a look at this.
Attached is the Houdini file and a test render. I think it turned out really nice and would love to finish this.


Cheers

I had some memory issues with a whitewater project on a much smaller project (forum link [sidefx.com]); some of the replies might be of help.

Though I think most of my problem was just Windows in general; really wish that could get sorted out.

That's looking really nice.

There are a few things you can do to reduce memory for big whitewater sims. In particular a lot of the memory use comes from loading the FLIP sim into Whitewater Source.

So, some things to try:

• Delete all possible attributes from the original FLIP particles to save both disk space and memory when reloading into Whitewater Source. With something that big, I'd get rid of everything but v. If you need pscale for surfacing, you can re-create it later after load. (Usually I do this on the Compression tab of the DOP I/O SOP, but you don't need to re-sim; just load the existing geo in and run through an Attribute Delete, although the next step might make even that unnecessary in this case…)
  
  
• Cache out the Whitewater Source results from the whitewatersource_cache SOP. This will take extra disk space since you're saving the volumes to disk twice, but then you'll only need to load in a very small subset of the original FLIP particles when running the whitewater sim. If that's too much disk space in general, you can save the original FLIP particles and volumes separately, but that takes more setup. I can elaborate if you don't have the disk space.
  
  
• Turn off Allow Caching for the Whitewater Object, since you won't be playing it back in real-time obviously. This just prevents making any copies of the whitewater object data, which includes the volumes. (You can do the same for the FLIP object under the Creation tab).
  
  
• Maybe copy/paste the static object from the FLIP sim into Whitewater to get terrain collisions, though I'd try it the way you have it set up first.
  
  
• Run your simulations from the command line. It takes a lot of memory to load and display the particles and terrain and such. Rendering from the command line avoids any display-related memory use. For the whitewater sim in this file it would just look like:
  
  $ hbatch newRiverWhitewaterRamTest.hiplc
  hbatch Version 14.0.260 (Compiled on Mar 10 2015)
  WARNING: Entered limited commercial session mode.
  / -> render /obj/sim/rop_geometry2
  
  or
  
  $ hrender -d /obj/sim/rop_geometry2 newRiverWhitewaterRamTest.hiplc
  
  
  
  There are a few additional steps you can take, but they generally require more setup and hopefully these will get you started. Let us know if it helps!

johner
Run your simulations from the command line. It takes a lot of memory to load and display the particles and terrain and such. Rendering from the command line avoids any display-related memory use.

What about running the simulation by clicking the “render” button on a DOP I/O node? How does this compare to the memory usage of the command line setup?

Luke Letellier
What about running the simulation by clicking the “render” button on a DOP I/O node? How does this compare to the memory usage of the command line setup?

Render will just render in the current process on top of all the current memory use. Background Render does launch a separate process and is essentially the same thing as rendering from the command line, so I suppose you could start it that way then quit your main Houdini session to free up memory.

But for something like big overnight sim on a single machine I usually recommend the command-line, though setting up your own local hqueue server is also an option (requiring a bit more setup, of course)

johner
But for something like big overnight sim on a single machine I usually recommend the command-line, though setting up your own local hqueue server is also an option (requiring a bit more setup, of course)

Hqueue sounds like a great option. I just set mine up last week through the help of SESI support, in the process generating two new entries for the Hqueue manual - so yes, it was definitely a bit of work, haha.

Hallo johner,

I only have seen your post today. Thought I had set up my account for automatically receiving emails for replies to my posts. It seems I hadn't. Thanks for the tips but sadly I had to delete the simulation data to free disk space. The solution I came up with for now was to cut the simulation into smaller pieces and make five shots of it. I wasn't very happy about this. Costs me a lot of extra nights. I'm just rendering the first shot and it looks crazy real but simulation and render times are nuts. Will defiantly post the result here, but it will probably take another extra two months.

Cheers

looks nice!

i had same problem, i as to dissable Clumping the “ disabble density control ” for can simulate complete the ww, else crashes me ram out 300%.

other thing is, where is the mist? i dont see it, its inside of WW? also you know how can dissable the bubbles or splash or leave only splash?

thanks and keep up nicely waterfall!
cheers
C