Hey all,

I'm playing around with the new Toucan groom setup that was posted on August 30th:
https://www.sidefx.com/contentlibrary/toucan-groom/ [www.sidefx.com]

It seems that the network crashes Houdini, has anyone else seen this/found a solution? I'm running 20.5.331.

From some digging around, everything cooks fine up to "/obj/feathers_modelling/feathersurfaceblend3" node but, if I cook the "featherdeintersect1" node, Houdini crashes. I've noticed there are a few curve artifacts which seem to get introduced after "/obj/feathers_modelling/blendshapes1, but the setup still crashes even with that disabled. I'm fairly sure this might be a GPU memory issue, so
I've also tried setting my HOUDINI_OCL_DEVICETYPE to CPU, just in case, but, whilst it seems to cook for longer, Houdini still crashes.

If I disable the deintersection node(s), the network does allow me to cook a little further down, but, in the "Barb Randomisation" network group, the "feathersclump14" node also fails (but Houdini doesn't crash). From inside the node, I get this error:

(Warning: ./featherbarbxform5/featherutility1/attribdelete1/attribute1: Invalid attribute specification: "no match for __featherutility_*".

This means the remainder of the network then won't cook as that side of the network eventually gets fed into the "Colour" network group. I can disable it, but then the result is a mess anyway, so thats not ideal.

I can probably work around this by tweaking the groom itself (it feels like maybe its a memory issue on my end) but would love to work directly with the setup. Thought it might be worth flagging too, as I expect other users might come across this if they download the setup from the content library

If anyone has any ideas how to fix this or whats causing it more specifically, I'd love to hear them.

Thanks,

Matt

Hi Matt, good to hear from you.

Thanks for reporting this, I'll check if I can reproduce it.

I can cook the whole setup here. My houdini sits at 34GB RAM usage by the end. GPU memory spikes to 5.4GB during deintersect.

If you're running out of RAM it might help to cache out an intermediate step and reload houdini.

Or there might be something system specific going on. If it fails at Feather Deintersect its more likely a GPU thing, maybe a bug in the OpenCL code that only happens with certain driver versions. That's something we've certainly seen (fun). Got any more info about your system?

Can you switch your OpenCL device to CPU and test? You can do that in Edit->Preferences->Misc.

Ok I'm actually getting a crash in CPU mode now. So there's likely to be a bug - the CPU driver is generally better at catching those.

Hey Kai!

Hope you're well! Thanks for taking a look.

I'm using a 64GB machine with 8GB GPU memory so judging by what you're seeing on your end that should be more than enough. Running Nvidia driver version 510.73.05 which is far behind the latest I think so, I don't know if thats likely to cause problems?

Either way, I was also seeing a crash as well in CPU mode. That being said, I'll double check this with some intermediate caching anyway, just to be sure and get back to you.

Cheers,

Matt

Well, in classic "this thing that should work isn't working" fashion, I now can't recreate the crash at all.

For the record, I tried all of the following things to be 100% sure:
- Opening the scene file fresh and cooking the de-intersection in both CPU & GPU modes
- Caching the network up to the de-intersection nodes and then cooking (again, with CPU & GPU modes)
- Cooking to the end of the network in GPU mode

No crashes whatsoever.

My feeling is that this was 100% a memory issue. My GPU memory peaked at around 6GB and the CPU execution was at about 30GB. My guess is that I was likely running a few other processes on my machine when I tested this on Monday (sadly, I can't remember what) but these were taking up enough memory to tip the scales and cause the cook to crash. My machine hasn't restarted since then, so I don't think that comes in to it.

Weirdly, I also don't see the strange artifacts coming out of the "blendshape1" node (in the feather_modelling network) anymore either. I've seen the old (pre-vulcan) scene viewer do similarly strange things when it ran out of GPU memory before too, so perhaps this was a symptom of the same underlying problem. I'm wishing I'd taken a screenshot now.

Either way, I feel like I've solved my own problem, but I'm intrigued that you were able to recreate a CPU crash on your end... If you want me to open a proper bug report for this, I'm happy to, otherwise I think everything is under control again!

Thanks,

Matt

Also I should definitely mention - one cool side effect of doing these tests in both CPU and GPU mode was seeing just how much quicker the GPU de-intersection mode is compared to the CPU version. Nice one!