Hi,

I'm rendering on a second workstation using hqueue. Submitting from the stage with a fetch rop grabbing the "USD Render Rop", which is then wired in to an hqueue rop.

The behaviour I'm seeing is that it creates the job with the first task generating the subsequent tasks which are the frame batches.

I'm used to a workflow where that would be generating ifds, then the subsequent tasks are just mantra. Quick loads, only requires render license.

What I see is those tasks for rending the frames are just pointing back to the original hip. Fine - means it uses the license but it's a one-node farm so who cares.

However - the first tasks creating those frame tasks is incredibly slow - tying up the machine for perhaps half an hour. Really significant. Since I'm not waiting for it to generate USDs, what is it that is taking the time here?

( I know the proper response is to write submission and wrapper scripts - which I've not done for hqueue and wanted to avoid doing. This is the out-of-the-box experience which I was hoping would suffice for my one man show )

Thanks in advance for any tips.

r.

I'm new to USD and Solaris, so I could be wrong, but I think that if you are fetching the USD render ROP, then the first job is generating USD's. It's very similar to generating IFD's, so it can take a long time.

I made an HDA for sending husk jobs to hqueue. It's super bare-bones. Someone else has probably done the same more elegantly. You still have to generate the USD somehow. Then my tool sends husk commands to hqueue, telling it to render that USD. So far, I've only used it to run a very basic test and I have no idea whether it will work with more complicated scenes or outside of my environment. You are welcome to give it a try. I make no guarantees that it will work.

Yeah - generating the USD - I guess that's defined with the config-layer LOPs - I don't have a final one in my scene... and there's no specifying a path like there is with IFDs. So I need to do more reading / playing to work out what exactly is happening. I'm pretty sure it's cooking way too much stuff given the time it's taking. Solaris still pretty mysterious to me.

Thanks for sharing your asset - that's exactly the workflow I'd expect. Will check it out next week when I have a moment.

Hey Brad,

Thanks again for sharing your HDA... got me in the right direction. It is tonnes faster and more elegant than sending the whole rop to hqueue.

I'm attaching my version of the asset in case anyone's reading along.. I've
* Moved to code to the python module
* Fixed issue with missing last frame
* Add redshift flag
* setup expressions to pull most of the info from a USD Render ROP LOP.
* Added eval so supports per frame usd

There's a little bit of guss (ver_cntl_path and jobname expression) that'll only work with my setup - but hopefully this helps others the way your asset helped me.

Cheers,
r.

Hello,

I'm trying to get this working on Linux.
And I don’t really know what exactly the script is doing.

Am I correct in assuming that the equivalent to hcmd is the same as a normal terminal that has been sourced with the houdini.setup? In Linux

The render node group is also confusing to me as in my head the information should be in the rendersettings in the usd file itself, no ?

Does ver_ctrl_path just create subdirs on the share ?


Also, am I correct in thinking that this will only work locally if working with usd file references ?
And that an extra script taking all the usd files rereferencing them and putting them in the share is needed for this to actually work?

Right now, the job just hangs on
waiting for resources
Indefinitely.

NicTanghe
Am I correct in assuming that the equivalent to hcmd is the same as a normal terminal that has been sourced with the houdini.setup? In Linux

The render node group is also confusing to me as in my head the information should be in the rendersettings in the usd file itself, no ?

Does ver_ctrl_path just create subdirs on the share ?

Also, am I correct in thinking that this will only work locally if working with usd file references ?
And that an extra script taking all the usd files rereferencing them and putting them in the share is needed for this to actually work?

Indefinitely.

Yes, hcmd is a windows executable that opens a terminal with houdini setup.

The render node group is an hqueue feature that lets you specify a subset of render nodes to run on, rather than the whole farm.

I think ver_ctrl_path is something that Rangi added. I'm not sure what that does.

The script doesn't handle any re-pathing beyond the usual $JOB, $HIP, $F, etc. It just submits a USD render job to hqueue using husk.exe. Whatever paths are in the USD file are what will be used.

On a side note, I haven't looked at this script in years. It worked the last time I used it, on a windows-only network and farm. I left the company I wrote it for and we use a different queueing system where I am now, and I no longer need this. I'm really surprised that SideFX still hasn't released something more elegant yet.

rangi
Hi,

I'm rendering on a second workstation using hqueue. Submitting from the stage with a fetch rop grabbing the "USD Render Rop", which is then wired in to an hqueue rop.

The behaviour I'm seeing is that it creates the job with the first task generating the subsequent tasks which are the frame batches.

I'm used to a workflow where that would be generating ifds, then the subsequent tasks are just mantra. Quick loads, only requires render license.

What I see is those tasks for rending the frames are just pointing back to the original hip. Fine - means it uses the license but it's a one-node farm so who cares.

However - the first tasks creating those frame tasks is incredibly slow - tying up the machine for perhaps half an hour. Really significant. Since I'm not waiting for it to generate USDs, what is it that is taking the time here?

( I know the proper response is to write submission and wrapper scripts - which I've not done for hqueue and wanted to avoid doing. This is the out-of-the-box experience which I was hoping would suffice for my one man show )

Thanks in advance for any tips.

r.

Hello,

Would you be able to post your original .hip file where you encountered these issues?

The first task that the HQueue Render ROP submits to HQueue should be labelled as "Generate USDs". The task should generate .usd files and also submit followup render tasks that render directly from the generated .usd files. The render tasks should only take up Karma licenses instead of Houdini licenses (if rendering with Karma).

Note though that the "Generate USDs" workflow wasn't added until Houdini 20.0 so if you are using Houdini 19.5 then that would explain why your tasks are rendering directly from the .hip file.

However, if you are using Houdini 20.0, then the tasks should generate and render from .usd files by default. Check that the USD Render Options -> USD Output Filecheckbox parameter on the HQueue Render ROP is checked on.

The other possibility is that the Fetch ROP is confusing the HQueue Render ROP causing the "Generate USDs" workflow to be skipped over entirely.

As to why the first task is slow when not generating .usd files, it's hard for me to say. However, the first task needs to evaluate a few parameters in order to gather information that's stored in the followup render tasks. Perhaps evaluating one of the node parameters triggers node cooking or some other heavy operation?

Cheers,
Rob