Been trying to wrap my head around all that is PDG because it really seems like it's going to save me about 1.6 billion years of iteration time. Getting there, but could use some help.

Breaking this into sections to make it easier to read…


1) Examples
So I've been through the SideFX tutorials, the Entagma tutorials, and through the Mutagen video. The first question I have is for SideFX. This page is a great example of what I–and I'm guessing a lot of users–want to do with TOPs/PDG:

http://www.sidefx.com/docs/houdini/tops/wedge.html [www.sidefx.com]

Are the example scene files for the shown simulation, variation, and level of detail wedging available somewhere? I've popped a lot of the related node example files, but haven't seen anything close to these real-world examples.


2) Resuming Work
Let's say you've been putting together a PDG. You get the geo written out but the recess bell rings and it's one of the three sunny days a year in Seattle and you don't want to miss it. So, you save your file, quit out of Houdini, and head out.

Once you reopen the file, how can you get a node to either understand that it should be looking for existing/pre-cooked data or guarantee that making it dirty will override existing data?

The results seem inconsistent; sometimes a node like ROP Geo Output sees a bake already on disc, sometimes not. Sometimes I dirty a node and it seems to think there's still usable data on disk when there isn't. I fell like And it's just painful when a ROPFetch render node gets accidentally dirtied…


3) Standalone-ish Node Options
If you have a series of images from a non-PDG wedge set (I'll get to the why of that in a sec), is there any way to reference those images with the ImageMagick montage and FFMpeg nodes to create animated contact sheets?


4) PDG and Redshift
As it stands now, I can get a Mantra-rendered, animated sim contact sheet out of PDG using a slightly modified setup from Adrian Meyer's Mutagen Toolset. That's the TOP_Mutagen network in the attached file.

I can also use an old school Wedge setup to create similar frame sets via Redshift, but not sure how to compile those into a contact sheet.

I've tried a bunch of different ways to get a Redshift ROP to work, and I'm almost there with a For loop that is getting iterations from upstream. Still isn't quite working, though. That's the incomplete TOP_Wall_Sim network in the file.

I know people have managed to get PDG to work with Redshift even though the functionality isn't technically there yet. Any guidance there would be a huge help. Have posted in the RS forums but no luck yet.


Oof…sorry, that was a lot. Just have burned a lot of time on this without quite getting what I need. Any help woiuld be great.

Thanks!

–
mC

1. I've attached the sample files from the launch here.

2. There is a feature in PDG where it calculates the “expected output” of workitems. It checks on disk to see if the “expected output” is there. If it is, it'll mark the workitem as cooked instantly. This is currently implemented for the ROP Fetches, and the HDA Processor. Also, this will only work if you are using the @syntax. So if you're using environment variables, or using a node outside of what's mentioned above, that feature won't work. This may explain the inconsistencies you're seeing - if not, please give us a file that we can take a closer look at. Note also we recently added some niceties for the ROP Fetch to make this work better - there is a “Output Parm Name” field that you can use to specified what the expected output files should be. Details here:

https://www.sidefx.com/forum/topic/66828/ [www.sidefx.com]

3. Yes, those nodes are pretty agnostic. Try to use a file pattern for example to pick up the images as workitems and then send them to the ImageMagick or FFMpeg nodes.

4. I don't have too much to add in terms of redshift, but again try the file pattern to pick up the redshift rendered images, or use the PDG API to attach result data to a workitems (addResultData method on pdg.WorkItem).

I see also there is already a whole bunch more discussion around redshift, so perhaps the issue is settled:

https://www.sidefx.com/forum/topic/66828/ [www.sidefx.com]

Hey, Ken! Thanks for the detailed reply.

First, crazy grateful for the launch files. Can't wait to dive in.

I've been banging away on this over the weekend and with a few threads running, I don't think I got them all updated. I'm actually on the other Redshift thread as well.

Turns out that item #2 was related to the output path issue. Since the ROPFetch node ‘lost’ the location of the image files immediately after rendering third-party nodes that didn't pass vm_picture, it considered itself perpetually dirty for cache operations.

Looks like both you and the Redshift devs are hitting this shortly, which is great…unless of course you create some sort of recursive parameter cross-reference that unravels time/space. On the plus side, we'll be able to iterate on what the end of the universe looks like with Redshift-rendered frames in a PDG…so we've got that going for us, which is nice.

I'll poke with additional file-handling options with #3.

And on #4, I'm feeling pretty good about scheduler and ROPFetch/ROPNet settings to reliably get Redshift images out of a PDG now. Seems like I kept coming up one flag short, but Matt Ebb, Saul Espinosa, and Dylan Neil got me straightened out.

The only thing I'm still wondering about is a definitive answer on resource management with multiple GPUs. Is there a clear correlation of CPU to GPU in the scheduler references/count? Really want to find that sweet spot between maximizing efficiency with the GPUs and completely swamping them.

–
mC

Glad to hear of the progress

Unfortunately, there is no correlation between CPU and GPU. We have no information on our end about the process to be spawned and what its performance characteristics are… the best thing to do is to look up / ask about the performance characteristics of the application to be spawned, then work backwards from that information to determine the right CPU per task setting for the most optimal results.

Hi MoltenCrazy

And on #4, I'm feeling pretty good about scheduler and ROPFetch/ROPNet settings to reliably get Redshift images out of a PDG now. Seems like I kept coming up one flag short, but Matt Ebb, Saul Espinosa, and Dylan Neil got me straightened out.

could you elaborate how do you get redshift to work with TOP's please ?

Thanks,

/Risto

could you elaborate how do you get redshift to work with TOP's please ?

Hi, I just did the basic PDG city building tutorial (again ), and I could easily use a ROP Fetch TOP, instead of a ROP Mantra TOP. Set it to Cache Mode: Write Files and Point it to the Redshift ROP using the ROP Path.
I used a variable inside the Redshift ROP output filepath, and it worked, showing 4 different results in an Imagemagick mosaic.

Best regards, Martin