Hi,

I'm currently looking into using PDG for processing a tiled terrain in Houdini.
The terrain in question is based off DEM data and then detailed using the heightfield nodes (erosion, noise etc..)

The workflow in detail is to bring in a piece of terrain using MapBox and then split up the terrain into multiple tiles which are then processed independently and then merged back together for another tile splitting pass to be used in a game engine.

The reason why I decided to split up the terrain into multiple tiles is to take advantage of the OpenCL capabilities when doing the detailing pass.
The actual terrain is quite large (12x12km) and I'm looking for a resolution of 1 voxel/meter. Therefor the need to split up the terrain with some padding before processing.

Considering the fact that I have a 6 by 6 tiles setup, I want to have Houdini go through each tile and then run an HDA and export the result to disk.

Do I need to use PDG for this use case or is there a simpler way to make this work without using PDG.
With my current setup I would have to duplicate the node graph 36 times (6x6 tiles) before getting the final result.

I have looked into some tutorials for PDG and a lot of them just seem to skip the reasons why they make some choices. They just tell you click here and there and do not explain the reasoning behind it.

There are several reasons for using PDG with tiled terrain. One would be to split the work to take advantage of parallelism, whether locally (multi core) or via a farm. Another would be to take advantage of incremental updates (or minimizing the work that needs to be done). You mentioned having to duplicate the node network, but PDG can eliminate this by running the HDA over as many tiles as needed, so there is work reduction there as well.

Have you have watched Kenny Lammers' videos on this topic: https://www.sidefx.com/learn/collections/pdg-for-indie-gamedev/ [www.sidefx.com]
His tutorial addresses this exact topic but if you have specific questions, feel free to post here.

seelan
There are several reasons for using PDG with tiled terrain. One would be to split the work to take advantage of parallelism, whether locally (multi core) or via a farm. Another would be to take advantage of incremental updates (or minimizing the work that needs to be done). You mentioned having to duplicate the node network, but PDG can eliminate this by running the HDA over as many tiles as needed, so there is work reduction there as well.

Have you have watched Kenny Lammers' videos on this topic: https://www.sidefx.com/learn/collections/pdg-for-indie-gamedev/ [www.sidefx.com]
His tutorial addresses this exact topic but if you have specific questions, feel free to post here.

Thanks for the info Seelan, yeah I have seen Kenny's tutorial. I did manage to get PDG working on the tile processing part.
Basically it takes the whole terrain splits it up and then erodes the tiles and saves the result as .bgeo files in the project folder.
PDG allowed me to process large amounts of data without running out of memory. Something which wasn't possible in the previous versions of Houdini.

Right now I'm stuck at fetching all tiles (pdg output)and feeding that back into a tile splice to merge all the volumes.
While I can manually load the pdg output by using the “=” shortcut, it seems to me like I'm doing this the wrong way.

I'm guessing that I have to use a file pattern node and point it to the directory of the pdg output in the top network while unchecking the option “Split results into separate items”?

For info, when I do this step at the SOP level, I use a Merge node to combine the eroded tile volumes and then point the result to a Heightfield tile splice node before finally using the height output for exporting with correct tile naming.

It took me a bit to figure this one out too, but I think the geometry import top was created for this task. It will merge geometry file output of upstream items into a single item for processing.

Thanks for the tip Jsmack, will give it a try

You can use a file pattern node if its not connected to your TOP network or if it doesn't have the upstream items. Yes, you'd have to uncheck the Split Results into Separate Items. Then you can attach a Geometry Import and turn on Merge Input Geometry.

Another way, if you have the upstream work items containing the heightfield files as outputs, is to use a Wait For All then a Geometry Import (with Merge Input Geometry enabled). This is better as you don't have to worry about pointing to the right folder/path with a file pattern.

See screenshot, with the 2nd method highlighted in red.

Thanks a lot for pointing me to the right direction, looks like the Waitforall + geometryimport solution seems to be cleaner and more streamlined, will give this a try and share the results.
One more question, is PDG supported in Houdini engine for Maya? I'm considering making some HDAs which I could share with community later on.

There is no PDGAssetLink for Maya yet, if that is what you are asking. This means you won't be able to manually execute PDG dirty & cook functionalities from within Maya over Houdini Engine yet.

But you can still use TOP nodes in your HDA, either via the TOPNet SOP or the TOP network object.

A TOPNet SOP will automatically cook the PDG graph and merge the display node's generate files and provide that as the output of the SOP. Note that if the display node in the TOPNet SOP does not produce any output files, then the TOPNet SOP will generate an empty geo, so your display node must output geometry files (such as an HDAProcessor or ROP Geometry).

A TOP Network object will not automatically run the PDG cook. To work around this, you can attach a button parm to your HDA which can then manually cook the TOP Network via python script callback. Then you'll need to load in the generated geometry somehow.

Thanks for the reply Seelan, I'm currently trying to output the heightmaps using the TOP network.
I built and HDA which contains a heightfiled output node with some paramaters :





What's inside the HDA :



I'm not sure if I am doing this right but it seems like the final HDA processor isn't outputting any image files after cooking the node. Do I have to make use of the ROP output node in the TOPs network?

Edit : I saw that I didn't set the heightmap output channel to “height”. Trying again to see if this works.
Edit 2 : none of the solutions I came up with worked so far. Maybe I will have to make a custom Heightmap_output node which works with PDG at some point.

The reason its not outputting is because the HF Output Node doesn't automatically create the file, but rather you have to invoke the Save to Disk button manually to do so. So in PDG, while the HF Output Node cooks, it doesn't automatically create the image file.

To fix this, you need to do the following:

Expose the Save to Disk button from the HF Output Node to the HDA parm interface.
Then make the HF Output Node as editable in the Operator Type Properties windows.
Both of these are outlined in red below.


Then, in your HDAProcessor, hit the Update HDA Parameters to get the latest HDA parms, and make sure the Save to Disk checkbox is enabled under HDA Parameters, like so:


HDAProcessor will then make sure to invoke the button when cooking. And the HF Output Node editable allows to change the state of the button.

Hi Seelan, the guide you provided works and it now saves the heightmap tiles to disk. Thank you!
All I was missing was making HF Output Node as editable in the Operator Type Properties windows.

Great information in here.

I was trying to achieve the same thing and made it pretty far on my own, but got stuck because I didn't have the additional geometry import between my waitforall and the tilesplice.

Thanks for the tips!

Hi,

I followed those steps and it works great to output some Heightfield masks.
The thing I want to do now is to get those output masks as work items to process with Image Magick
So, how to get those generated images as work items?

I unchecked “Write Geometry” in the Asset Output section of the HDA Processor because I didn't need any geo output, but I don't know how to get those images the HF Output node has rendered.


I've used a “File Pattern” TOP node to get those generated images, but I'm worried it might get more work items than necessary because when I “Dirty and Cook” my mask generator, it will create new images in the same folder. So for instance, if it generated 4 images, I will have 8 images on a 2nd cook.

Thanks!
Alex

Why not just do a wedge, this feels like am overkill to me.

chekboom
Hi,

I followed those steps and it works great to output some Heightfield masks.
The thing I want to do now is to get those output masks as work items to process with Image Magick
So, how to get those generated images as work items?

I unchecked “Write Geometry” in the Asset Output section of the HDA Processor because I didn't need any geo output, but I don't know how to get those images the HF Output node has rendered.


I've used a “File Pattern” TOP node to get those generated images, but I'm worried it might get more work items than necessary because when I “Dirty and Cook” my mask generator, it will create new images in the same folder. So for instance, if it generated 4 images, I will have 8 images on a 2nd cook.

Thanks!
Alex

I suppose you can write out a csv file with the output files generated, then use that csv file to create work items in another TOP graph to do the image magic part.

Thanks for your suggestions, will try that

hi, i'm currently looking for use of pdg to handle large terrains. If possible, please attach the asset files. This will help me a lot.