Hello.

We are experiencing heavy issues using PDG/TOP nodes via Houdini Engine.
Top nodes tend to not cook/cook indefinetly.

We used both Windows 10 and 11, Houdini 19.5 on all machines. Builds are 19.5.605 and 19.5.640.
We design HDAs inside Houdini Core and we use them with Houdini Engine 5.0.
We used 2 different kind of licences :
-Houdini Core
-Houdini Engine

Our goal:
We have an HDA "Stamp tool" that loads Heightfield tiles stored on disk. We load a "stamp" (Heightfield of a mountain) and blend it with the loaded tiles. We then export the result to update the tiles.


Our setup uses Houdini Engine HAPI with Unity. We do NOT use the provided plugin, only the DLL imports and we built a custom plugin with it.

The entire custom plugin is based on the “.cs” files we found with the base Unity plugin that accompanied Houdini 19.5.457.
HAPIConstants.cs
HAPIEnums.cs
HAPIFunctions.cs
HAPIStructs.cs
HAPIStructsMembers.cs
HARCImports.cs
HoudiniVersion.cs

We figure the different version number should not affect the behavior: we make low level calls to Houdini Engine 5.0 regardless.

Our problem is related ONLY to Houdini Engine. Everything tried inside Houdini Core worked, everytime, perfectly.


We have a process that uses an HDA structured like this:

-Input Node (connected to)
-Main SOP Stamp_Tool HDA
-SOP Processes
-SOP Exporter HDA
-Sop pre-process for export
-TOP Network
-HDA Processor

We provided a to show the basic structure of everything, nothing too fancy. Making you reproduce the error is close to impossible since we use a very custom pipeline.
The problem is located inside the Exporter HDA.
We figure our problem is largely related to TOP Nodes (making a no-pdg switch to export sequentially our data works)
The HDA Processor is setup to work in “service” mode. This is a design constraint : we must export a lot of data the quickest possible; any other approach is slower.

We tested on 4 different machines and got 3 different results.

--- Problem ---
The problem is that on two machine the process never finishes.

Machine 1 and 2 : the process finishes correctly
machine 3 : the process finishes if the HDA Processor node is in “out of process” mode but not in “service” mode
Machine 4 : the process never finishes in either service or out of process mode.

To manipulate PDG from unity using the HAPI we base our code on the example in the documentation (https://www.sidefx.com/docs/hengine/_h_a_p_i__full_source_samples__p_d_g_cook.html), We read PDG events and never receive either a complete, error or warning event.

--- Hardware ----
- Machine 1 :
It works in service and out of process mode (licence core)
CPU: Intel(R) Core(TM) i7-4770 CPU @ 3.40GHz
Memory: 16,0 Go DDR3
GPU: NVIDIA GeForce GTX 980 Ti

- Machine 2 :
It works in service and out of process mode (licence core)
CPU: AMD Ryzen 7 5800X 8-Core Processor
Memory: 32,0 Go
GPU: NVIDIA GeForce RTX 3090

- Machine 3 :
It works only in out of process mode (licence core ou engine)
CPU: AMD Ryzen 9 3900 12-Core Processor
Memory: 32,0 Go
GPU: NVIDIA GeForce RTX 3090

- Machine 4 :
It doesn't works at all (licence core ou engine)
CPU: AMD Ryzen 9 3900X 12-Core Processor
Memory: 16,0 Go
GPU: NVIDIA GeForce RTX 3090

--- Process manipulation from unity ---

- open a session
HAPIFunctions.HAPI_Initialize (Pipe session)

- Load an HDA
HAPIFunctions.HAPI_CreateNode (using the operator name to create a node from the HDA loaded through the env file)

- Create Input node
HAPIFunctions.HAPI_CreateInputNode
Then we get or create a "default part" for the input node using HAPIFunctions.HAPI_GetPartInfo and HAPIFunctions.HAPI_SetPartInfo

- Find some specific node in the HDA sub graph
using HAPIFunctions.HAPI_ComposeChildNodeList and HAPIFunctions.HAPI_GetComposedChildNodeList
then retrieving the names of the nodes and checking for the ones we wants using HAPIFunctions.HAPI_GetStringBufLength and HAPIFunctions.HAPI_GetString

- Connect Input node and HDA node
HAPIFunctions.HAPI_ConnectNodeInput

- Set some parameters and attributes using functions like
HAPIFunctions.HAPI_SetParmStringValue
or
HAPIFunctions.HAPI_SetAttributeFloatData
Depending on the type.

We also update the part to reflect changes in point count and things like that using HAPIFunctions.HAPI_GetPartInfo and HAPIFunctions.HAPI_SetPartInfo

- Cook the input node to effectively register attributes on houdini side.
- Cook the HDA Node
- Cook a specific node inside the HDA graph

Cooking those nodes rely on the same functions :
- HAPIFunctions.HAPI_CookNode to start cooking
- HAPIFunctions.HAPI_GetStatus to wait for the end of the cook

- Dirty PDG Node using HAPIFunctions.HAPI_DirtyPDGNode
- Cook PDG Node using HAPIFunctions.HAPI_CookPDG(
- Watch the cooking using events using :
- HAPIFunctions.HAPI_GetPDGGraphContextsCount
- HAPIFunctions.HAPI_GetPDGGraphContexts
- HAPIFunctions.HAPI_GetPDGGraphContextId
- HAPIFunctions.HAPI_GetPDGEvents


--- Conclusion ---
While we tried to have as many debug data as possible; we cannot find out why the TOP node would lock itself in infinite cook; or at least not be able to warn us that something went very wrong.

Debugs:
-open session/export as HIP
-Unity logerrors linked to session existence, and inspired by the plugin to display the result of every HAPI boolean
-recursive list of all node errors (not very useful with PDG) in a text file

Using TOP nodes in Houdini Engine is quite a hassle to be very honest. Do you have any tips on how to tackle all this more easily ?

Regards,
Anael BELLE