Still on the base release of 17.5.173, so I might be a little out of date.

We've had a little consternation with an HDA Processor that was silently failing in PDG and we had trouble working out what the issue was. The PDG WorkItem log would display simply that there was “No geometry generated!” and no other useful information.

It transpired that actually it was due to a command raising an exception inside a Python node, within that HDA.

So I've got a bit of a broad question in terms of how we should approach debugging PDG failures in cases like this. I'm not sure what is the expected behaviour here and if there are tools I am missing.

• It's surprising that the Python exception is not elevated to the PDG log. If I replicate the error conditions inside Houdini, the node goes red and I get a clear error message explaining why it has failed, but PDG simply tells you that “No geometry generated!” error message. Especially quite odd since if I use print() functions inside a Python node, those do show up in the PDG log.
  
• If there are any more detailed log files being baked out that give details as to specifically which node failed within the HDA, I haven't been able to find any output.
  
• The Debug .hip File function is useful, but it seems somewhat poorly documented and there is no easy way to find where the debug .hip was saved. I've had to resort to searching for *.pdg through my project folder and then ordering those by date modified in order to find the relevant ones. It seems like the debug hip path ought to get saved out as a workitem attribute similar to the outputpath.
  

We've ended up adding Python nodes that print() debug data, so as to get a better idea where certain operations are failing, but this seems unnecessarily complicated.

Am I missing any obvious paths for debugging, or do are these potential areas for future development?

So this is a specific issue with the HDA Processor node. It's wrapper program we wrote to just run an HDA, implemented with HAPI. It's quite possible that it's not digging deeper into the failure to get all the logs it can out of HAPI. Attaching the debug hip paths as attributes is something that makes sense too. So yes, all things we should be improving upon. I've logged RFE 97150 to capture it.

The other bit of advice to give for now would be to use a filter by expression node to limited the number of workitems, then run the Debug HIP file function on the subset to cut down the number of files being dumped.

Thanks Ken! Good to know

Thanks, Ken,
I find the Python Script TOP node has the same problem, too. If I turn off the “Evaluate In Process” checker, I can't see the python print out info in Houdini, the only thing I find is it prints to a .log file. The Python Processor TOP node works the same way as far as I remember. It's just difficult to debug.

Yeah unfortunately there is not much we can do about the fact that the info goes to a log file - when it's running out of process, it's being run in a different program than Houdini entirely, so capturing the information in a log file is pretty much the only thing we can do in that case.

I wonder if in the future there is scope for capturing generic out-of-process output in a standardised way, e.g. capturing log file output and make it more easily viewable in the PDG execution graph, provided PDG knows where to look for the log files. I can see some pipelines that rely on a lot of third-party software being quite clunky to debug if you have to hunt down the logs separately for every different process in the chain, so perhaps there would be value for trying to route all of that through to the user.

Yes, that could make sense. Could go along with features to better visualize / organize the various outputs.

Where exactly does the Debug hip file get written?

The location can be set in the Local Scheduler, under “Directory Location”.