I'm looking for a way start a listening/interactive mode with tops.
Can I have a loop that polls a database mysql/sqlite or something for changes and then query something and be processing whatever path I get back from it?

Not sure where to start. I can do it in python outside of houdini, but I'd rather have it all running inside of a houdini session that is just listening to a database.

Not sure where to start, if at all possible. Any pointers, help file or tutorial would be great!

Thanks!

Ok, so I have found a solution, which I think could be improved upon.
This setup keeps a heartbeat runnig for 30 seconds and each second you can generate new work_items. You can obviously change number of heartbeats and wait time. So this allows you to fetch live data form a database or folder for example and act accordingly. There's no logic for tracking what was processed, you need to build that yourself. But it's at least a way to keep houdini polling.

Now in the file, I have also 2 python generators which I think would be the proper solution, but can't get to work.
I'd love to know if there's anything better of more robust perhaps.

Hope it helps someone and if you have feedback please let me know!

There isn't a built in way to do that right now, but we are planning to add support for file system watching/triggers in the future. The same functionality could be generalized to other external resources.

One option that exists now is to use an experimental feature that's not available from the UI, but can be used from the Python Shell or a button callback script. You can cook a TOP graph with “continuous cooking” enabled – when the network finishes it will automatically recook itself after a time out period. This can still be interrupted by canceling with ESC or from the Tasks menu. You can use that in combination with a Python Processor node that has custom work item Regeneration logic. The onRegenerateStatic hook will run at the beginning of each cook, and you can use it to delete, add or dirty existing work items.

The actual cook options are described here: https://www.sidefx.com/docs/houdini/tops/pdg/CookOptions.html [www.sidefx.com]

To use that, you'd need to manually cook the TOP graph using something like the following:


You can put in a button callback for example, to trigger the cook. Once a cook completes PDG will recook the output node automatically in a loop, until the graph is canceled. When used with a node like the File Pattern, for example, changes to the file system will cause the work items to be dirtied/updated on the next cook.

Yes, this is exactly it!!
Thanks so much for this, it's so much better now, python processor recook/regenerate works great!

Really great stuff, can't wait for this to be available as nodes. It would be great to be able to have portions of the flow doing polling while other parts are collecting and processing for example.
But that's icing on the cake, this helps a lot.

Cheers!

As an addendum, it might be worth mentioning that any python operation that is calling file operations like process.Popen/os.call etc is best called in a python server and a Command Sent node. After many calls to it in a normal python node it locks up the UI and eventually stops processing all together. Not sure wby, but I think in the second run some thread/hooks are not cleaned up properly otherwise. I'm seeing this issue

tpetrick
There isn't a built in way to do that right now, but we are planning to add support for file system watching/triggers in the future. The same functionality could be generalized to other external resources.

Hi!

I'm revisiting this topic for Houdini 19.5, since the old setup I had, that worked for 18.5 stopped functioning in 19.5. The setup is a python processor with parts of the above code implemented.

1. Is the above setup still valid for 19.5? The goal hasn't changed, I would like a "polling graph" setup. In my case check a database for new items to process.

2. Separate from that I would also be interested in knowing if a setup that can handle web-request payloads via the Houdini web-server for example, preferably non blocking. Probably reasonably big topic, but I would be very interested in a tutorial of some sorts on topic of having a polling or listening Houdini/hython instance.

Thanks for any info! 🙏
-Johan

That code should still work, but you won't be able to trigger a cook of a TOP network from within a Python Processor that's evaluating in the same network. You'll need to run it from e.g. a button callback, shelf tool or the Python shell.

What issues/errors are you running into?

Thanks!

tpetrick
What issues/errors are you running into?

I dove a little bit deeper and
has become obsolete. When I remove it, it works again without errors.

I'd also like to mention that is undocumented in the GraphContext : https://www.sidefx.com/docs/houdini/tops/pdg/GraphContext.html [www.sidefx.com]
The only mention of it I could find is here, 2nd alinea : https://www.sidefx.com/docs/houdini/tops/pdg/CookOptions.html [www.sidefx.com]

I think this topic is still of great interest to pdg users that want to use Houdini as a service or just have it polling/listening to changes. So would it make sense to RFE for a more quick solution for this, pressing a button on the scheduler, or a TOP's HDA with the soul purpose to loop the current TOPnet?