Hi There,
I'm looking for a bit more information on MQserver as we setup our custom scheduler, I found the MQserver documentation page but no documentation on how to communicate with it is there any?


I'm also bit unclear on a few other things such as:



what sort of things you can communicate, is is just the status of the work items, or can you update the data on work items or even send new work items through Mqserver?

mqserver is used by PDG jobs when they are running with deadline and hqueue schedulers to RPC back to PDG, it's also used to coordinate 'services', used in local graph cooks. The python module pdgutils can be used to communicate with mqserver at a low level, but jobs use the pdgcmd/pdgjson APIs as documented here:

https://www.sidefx.com/docs/houdini/tops/jobapi.html [www.sidefx.com]

There aren't any online docs for pdgutils yet, but there is the usual pydoc on the module.

> what sort of things you can communicate

The communication is primarily job -> PDG, through pdgcmd. However mqserver is a general purpose pub-sub server that could be used for custom messaging as well.

If you want to use this for a custom scheduler, I would recommend looking at our implementation for HQueue Scheduler in $HH/pdg/types/houdini/hqueue.py. The simplest approach is probably to only support MQUsage.CONNECT, which uses a known running mqserver so there's no need to spin anything up per submission.

Thanks Chris thats very helpful, we will have a dig through that JobApi and hQueue example..

I will mention that we are likely to start with a Static Scheduler implementation, and assume theres no reason why we cant use MQserver/JobApi to send data back to PDG once scheduled