Writing a Processor

Overview

Processor nodes define logic for creating new work items and adding attribute data to them. Processors can create work items as a function of inputs, or create completely new work items without any dependencies.

In order to write a custom processor, you should create a class in C++ that derives from PDG_NodeCallback. Unlike Partitioner nodes, processors have a longer list of callback functions that can be implemented to customize the behavior of the node. Most of these functions are optional – in fact the only one that's required to produce a functional node is in the onGenerate() method, which generates new work items.

Processor Callbacks

Processor nodes can implement various callbacks to control the configuration of the node itself, the creation of work items, and their evaluation. PDG will make sure that the attributes for any work items passed as callback inputs are safe to read from for the duration of the callback. It also safe to read and write to any work items created during the callback. It is not valid, however, to store work items to member variables on the node, or to write attributes to the upstream work items.

Processors also need to be thread safe – if a node is dynamic, it may generate new work items in parallel if multiple upstream work items finish at the same time.

All callback function implementations should return a PDG_CallbackResult value, which indicates that status of the function.

onGenerate(PDG_WorkItemHolder*, const PDG_WorkItemArray&, GenerationType)

This callback is evaluated any time the processor node should generate new work items. The PDG_WorkItemHolder argument is a factory object used to create new work items in the node. Work items are not added to the processor node or PDG graph until the callback successfully returns. If the callback fails, anything created during the portion of the callback that did evaluate is deleted and discarded.

Work items are added to the holder using the PDG_WorkItemHolder::addWorkItem method, which is invoked with a PDG_WorkItemOptions helper struct to configure the work item.

The upstream items list passed to the callback contains no work items if the node has no inputs. If the node has input nodes, the list contains one work item if the node is dynamic or the full list of upstream work items if the node is static. Finally, the generation_type argument determines whether the callback is being called on a static node, dynamic node or for work item regeneration.

PDG_CallbackResult
PDG_CustomProcessor::onGenerate(PDG_WorkItemHolder* holder,
                                const PDG_WorkItemArray& upstream_items,
                                GenerationType generate_type)
{
    UT_WorkBuffer errors;
    PDG_WorkItemHolder::CommitResult result;

    PDG_WorkItemOptions options;
    options.myCloneOutputFiles = true;

    for (auto&& upstream_item : upstream_items):
    {
        options.myParent = upstream_item;

        PDG_WorkItem* new_item = holder->addWorkItem(result, options, errors);
        if (!new_item)
        {
            myNode->addError(errors.buffer());
            return PDG_CallbackResult::eFailure;
        }


        // .. add attribute data to the new work item
    }
}

onPreCook()

This callback is called once on all nodes that will be evaluated during a cook, at the beginning of the cook. It is called before any onGenerate callbacks and gives the node the opportunity to load shared resources or initialize variables.

onPostCook()

This callback is called once on all nodes that were active during the cook, after the last work item in the graph has completed cooking. This gives nodes an opportunity to unload shared resources used during the cook.

onPrepareTask(PDG_WorkItem*)

This callback is called once for each work item in the node immediately before it is scheduled for cooking. It's called before PDG checks for cached files on disk for the work item, so it can be used as a way to add expected output files for caching purposes. It can also be used to access output files or attributes of the work item's dependencies, modify the command line of the work item, or do other last-minute work needed before cooking the item. If this method returns a failure status code, the work item is marked as failed.

onCookTask(PDG_WorkItem*)

This callback is called when an in-process work item needs to cook, and the node definition does not use a standardized job script. The callback is able to write attributes to the work item and attach output results, a should run whatever in-process logic is necessary to cook the work item. The callback runs inside the main Houdini process on a background thread.

onPostCookTask(PDG_WorkItem*)

This callback is called when a work item successfully cooks, but before any downstream nodes generate from the work item. It can be used to do validation of the work item's output files or attributes, or to make changes to the attributes based on output files create during the cook. The callback is run even if the work item was not cooked in process. If the callback returns a failure status code, the work item will be marked as failed.

This callback is only called if the work item has been marked as needing to run post cook logic, using the PDG_WorkItem::setIsPostCook() method.

onSelectTask(PDG_WorkItem*)

This is a UI callback specific for use with TOPs. When a work item is selected in the TOPs UI, this hook is executed after the item has been selected. For example, is is used by the Wedge node to push work item parameter values to the scene, but could also be used to run custom visualization logic.

onDeselectTask(PDG_WorkItem*)

This callback is the opposite of the onSelect callback, and runs whenever a work item is deselected in the TOPs UI.

onConfigureNode(PDG_NodeOptions*)

Invoked with a PDG_NodeOptions object that you can use to describe the configuration of a node instance based on the current parameters. You can set a text description of the node or configure whether or no the node requires all inputs to be generated in order to generate. This affects the behavior of the Generate When parameter when it is set to Automatic. If the node requires inputs to be generated, the Automatic parameter will always mean All Upstream Items are Generated.

The callback can also configure the list of attributes needed by the node and the attributes the node intends to add to work items, in order to better inform PDG about the node's behavior

Registering the Node

The shared library that contains the custom node will need to have a registerPDGTypes function, which is responsible for registering any custom PDG nodes. The PDG/PDG_ProcessorRandom.C and PDG/PDG_ProcessorRandom.h example shows how this might be implemented:

void registerPDGTypes(PDG_TypeRegistry* registry)
{
    PDG_NodeInterface processor("randomprocessor");

    // Random seed integer parameter
    processor.addParameter(
        PDGT_Value::eInteger, "seed", "Seed");

    // Random work item count min and max range parameter
    auto&& port = processor.addParameter(
        PDGT_Value::eInteger, "range", "Range", UT_StringArray(), 2);
    port->value(0)->setValue(1);
    port->value(1)->setValue(10);

    // ... other parameters ... 

    registry->registerProcessor<HDK_Sample::PDG_ProcessorRandom>(
        "workitemrandomize", "Work Item Randomize", "Processors", processor);
}