Houdini 20.5 Executing tasks with PDG/TOPs

File tags

Work items track the “results” created by their work. Each result is tagged with a type.

On this page

Overview

Work items track results created by the item in the output attribute. The results are strings, usually file paths to files created by the work item. All results are tagged with a “tag” string that describes the type of the result. The tag string is a hierarchy of broad types down to more specific types, separated by slashes (similar to MIME types but totally free-form). For example:

Tag

Description

file/geo

Geometry file

file/geo/terrain

Terrain geometry file

file/geo/building

Geometry file containing a building

Some nodes and API functions let you match files by their tag. Tag matching is by prefix, so a file tagged as file/geo/terrain will match both file/geo and file, but not file/geo/building.

Although work item outputs are typically files, it is possible to have other types of information reported back as a result. For example, when using the ROP Fetch to cook a distributed simulation, the sim tracker work item will report a socket/ip and socket/port result, which other work item can use to connect to the tracker.

Tagging results

  • Nodes that generate files usually have a parameter for specifying how to tag work item results. The default tag is usually set based on the intended use of that node.

    For example, the Geometry Import lets you cook a SOP network and write the result to a geometry file on disk. The node has a Result tag parameter that sets the tag for the generated file. The default tag is file/geo.

Using tags

  • You can see the tag assigned to a result in the work item info window, and in the task graph table.

  • Tags can be useful when a work item has muliple files associated with it.

    For example, the Partition by Bounds node can create partitions from terrain tiles, and road geometry that intersect the tiles. The partition may end up having several files associated with it: a terrain file and zero or more road files. Tagging the files as file/geo/terrain and file/geo/road would make it easier to work with the files.

  • You can query results by tag using the Python scripting API. The pdg.input() function takes a tag parameter that only returns the results where the tags match by prefix.

    # Returns all input files tagged file/image on the incoming work item
pdg.input(tag='file/image')

Executing tasks with PDG/TOPs

Basics

Beginner Tutorials

Next steps

  • Running external programs

    How to wrap external functionality in a TOP node.

  • File tags

    Work items track the results created by their work. Each result is tagged with a type.

  • PDG Path Map

    The PDG Path Map manages the mapping of paths between file systems.

  • Feedback loops

    You can use for-each blocks to process looping, sequential chains of operations on work items.

  • Service Blocks

    Services blocks let you define a section of work items that should run using a shared Service process

  • PDG Services

    PDG services manages pools of persistent Houdini sessions that can be used to reduce work item cooking time.

  • Integrating PDG with render farm schedulers

    How to use different schedulers to schedule and execute work.

  • Visualizing work item performance

    How to visualize the relative cook times (or file output sizes) of work items in the network.

  • Event handling

    You can register a Python function to handle events from a PDG node or graph

  • Tips and tricks

    Useful general information and best practices for working with TOPs.

  • Troubleshooting PDG scheduler issues on the farm

    Useful information to help you troubleshoot scheduling PDG work items on the farm.

  • PilotPDG

    Standalone application or limited license for working with PDG-specific workflows.

Reference

  • All TOPs nodes

    TOP nodes define a workflow where data is fed into the network, turned into work items and manipulated by different nodes. Many nodes represent external processes that can be run on the local machine or a server farm.

  • Processor Node Callbacks

    Processor nodes generate work items that can be executed by a scheduler

  • Partitioner Node Callbacks

    Partitioner nodes group multiple upstream work items into single partitions.

  • Scheduler Node Callbacks

    Scheduler nodes execute work items

  • Custom File Tags and Handlers

    PDG uses file tags to determine the type of an output file.

  • Python API

    The classes and functions in the Python pdg package for working with dependency graphs.

  • Job API

    Python API used by job scripts.

  • Utility API

    The classes and functions in the Python pdgutils package are intended for use both in PDG nodes and scripts as well as out-of-process job scripts.