Houdini 20.5 Solaris and Karma

Writing out USD

On this page

Overview

See USD basics and About LOP node for background information on USD and how LOP nodes generate USD layers.

A LOP network generates a USD stage, either from scratch or starting with a top-level USD file, possibly adding in USD layers from files. At the end of the network, you can write out the finished USD, which will often involve writing out multiple files: a top-level file and separate files for each layer.

Layers being read from disk are always left unmodified by the save process, though an anonymous layer generated by LOP nodes may overwrite an existing layer file on disk.

How to

  • The main node that writes out the USD is the USD node. This node is available as a LOP (which you can put at the end of the LOP network) or a ROP (which you can put in a render network and point at the LOP network you want to output).

    (See other ways to write out USD below for more specialized options.)

  • To create a USDZ file, you can use the USD Zip node which creates a single self-contained USDZ file from an existing set of USD files on disk. This file is an archive containing all the layer files and textures used by those layer files. This is very useful for publishing final USD products on the Internet, but there are limitations to this file format that make it inappropriate for most uses in a pipeline. It is harder to modify (it must be extracted into individual files, modified, then repackaged), and does not support volume files inside the USDZ archive.

Where the files go

  • In the USD node, you specify a file path (in the Output file parameter) for the “top-level” USD file, containing data from the root layer.

    In addition to this file, the node writes any layers that have their save path metadata set to their own USD files.

  • You can use the Configure Layer LOP to assign/change a save path for any layer in the network. When you render to USD the layer will be written out to disk.

  • The SOP Import LOP and SOP Create LOP also let you specify a save path where the geometry will be written when you write out USD.

  • Save paths should be specified as absolute paths, often using global variables such as HIP ($HIP/props/lamp.usd). During the save process, the USD node by default will use an output processor that translates these absolute paths to relative references between the layer files, which makes it easy to move all the layer files from one location to another.

  • The Flush Data After Each Frame parameter on the render node controls whether it writes out data to disk after calculating each frame. This option can be used to generate a sequence of USD files each containing a single time sample (for example, lamp_0001.usd, lamp_0002.usd, and so on), or single files containing time sample data from all frames, depending on whether the output file name or save paths contain a time-varying component (such as $F).

Output processors

Output processors are Python plugins that can alter the file locations and file path strings used for referencing external files. They also allow for arbitrary last minute edits to each layer file being saved.

The USD node starts with one default output processor that makes file path references in layer files relative.

Writing a custom output processor

  1. Create a new directory $HOUDINI_USER_PREFS_DIR/husdplugins/outputprocessors.

    (We’ll create the plugin in the user prefs directory in this example. Of course, if you could put the husdplugins/outputprocessors directory under any directory on the Houdini path.)

  2. Inside $HOUDINI_USER_PREFS_DIR/husdplugins/outputprocessors. create a Python file for your output processor plug-in. For example:

    $HOUDINI_USER_PREFS_DIR/husdplugins/outputprocessors/outputreview.py

  3. In the Python plug-in file, sublcass the husd.outputprocessor.OutputProcessor class.

    In the subclass, implement the name() static method to return a unique name, and displayName() static method to return a string describing what the processor does (for example, Save Paths Relative to Output Path).

    import hou
from husd.outputprocessor import OutputProcessor


class ReviewOutputProcessor(OutputProcessor):
    @staticmethod
    def name():
        return "review"
    @staticmethod
    def displayName():
        return "Manually Review Every Output Path"

  4. At the bottom of the file, define a function called usdOutputProcessor() that takes no arguments and returns your class whenever it’s called.

    The existence of this function is what marks this module as implementing an output processor (Houdini looks for modules containing this function), and how Houdini creates instances of your output processor object. Each time a USD save operation is started, a new instance of your output processor is created. This instance is used for the duration of that save operation.

    import hou
from husd.outputprocessor import OutputProcessor


class ReviewOutputProcessor(OutputProcessor):
    @staticmethod
    def name():
        return __class__.__name__
    @staticmethod
    def displayName():
        return "Display the list of output files"


# Must have: module-level function to return a processor instance
outputprocessor = ReviewOutputProcessor()
def usdOutputProcessor():
    return outputprocessor

  5. Override API methods to implement your output processor. See the output processor API below.

    import hou
from husd.outputprocessor import OutputProcessor


class ReviewOutputProcessor(OutputProcessor):
    @staticmethod
    def name():
        return "displayoutputfiles"
    @staticmethod
    def displayName():
        return "Display the list of output files"

    def processSavePath(self, asset_path, referencing_layer_path, asset_is_layer):
        # Make the asset path absolute
        asset_path = hou.text.abspath(asset_path)

        # This processor asks the user to manually rewrite every file path.
        # This is just an example, don't do this! It would be annoying!
        return hou.ui.readInput(
            message="Rewrite this output file path if you want",
            initial_contents=asset_path,
            buttons=("OK",),
        )

    def processReferencePath(self, asset_path, referencing_layer_path, asset_is_layer):
        # Make file path pointers relative to the source file's location
        return hou.text.relpath(asset_path, referencing_layer_path)

    def processReferenceExpression(self, asset_expression, referencing_layer_path, asset_is_layer):
        # Leave stage variable expressions unchanged
        return asset_expression


# Must have: module-level function to return the processor class
def usdOutputProcessor():
    return ReviewOutputProcessor

Output Processor method API

@staticmethod hidden() → bool

If this method returns True, this processor is not included in the list of output processors shown to the user. The default implementation returns False.

@staticmethod name() → str (REQUIRED)

Returns a unique name for the processor.

Note

You must override this method in your subclass, otherwise it will raise a NotImplementedError exception when Houdini tries to instantiate your class.

@staticmethod displayName() → str (REQUIRED)

Returns a label to describe the processor, in the list of output processors shown to the user.

The label should describe the function/of the processor, for example: Save Paths Relative to Output Path.

Note

You must override this method in your subclass, otherwise it will raise a NotImplementedError exception when Houdini tries to instantiate your class.

@staticmethod parameters() → str

Returns a string containing Houdini “dialog script” describing the parameters this processor shows to the user for configuration.

The default implementation returns the script for an empty parameter group, so if your processor doesn’t need any parameters, you don’t need to override this method.

You can generate the dialog script by building up a hou.ParmTemplateGroup with hou.ParmTemplate objects inside, and then returning the value from hou.ParmTemplateGroup.asDialogScript.

group = hou.ParmTemplateGroup()
group.append(hou.StringParmTemplate(
    "texturedir",
    "Texture Directory",
    string_type=hou.stringParmType.FileReference
))
return group.asDialogScript()

The internal names of any parameters you create here must be unique among all other parameters on the render node, so you probably want to use a naming scheme like ‹modulename›_‹parmname› (where ‹modulename› is the name of the Python module under husdplugins/outputprocessors).

stageVariables(self) → dict

This method should return the stage variables that are set on the stage being saved. The default implementation returns the stage_variables dictionary passed to the base class implementation of the beginSave method.

beginSave(self, config_node, config_overrides, lop_node, t, stage_variables)

Called when a render node using this processor starts to write out files. This gives you the chance to read parameter values (either the configuration parameters added by parameters(), or the render node’s own parameters, depending on what information you need). You should always call the base class implementation of this method, which will store the config_node, lop_node, and t parameters into self.config_node, self.lop_node, and self.t respectively for use in the processing methods.

config_node

A hou.Node object representing the render node.

config_overrides

A dict of values that should be used to override whatever may be set on the node. The OutputProcessor.evalConfig method can be used to query a configuration value, accepting the override value from this dictionary if available, otherwise falling back to evaluating the matching parameter on the config_node.

lop_node

A hou.LopNode object representing the LOP node that generated the stage being saved.

t

A floating point value representing the time (in seconds) along the timeline which the node is rendering. If you read parameter values from the node, you should use Parm.evalAtTime(t) in case the parameter is animated.

stage_variables

A dictionary containing the stage variables set on the root layer of the stage being written to disk. This parameter should be passed to the base class implementation of this method so this dictionary can be accessed from the processReferencePath and processReferenceExpression methods through the stageVariables method.

processSavePath(self, asset_path, referencing_layer_path, asset_is_layer) → str

Called when the render node needs to determine where on disk to save an asset. The asset_path is the file path as Houdini knows it (for example, from USD metadata or a Houdini parameter).

This should return an absolute path to where the asset should be saved. (If you return a relative path from this method, it will be relative to the current directory (os.getcwd()) which is probably not what you want.)

asset_path

The path to the asset, as specified in Houdini. This string comes with expressions and environment variables (such as $HIP) expanded already, so if you want to compare to another path, you should also expand that path (for example, with os.path.expandvars() or hou.text.expandString()).

referencing_layer_path

The processed save path of the layer file that is referencing this asset. This parameter allows assets to be saved in specific locations relative to the layer that brings them into the scene. This can be particularly useful for non-layer files that may get written out during the save process, such as volume files.

asset_is_layer

A boolean value indicating whether this asset is a USD layer file. If this is False, the asset is something else (for example, a texture or volume file).

Note

Only the first processor in line sees the asset_path as it was originally in Houdini. For all other processors in line, they receive the absolute path the first call returned.

processReferencePath(self, asset_path, referencing_layer_path, asset_is_layer) → str

Called when the render node needs to write a file path pointing to an asset (to sublayer or reference in the file).

asset_path

The path to the asset. If the asset is being created as part of the USD save process, this will be the final save path for the asset after running all output processors. This value will always be a full path.

referencing_layer_path

The processed, absolute save location of the layer file containing the reference to the asset. You can use this to make the path pointer relative. (For example, hou.text.relpath(asset_saved_path, referencing_layer_path)).

asset_is_layer

A boolean value indicating whether this asset is a USD layer file. If this is False, the asset is something else (for example, a texture or volume file).

processReferenceExpression(self, asset_path, referencing_layer_path, asset_is_layer) → str

Called when the render node needs to write a file path pointing to an asset (to sublayer or reference in the file).

asset_path

The path to the asset. If the asset is being created as part of the USD save process, this will be the final save path for the asset after running all output processors. This value will always be a full path.

referencing_layer_path

The processed, absolute save location of the layer file containing the reference to the asset. You can use this to make the path pointer relative. (For example, hou.text.relpath(asset_saved_path, referencing_layer_path)).

asset_is_layer

A boolean value indicating whether this asset is a USD layer file. If this is False, the asset is something else (for example, a texture or volume file).

processLayer(self, layer)

Called immediately prior to writing a layer file to disk. The layer parameter is a pxr.Sdf.Layer object which is editable, and can be modified in any way. The default implementation does not modify the layer.

Return True if this method modifies the layer, otherwise return False.

Adding an output processor in a script

  • Houdini will use an output processor on a USD node if the node instance has a spare checkbox parameter named enableoutputprocessor_‹modulename› that is turned on (where ‹modulename› is the name of the Python module under husdplugins/outputprocessors).

    For example, if you implemented your class in $HOUDINI_USER_PREFS_DIR/husdplugins/outputprocessors/myprocessor.py, then you would need to have a spare checkbox parameter named enableoutputprocessor_myprocessor on a node to activate the processor for that node.

  • This means you can enable a processor by creating the correct parameter and turning it on, such as in a script, even if the processor is hidden from the processor list in the user interface (see the hidden() method above).

    You can disable a processor by turning off the spare checkbox, or deleting the spare parameter.

  • If the output processor has additional parameters for configuring it (see the parameters() method above), you can create and fill in those parameters in a script as well.

Saving animation

The Flush Data After Each Frame parameter on the USD render node controls whether it writes data out after each frame of data is generated. This feature can be used to create individual files containing the data for each frame, or arbitrarily large files containing time sample data across all frames.

When the USD render node writes out a frame range with Flush Data After Each Frame off:

  1. For each frame the ROP will generate a set of layers ready to be saved to disk, but which still exist in-memory.

  2. It uses USD Stitch to combine the generated frames with previously cooked frames in-memory.

If the LOP Network is generating a lot of data, this can quickly use a lot of memory (even though the stitch operation does not duplicate data which is the same from frame to frame).

If you find writing out animated USD runs out of memory in Houdini, you can enable this option to limit Houdini to only have a single frame’s data in memory at any one time. The result may take longer to write to disk, and the final file size may be larger than with this option disabled. But the amount of data that can be written will not be limited by the computer’s available memory.

Another approach is to write out a sequence of USD files each containing a single time sample of data, then using the USD Stitch Clips ROP to generate a USD value clip. This approach only works if there is an isolated branch in the scene graph tree where the large data set exists, and the data for this branch can be written to a separate USD file.

Other ways to write out USD

  • You can use Python (interactively in a Python shell or procedurally in a Python Script LOP) to write out individual layers.

  • You can right-click a node, open the LOP Actions sub-menu, and choose Inspect flattened stage or Inspect active layer. These menu items open a viewer window showing the stage/layer as usda code. You can save the usda code to a file from this window.

Solaris and Karma

USD

Geometry

  • SOP Geometry I/O

    Details of how Houdini converts SOP geometry to USD, and how you can control the process.

  • Component Builder

    The Component Builder tool puts down a network snippet for creating a USD model from SOPs, with support for materials, variants, payloads, and layering.

Layout

  • Stage Manager

    How to work with the Solaris stage effectively.

  • Edit node

    Interactively transforms prims in the viewer. Can use physics collisions to position props realistically.

  • Layout node

    Provides tools for populating a scene with instanced USD assets. You can place individual components, paint/scatter components in different ways using customizable brushes, and edit existing instances.

  • Custom Layout Brushes

    How to create layout brush digital assets you can use to customize the behavior of the Layout LOP.

Look Development

  • MaterialX

    Houdini has VOP node equivalents of the MaterialX shader nodes. You can build a shader network using these nodes, or import an existing MaterialX-based shader, and use them with Karma (Houdini’s USD renderer).

  • UDIM paths

    You can encode different tiles of a texture space into different texture files, each with its own resolution. You can then specify a texture filename such as kaiju.exr, and Houdini will replace the token with the specific tile address at load time.

  • Shader translation framework

    Describes the Solaris shading framework, including shader node translation to USD primitives.

Karma User Guide

Karma basics and workflows