Houdini 20.5 Nodes Geometry nodes

ONNX Inference geometry node

Applies inference from an ONNX Machine Learning model.

On this page

Machine learning consists often of two distinct stages. Training is the process of building a model that can solve a problem. Inference is using the model to actually solve the problem.

This node performs inference. It uses a pre-trained model on the input volumes and point data to generate the machine-learned solution.

There are many systems to do training and these often require significant dependencies. However, often the resulting model consists of only a few simple operations. The ONNX format is a file format and inference engine that allows one to use models trained in PyTorch or TensorFlow in a common framework.

An ONNX inference model consists of a number of inputs and outputs. Each input and output is a tensor; which is a fancy way of saying multi-dimensional data. For dimension 1 this is an array; for dimension 2 an image; for dimension 3 a volume; and higher dimensions can be generalized from there.

Since Houdini doesn’t have objects of higher dimension than 3, the setup is that one specifies input data that fills a large 1-d array. This is then reshaped into the specified high-dimensional tensor.

For an example that shows how linear blend skinning can be improved on by learning from random poses, download the ML Deformer files from the Content Library.

For an example that shows how you can use TOPs to generate terrain data, and then use that data to train an ML model that’s able to produce new terrain given a simple sketch from an artist, download the ML Terrain files from the Content Library.

Note

ONNX models can contain inputs and outputs of types other than tensor, but currently this node only supports tensors.

Parameters

Model File

The name of a .onnx file to load. The inputs and outputs must match the settings of the ONNX file.

Reload Model

Force a reload of the .onnx file.

Setup Shapes from Model

This callback reads the .onnx file for how many inputs and outputs it has, and what the specified shapes are. It initializes them properly. Dynamic axes are given -1 for shape size.

Keep Input

Works in place on the provided geometry. This requires the number of points generated will match the number of points input, but allows for per-point inferences to work in-place.

Number of Inputs

Name

The semantic name of the input. Actual wiring to the ONNX model is done by index, not by name, but this can be used to label them. The Setup Shapes from Model button will populate this with the input name from the model.

Input Tensor Shape

The shape of the input. -1 is used for dynamic axis. The first 0 axis terminates the shape. The total shape must match the number of elements in the input.

Input Type

The source for the data for this input.

Volume

All volumes matching the name are concatenated for the input. They must all be the same resolution and tuple size.

Detail Attribute

The specified detail attribute is treated as the 1d source.

Point Attribute

This attribute on all points will be concatenated together to form the input.

Primitive Attribute

This attribute on all primitives will be concatenated together to form the input.

Volume Order

When unrolling a volume into 1d tensor one can place the x-samples close together or the z-samples.

ZYX

Z-major extraction of the volumes. Neighboring x-samples are placed together.

XYZ

X-major extraction of the volumes. Neighboring z-samples are placed together.

Collate Channels Separately

When converting multiple volumes or vector volumes into 1d tensor, they can be interleaved or the volumes placed one after another. With this off, an RGB volume would be stored rgbrgbrgb. With it on, it is instead stored rrrgggbbb. In particular, when it is off the final dimension of the shape should be the number of volumes and components; with it on, the first dimension should be it.

Data

The names of the volumes or attributes to extract. This can be multiple attributes in which case they will be collated together in the created tensor. Ie, if you specify P Cd, the tensor will be filled with P.x P.y P.z Cd.x Cd.y Cd.z, repeated for each point.

The order within the tensor will match the order listed here. For volumes, globbing can be done, in which case within the glob the volumes will be collated according their order in the geometry.

Number of Outputs

Name

The semantic name of the output. Actual wiring to the ONNX model is done by index, not by name, but this can be used to label them.

Output Tensor Shape

The shape of the input. -1 is used for dynamic axis. The first 0 axis terminates the shape. The total shape must match the number of elements generated by the model.

Output Type

The destination for the data for this output.

Volume

Volumes are created and filled with the tensor data, each volume in turn.

Detail Attribute

The specified detail attribute will be filled with the flattened tensor.

Point Attribute

The output tensor will be written to each point’s attribute in turn.

Volume Order

When unrolling a volume into 1d tensor one can place the x-samples close together or the z-samples.

ZYX

Z-major extraction of the volumes. Neighboring x-samples are placed together.

XYZ

X-major extraction of the volumes. Neighboring z-samples are placed together.

Collate Channels Separately

When converting multiple volumes or vector volumes into 1d tensor, they can be interleaved or the volumes placed one after another. With this off, an RGB volume would be stored rgbrgbrgb. With it on, it is instead stored rrrgggbbb. In particular, when it is off the final dimension of the shape should be the number of volumes and components; with it on, the first dimension should be it.

Data

The name of the volume or attribute to create. This can specify multiple attributes or volumes, in which case the last dimension of the tensor is split between them in the given order. By default each is a float-tuple size, but : can be used to specify a specific size, such as depth N:3 to convert a 4-float wide tensor into the scalar depth and vector N attributes.

Max Batch Size

Usually models are configured to have the first dimension be dynamic and control batches. This is often done to improve training, but in many cases, such as point-based models, it is also very useful for inference.

Dynamic axes are usually expanded to the size of the input data and run in a single pass. This can require a lot of memory, however, so this option allows limiting the number of elements that can be processed at once, instead the model will be re-run on successive batches.

Note

Only the first dimension eligible for batching is handled in this manner.

Enable CUDA

Enables performing inference on the GPU (requires CUDA and cuDNN).

Locals

See also

Geometry nodes