Preprocess OIDN TOP node

Processes training and validation datasets for OIDN model training.

On this page	Overview Installation: Parameters Advanced
Since	20.5

Overview ¶

This node is a wrapper around the OIDN preprocessing script used to preprocess training and validation datasets compliant with the OIDN dataset naming scheme.

Installation: ¶

The OIDN scripts require an additional installation of torch (PyTorch) to hython.

Firstly, we intitialize the hython environment by navigating to the Houdini installation directory.

Linux

cd /opt/hfsx.x.x

Mac

cd /Applications/Houdini/Houdinix.x.x/Frameworks/Houdini.framework/Resources

Windows

cd "C:\\Program Files\\Side Effects Software\\Houdini x.x.x"

Then we source the houdini_setup script, followed by an installation of torch to hython.

source houdini_setup
hython -m pip install torch

Parameters ¶

Input Features

The set of input features of the dataset to preprocess for training. The following image features are supported:

hdr

color (high dynamic range) with file extension .hdr.exr

ldr

color (low dynamic range) with file extension .ldr.exr

sh1

color (normalized L1 spherical harmonics) with file extensions .sh1x.exr, .sh1y.exr, and .sh1z.exr

alb

albedo with file extension .alb.exr

nrm

shading normal with file extension .nrm.exr

All input features are assumed to be noisy, including the auxilliary features (e.g. albedo, normal). The auxiliary feature images are optional inputs which usually improve denoising quality and preserve more details.

Clean auxilliary features

This parameter should be enabled when the auxiliary images are noise-free, in which case the reference auxiliary features are used in training instead of the various noisy auxiliary images.

Filter

The filter to train. The filters are:

RT

Generic ray tracing denoising filter suitable for denoising images rendered with Monte Carlo ray tracing methods like unidirectional and bidirectional path tracing.

RTLightmap

Variant of RT filter optimized for denoising HDR and normalized directional lightmaps and does not support LDR images.

The choice of filter determines the HDR/LDR transfer function to be used

Output Directory

The directory to output preprocessed results.

Training Dataset

The name of the training dataset folder.

Validation Dataset

The name of the validation dataset folder.

Data Directory

The directory of the training and validation data.

Device

The device (e.g. CPU, GPU) to use for training computations. The options are: cpu, cuda, and mps.

Device ID

The specified device to use if there are multiple devices of the same kind available (e.g. multiple GPUs).

Device Count

The number of devices to use for data-parallel execution for faster performance.

Advanced ¶

These are advanced parameters that provide finer control over the behavior of the preprocessing.

Transfer Function

The HDR/LDR transfer function to be used.

Deterministic

Make computations deterministic, resulting in slower performance.

See also

Train OIDN

TOP nodes

Attribute Array

Creates and modifies array attributes on work items.
Attribute Copy

Copies attributes from work items in one branch onto work items in another branch.
Attribute Create

Creates or sets an attribute on all incoming work items.
Attribute Delete

Removes attributes from work items.
Attribute Dictionary

Modifies dictionary attributes by adding key/value pairs, removing keys, or merging existing dictionaries
Attribute Promote

Copies attributes and fields between work items, global attributes, and output files
Attribute Randomize

Create randomly-generated integer and float attributes on work items.
Attribute Reduce

Reduces array attribute values to a single value.
Attribute Rename

Renames attributes on work items.
Attribute from File

Parses attribute values from a file into work items.
Attribute from String

Parses attribute values from a string, such as a file name.
Block Begin Feedback

Starts a feedback loop. TOP nodes within the block execute serially, optionally looping for each incoming work item.
Block End Feedback

Ends a feedback loop. TOP nodes within the block execute serially, optionally looping for each incoming work item.
CSV Input

Copies data from a CSV file into work item attributes.
CSV Output

Writes work item attributes to a CSV file.
Deadline Scheduler

The PDG scheduler for Thinkbox’s Deadline software.
Download File

Downloads the contents from one or more URLs into files.
Environment Edit

Edits the variables set in the environment work item command lines execute in.
Error

Issues warnings or errors when a condition is met.
FFmpeg Encode Video

Encodes a sequence of still images as a video file.
FFmpeg Extract Images

Extracts a sequence of still images from a video file.
File Compress

Compress files into an archive.
File Copy

Copies a file from one location to another, either at runtime or whenever the node generates.
File Decompress

Decompresses archive files specified by incoming work items into individual files.
File Pattern

Creates work items based on files that match a certain pattern.
File Range

Creates a range of work items based on files that match a certain pattern.
File Remove

Deletes a file at a specified path.
File Rename

Renames or moves a file.
Filter by Attribute

Filters upstream work items by their attribute names or values
Filter by Expression

Conditionally filters upstream work items.
Filter by Range

Filter upstream work items that fall within a specified frame or attribute range.
Filter by State

Filters upstream work items by their state
Generic Generator

Generates work items with no attributes that run a command line.
Geometry Import

Loads points or primitives from a SOP or geometry file into work item attributes or a temporary file.
HDA Processor

Creates work items that cook a digital asset
HQueue Scheduler

Schedules work items using HQueue.
Houdini Block Begin

Starts a Houdini service block
ImageMagick

Provides easy access to ImageMagick functionality such as mass image convert, resize, and image mosaics.
In Process Scheduler

Controls the scheduling of in-process work items.
Invoke

Invokes a compiled block on input geometry
JSON Input

Extracts data from JSON files and creates attributes
JSON Output

Performs various operations that produce JSON output
Labs Archive Project

Utility TOP to Collect and Archive current HIP file dependencies
Labs Concatenate Text

Merges one or more text files into a single file.
Labs Cut Geometry to Partitions

Cuts up incoming geometry
Labs Data Diff

Diffs a variety of geometry, text and image files.
Labs Extract Image Metadata

Extracts image metadata, such as the resolution, bit depth, color model, etc.
Labs File Cache Filter

Partition work items used for file caching
Labs File Cache Partitioner

Partitions and generates work items based on range and wedge values for file cache.
Labs Filter by Value

Filters one or multiple work items by specific attribute values.
Labs Generate from Imageplanes

Generates work items based on image planes found in an image
Local Scheduler

Schedules work items on the local machine.
ML Regression Train

Trains a neural network for regression.
Make Directory
Maya Block Begin

Starts a Maya service block
Merge

Merge all upstream work items
Node Pattern

Creates work items based on matched nodes.
Nuke Block Begin

Starts a Nuke service block
Null

Does nothing
OP Notify

Notify an OP node that some TOP work has completed
OpenImageIO

Provides easy access to the OpenImageIO’s command line tool, also known as oiiotool, to perform operations on images such as color space conversion, color mapping, resizing, text overlay, and box overlay.
Output

Subnet output
Partition by Attribute

Partitions work items based on their attributes
Partition by Bounds

Partitions source items spatially using the bounding items.
Partition by Combination

Partitions work items into pairs, triples, etc
Partition by Comparison

Partitions work items using existing comparisons
Partition by Expression

Partitions work items based on an expression
Partition by Frame

Partitions work items based on their frame
Partition by Index

Partitions work items based on their index
Partition by Iteration

Partitions work items based on their feedback loop iteration.
Partition by Node

Partitions work items based on their node
Partition by Range

Partition work items based on range values
Partition by Tile

Partitions work items spatially using axis-aligned bounding boxes.
Perforce

Execute Perforce commands through PDG
Python Block Begin

Starts a Python service block
Python Partitioner

Partitions work items using a Python script
Python Processor

Generates work items using a Python script.
Python Scheduler

A Python-based programmable Scheduler for PDG.
Python Script

Creates work items that run a Python script.
Python Virtual Environment

Creates a Python virtual environment using TOPs
ROP Alembic Output

Creates work items that cook an embedded Alembic ROP node
ROP Composite Output

Creates work items that cook an embedded Composite ROP node
ROP FBX Output

Creates work items that cook an embedded FBX ROP node
ROP Fetch

Creates work items that cook a ROP node or network.
ROP Geometry Output

Creates work items that cook an embedded Geometry ROP node
ROP Image Output

Creates work items that cook an embedded Image ROP node
ROP Karma Render

Creates work items that cook an embedded Karma ROP node.
ROP Mantra Render

Creates work items that cook an embedded Mantra ROP node
ROP OpenGL Render

Creates work items that cook an embedded OpenGL ROP node
ROP USD Output

Creates work items that cook an embedded USD ROP node
Range Extend

Extend the frame range of upstream work items and adds new work items as necessary
Range Generate

Generates work items for the specified range.
Remote Graph

Connected to a remotely executing TOP graph and displays the nodes from that graph
Render IFD

Creates work items that render an IFD file with Mantra
SQL Input
SQL Output
Send Email

Sends an email.
Service Block End

Ends a service block.
Service Block Send

Sends code to a service block to execute.
Service Create

Creates a PDG service.
Service Delete

Deletes a PDG service.
Service Reset

Resets a PDG service.
Service Start

Starts a PDG service.
Service Stop

Stops a PDG service.
ShotGrid Create

Creates ShotGrid entities
ShotGrid Delete

Deletes entities from ShotGrid
ShotGrid Download

Downloads ShotGrid attachments
ShotGrid Find

Finds ShotGrid entities
ShotGrid Server Begin

Starts a persistent ShotGrid command server
Shotgun Update

Updates shotgun entities
Shotgun Upload

Uploads files to shotgun
Sort

Sorts work items by a list of attributes
Split

Splits upstream items in two
Split by Count

Splits upstream items into two using a fixed count
Subnetwork

Container for TOP nodes
Switch

Switch which between network branches
TOP Fetch

Cooks another TOP network.
TOP Fetch Input

Input to a TOP fetch-ed network
Text Output

Writes text to a new file or appends to an existing file.
Tractor Scheduler

Schedules work items using Pixar’s Tractor.
Train OIDN

Trains an OIDN model using preprocessed training and validation datasets.
URL Request

Creates work items that request data from a URL.
USD Add Assets to Gallery

Adds USD assets to the Asset Gallery.
USD Analyze

Fetches metadata, dependencies, and external references for a LOP Node or file.
USD Import Data

Creates work items from prims found in a USD stage.
USD Import Files

Creates work items from file references found in a USD stage.
USD Modify Paths

Modify asset path attribute values.
USD Render Files

Creates work items that render a USD file.
USD Render Scene

Creates work items that render a USD file or LOP network
Wait for All

Waits for all upstream work items to complete.
Wedge

Creates work items with varying attribute values.
Work Item Expand

Expands file lists or partitions into multiple work items
Work Item Import

Import work items from a .json file or another TOP node.
Xml Input

Extracts data from XML into Work Item attributes