Lookup Tables

A lookup table (LUT) can be used to color correct images. Houdini has a native LUT format with the extensions .lut (text) and .blut (binary). You can also add other LUT formats using a conversion script or program, and the LUTio table (found in $HH).

Loading, Saving and Using a LUT

PXL_Lookup is responsible for handling all lookup tables in Houdini. The easiest way to use it is via the following methods: load(), save(), evalLUT(), setLUT(), set3DLUT() and applyLUT().

To read a LUT from disk, create a PXL_Lookup object and call PXL_Lookup::load() one it (variables will be expanded in the filename). If the operation succeeded, load() returns true.

To evaluate a LUT for a single color, call PXL_Lookup::evalLUT(). To apply a LUT to a full image, place the image in a PXL_Raster and call PXL_Lookup::applyLUT().

To create a LUT, you need to determine if you are going to produce a 1D or a 3D LUT. A 1D LUT has data that looks like channels. A color component indexes the LUT, and the value of the LUT at that index is used as the output. A 3D LUT uses the entire color to index into a 3 dimensional array - lut[r,g,b]. The index contains an RGB color which is used as the output. 3D LUTs can represent almost any color correction, including all those that 1D LUTs can do. They are much larger than 1D LUTs, though. A 3D LUT is useful for color corrections that adjust hue and saturation. 1D LUTs can only adjust individual component levels (like gamma, brightness and contrast).

Once you've decided upon which type of LUT to create, use either the PXL_Lookup::setLUT() method (for 1D LUTs) or PXL_Lookup::set3DLUT() (for 3D LUTs). The setLUT() method expects data in a CL_Clip, from 1 to 4 tracks in size, of any channel length. The length of the clip determines the number of samples in the LUT.

The set3DLUT() method expects a uniform cube of FP32 RGB vectors - size^3 of them. The first entry is for (0,0,0). The size'eth entry is for (0,1,0), and the size^2'th entry is for (0,0,1) – in other words, sample red first, then green, then blue.

To save the LUT, call PXL_Lookup::save(). You can save native Houdini LUTs (.lut or .blut), or any of the LUT formats in the LUTio table by using the appropriate filename extension.

The PXL_Lookup class also has a number of methods for querying LUT properties and modifying the data and its attributes.

The Houdini LUT Format

The Houdini LUT format supports simple 1D LUTs, which are like channels, and 3D LUTs, which are color cubes like 3D textures. As of Houdini 10, both linear and logarithmic LUTs are supported.

1D LUTs

A 1D LUT maps the input values of a single component (R, G, B or A) to new values in the same component. It can represent simple color corrections, like gamma correction, contrast adjustment and brightness changes. The Houdini LUT format supports one to four component tables per LUT. Additionally, a single LUT can be used for all components.

The header of a 1D linear LUT for a text (.lut) Houdini LUT is:

Version     1
Format      [format]
Type        [type]
From        [in_start] [in_end]
To          [out_start] [out_end]
Black       [black_point]
White       [white_point]
Length      [LUT entries]

The version of all linear LUTs can be 1 for backwards compatibility with older versions of Houdini.

The format parameter specifies which data formats the LUT should be applied to. Generally, this is left at "any", but in cases where it should only be applied to images with specific formats, it can be set to any of the following: i8 (8b int), i16 (16b int), i32 (32b int), f32 (32b FP), f16 (16b FP) or any.

The type parameter defines which channels are affected by the LUT. It can take any of the following values:

R - 1 channel for red only
G - 1 channel for green only
B - 1 channel for blue only
A - 1 channel for alpha only
RGB - 3 channels, 1 each for red, green and blue
RGBA - 4 channels, 1 for each component
C - 1 channel applied to red, green and blue
All - 1 channel applied to all components

There are also a couple of types for 3D LUTs (see below).

The From parameter specifies the input range of the LUT. 0.0 is black, and 1.0 is white. Values before 'in_start' or after 'in_end' will be clamped. Values falling inside the LUT's range will be remapped to new values. Normally these values are 0.0 and 1.0, though LUTs can be defined for a higher range, like 0.0 to 10.0 (for HDR images, though log LUTs are better suited for HDR images).

The To parameter specifies the units that the LUT entries are in. For 8b notation (0-255), this would be "0 255". For normal floating point, this is left at "0 1".

The Black and White point parameters are obsolete, and should be left at 0 and 1.

The Length parameter specifies how many LUT entries there are. There must be at least 2. For a table of 200 values, this would be 200.

After the header, the actual tables begin.

LUT:
[chan] {
   [val]
   [val]
     :
   [val] }
[chan] {
   [val]
   [val]
     :
   [val] }

"chan" specifies the component name of the next lookup table. For RGB or RGBA LUTs, this should be R, G, B or A. Otherwise, C is generally used.

There should be "Length" values in each table. The values are separated by newlines. There should only be as many tables as LUTs specified by the Type parameter in the header (either 1, 3 or 4).

Each color value should be sampled at position (p) for LUT index (idx):

p = idx * (in_end - in_start) / (length-1)

Log 1D LUTs

A 1D LUT can also be sampled in log space, which makes it more useful for HDR images. These LUTs are similar in structure to normal Linear LUTs, except that the Version must be at least 3, and the header should include the line:

Sampling Log

The From range must have a positive starting value and ending value, such as "0.001 10". Zero cannot be used as a starting point, nor can negative values (since the log of these values is undefined). The values are in linear terms, so a range of (0.001, 10) will transform all linear values between 0.001 and 10

Log color sampling is done at position (p) for LUT index (idx):

p = e^(idx * (ln(in_end) - ln(in_start)) / (length-1) + log(in_start))

where ln() is the natural log function (log base e) and e is 2.718282. The actual base of the log doesn't matter, as long as it is the same throughout the computation.

3D LUTs

A 3D LUT can represent more complex color corrections than a 1D LUT since an input value in one component is capable of affecting the other two components. A 3D LUT is represented as a color cube (an NxNxN vector3 array).

A 3D LUT can be sampled in linear space, log space, or at custom intervals using a 1D prelut. The header the same as a 1D LUT, with a few changes:

Version      2
Format       any
Type         3D
From         0 1
To           0 1
Black        0
White        1
Length       [cube_size]

The cube_size parameter specifies the length of one of the sides of the cube, not the number of entries in the color cube (ie, 16 for 16x16x16). The 3D LUT then looks like:

LUT:
{
   [r] [g] [b]
   [r] [g] [b]
        :
   [r] [g] [b]
}

There is one table of cube_size^3 entries of RGB colors. Normally these are floating point values, though you can specify them in other units as well (0-255, though change the 'To' parameter in the header to "0 255").

The color cube is sampled in scanlines of the planes of the cube. When flattening the cube into a list of colors, red is incremented first, then green (scanline), and finally blue (plane):

i=0
for b = 0 to N-1
   for g = 0 to N-1
      for r = 0 to N-1
         cubedata[i] = sample (r,g,b)
         i = i+1

Log 3D LUTs

A 3D LUT can also be sampled in log space. A log 3D LUT has the same header as a normal 3D LUT, except that the Version must be 3 and the following line must be added:

Sampling Log

The values in the 'From' parameter must also be positive. The start cannot be zero.

Sampling is done in the same manner as a 1D LUT for each component of RGB:

p = e^(idx * (ln(in_end) - ln(in_end)) / (length-1) + log(in_start))

3D LUT with a 1D Prelut

A 3D LUT can be have its input color modified by a 1D prelut, which allows for custom color space sampling. A 3D prelut LUT has a similar header to a 3D LUT, except that its version must be 3, and the type must be "3D+1D". The Length parameter also has two values, the first for the 3D LUT and the second for the 1D prelut. The prelut size must be less than or equal to the cubesize^2.

In the LUT section, the prelut is listed first, followed by the 3D LUT.

Version      3
Format       any
Type         3D+1D
From         0 10
To           0 1
Black        0
White        1
Length       [cube_size] [prelut_size]
LUT:
Pre {
 [val]
 [val]
   :
 [val]
}
3D {
   [r] [g] [b]
   [r] [g] [b]
        :
   [r] [g] [b]
}

The 'From' parameter defines the range of values modified by the prelut. The prelut must contain values between 0.0 and 1.0. Each component is run through the prelut, which are then mapped to the 3D LUT (0 to N-1). There can only be one prelut.

Note: You can use color correction COPs to generate LUTs (color correction COPs are automatically colored blue when created). Select "Save LUT" from a color correction COP to generate a LUT. Based on the color correction chain, a 1D or 3D LUT will be created. You can specify whether the sampling is done in linear or log space.

Using the LUTio table to add more LUT formats

The LUTio table allows you to add external LUT formats to Houdini. You must have a conversion program or script written to covert to and from the Houdini LUT format. Once this is done, you can add your format into the LUTio table by adding a line such as:

# extension read LUT command write LUT command

.ext "toHoudiniLUT '%s' stdout.lut" "fromHoudiniLUT stdin.lut '%s'"

's' is replaced by the name of the external LUT being loaded. The program or script reads or writes directly from a pipe opened by Houdini (or MPlay) by reading from stdin or writing to stdout.

For the read LUT command, the script or program should translate the external LUT into an text Houdini LUT (.lut). Any of the above LUT formats (1D linear or log, 3D linear, prelut or log) can be written.

The write LUT command does the opposite. You need to parse the Houdini LUT passed to you and write out the external LUT. The Houdini LUT passed could be in any of the above formats. If your external LUT format does not support a Houdini LUT format passed to it, exit the script or program with an error code of 1.

Binary Houdini LUT format

The Houdini LUT format also supports binary encoding (.blut) which offers faster load times and smaller file sizes. It is more complicated to write and read, however, so you should only attempt to read & write these LUTs if you are very familiar with computer data representations.

All data elements larger than 1 byte are written in big endian format, used by CPUs such as MIPS and PowerPCs. Intel CPUs use little endian format, so if you are compiling on that platform, you must swap bytes around:

16b word:
   x = (((z & 0xff) << 8) | ((z >> 8) & 0xff))
32b word:
   x = ((z & 0xff) << 24) | ((z & 0xff00) << 8) \
     | ((z >> 8) & 0xff00) | ((z >> 24) & 0xff)
   where 'z' is the input word, and x is the byte-swapped output.

If you are using the HDK, you can find methods to swap bytes in UT_Endian.h (UTtomips()). You need to do the byte swapping whenever you read or write a 16b or 32b word from or to a LUT file (including floating point values). For arrays of 16b or 32b words, you only need to byte swap the individual words themselves (ie, 2 16b ints would require 2 16b swaps, not 1 32b).

You should also be familiar with the 16b floating point format used by graphics cards and OpenEXR, and have code that can convert it to floating point if you need to write the values as ASCII or 32b FP. If the external format supports 16b FP, you can pass them through as if they were 16b ints.

1D Linear and Log LUTs

Header:

Field	Format	Values
Magic number	4 bytes	'H' 'L' 'U' 'T'
Version number	1 32b int	0x1 (linear) or 0x3 (log)
Format Filter	1 32b int	see Data Format Filter table below, generally 0x5 (apply to all)
Type	1 32b int	see 1D LUT Type table below
Domain	2 32b floats	start, end of LUT range
Range	2 32b floats	range of color units, FP is 0-1
Black	1 32b float	black point (0.0)
White	1 32b float	white point (1.0)
Length	1 32b int	Lookup table size, >=2
Number of LUTs	1 32b int	1, 3 or 4
FP Precision	1 byte	'h' (16b FP) or 'f' (32b fP)

After the header, for each LUT, the following section is written:

Field	Format	Values
LUT values	'length' 16b FP or 32b FP	Lookup table values

After the last LUT section is written, the file is complete. The data type of the values is specified by the "FP Precision" field above.

1D LUT Type Table:

LUT Channel Type	Value
Red only	0x001
Green only	0x002
Blue only	0x004
Alpha only	0x008
RGB, 3 LUTs	0x017
RGBA, 4 LUTs	0x01F
RGB, 1 LUT	0x027
RGBA, 1 LUT	0x02F
Log sampling*	0x100

To specify log sampling, OR the above value with 0x100. For example, an RGB LUT with log sampling would be 0x017 | 0x100 = 0x117.

Data Format Filter table:

LUT Data Format	Value
8b Int	0
16b Int	1
32b Int	2
32b FP	3
16b FP	4
Apply to all types	5

3D Log, Prelut and Linear LUTs