|
HDK
|
The role of a Houdini Digital Asset (HDA) is to define a custom operator. Usually, an HDA encapsulates a node network that, as a whole, performs an operation intended for this custom operator. However, an HDA may specify its operation as script, for example, in case of a Python SOP or a GLSL shader asset. In addition to the network or code, an HDA can specify several additional bits of information for the operator type, such as a label, icon, and also creation script, shelf tools, and help description.
Digital assets are stored on disk in an Operator Type Library file. The OTL can contain many HDAs, but it can also store other operator definitions, such as scripted operators, though that is less common. When Houdini starts, it uses the HOUDINI_OTLSCAN_PATH to look for the OTL files or HOUDINI_OPLIBRARIES_PATH to look for OPlibrary files that specify the list of OTLs to load. In Houdini, the OP_OTLManager is responsible for managing the digital assets, and it holds a list of OP_OTLLibrary instances that in turn contain a list of OP_OTLDefinition objects, each corresponding to an HDA.
There may be different situations, when you may want to interact with HDAs in the HDK code. You may want to inspect the current Houdini session to see what HDAs are used and save a backup version for them, or you may want to programmatically add or remove some data to or from an HDA. You would use the three classes mentioned above, and the sections below describe them in more detail.
The OP_OTLManager class manages the operator type libraries. It also maintains the information about various preferences and settings associated with HDAs and OTLs, such as whether to use the OPlibrary files or not. However, its main task is to load and maintain a list of OP_OTLLibrary objects, which can be accessed with the OP_OTLManager::getLibrary() method.
There is only one instance of OP_OTLManager class and it is owned by the OP_Director global object. You can use OPgetDirector() global function to access that object (for more info on OP_Director please refer to About OP_Director) and then you can use OP_Director::getOTLManager() to obtain the OTL manager instance.
The following code lists the files of the loaded operator type libraries:
If you need to load a new OTL into Houdini, you should use OP_OTLManager::addLibrary() method, which does extra processing such as checking whether the library has been already loaded, warning about non-commercial assets it contains, and adding it to the meta source (see Metasources). After adding the library, Houdini will be able to use the operator definitions it provides.
Sometimes, the OTL file may have changed on the disk, when, for example, a new file has been downloaded or an old backup version has been copied from another directory. The Houdini will pick up this new copy of the file only the next time it starts up. But, there is a way to explicitly update the new definitions using OP_OTLManager::refreshLibrary() method. It will make sure Houdini uses the freshest copy of the OTL.
And finally, if Houdini no longer needs to use a certain library during the current session, you can unload it using OP_OTLManager::removeLibrary(). This call does not delete the file, but rather removes the library from the manager's list.
The OP_OTLLibrary class represents the operator type library stored as an OTL file on disk. It contains the name of that file, the information about how the library was specified to be loaded (ie, meta source, see Metasources), and a list of OP_OTLDefinition objects.
The OTL file name can be obtained with OP_OTLLibrary::getSource() and its meta source with OP_OTLLibrary::getMetaSource(). There are several utility methods to access information about the definition, but the basic one is OP_OTLLibrary::getDefinition().
Because it is cumbersome to obtain OP_OTLDefinition first before accessing name, there are a few utility methods for most common class members, which skip that step. For example, a single OP_OTLLibrary::getDefinitionName() method could be used in the above example instead of first invoking OP_OLTLibrary::getDefinition() and then following it with OP_OTLDefinition::getName().
The OP_OTLLibrary class is fairly simple, but it inherits some extra functionality from its FS_IndexFile base class. All of the inherited methods deal with data stored in an index file (see HDK_HDAIntro_Management_IndexFile below). One thing worth mentioning here, though, is that not all of the OTL file is loaded into the memory, because that would use up prohibitively large amount of RAM. Instead, Houdini reads in only the basic information about the asset definitions (e.g., name, label, and icon) and the rest of the data stays on the disc. Though, in addition to the name, label, and icon, the base class, FS_IndexFile, also reads in and stores the information about offsets and sizes of each index file section defining each asset.
Just like with OP_OTLLibrary, when Houdini loads an OTL, it reads in only minimal information about the operators it contains. This is evident by the fairly simple set of the member data in OP_OTLDefinition that describes an operator. It has the name of the operator, its label, icon, and the number of inputs. These are the main ones, and there is a handful of other supplementary information, such as an author and a modification date.
There is however, one other piece of data that is very important. It is the path, OP_OTLDefinition::getPath(), which provides the information about the location of a file that contains all description of all the remaining details for this operator. This file will contain information about the parameters, handles, scripts, etc. Usually, the file path looks like "oplib:/Object/myoperator?Object/myoperator" which means "go to the file of the operator library that defines the operator 'Object/myoperator' and look for the section whose name is 'Object/myoperator'". That's because Houdini interprets 'oplib:' as the library file that contains a particular operator, and the question mark delineates the index file name from the section name (see HDK_HDAIntro_Management_IndexFile for details), and, of course, because the library file is an index file.
So, the file that describes an operator resides in one section of an operator library index file. Moreover, the file that describes an operator (ie, an HDA) is itself an index file, so we are dealing with an index file (ie, operator definition file) contained within a section of another index file (ie, the OTL file). Each section of the (inner) index file that describes an operator contains a specific piece of information about that operator. For example, an HDA definition index file has a section called "DialogScript" that defines the parameters of the operator. The fact that the operator definition file is itself an index file is the reason why HDAs can be so flexible: each HDA can decide which sections it should add to its definition file. If an HDA function is defined by a network, such an HDA will have a "Contents" section, but if an HDA function is defined by a Python script it will have a "PythonModule" section, and if an HDA is a GLSL operator, it will have both "GlslVertex" and "GlslFragment" sections.
As a side note, the file path to the HDA section that defines, for example, help will look like "oplib:/Object/myoperator?Object/myoperator?Help", which means go the OTL index file that defines an operator 'Object/myoperator' and find a section named 'Object/myoperator', which is an index file, so go and find in it a section named 'Help'.
Coming back to the members of OP_OTLDefinition class, in addition to OP_OTLDefinition::getPath() described above, there are also two methods related to an index file, OP_OTLDefinition::getIndexPath() and OP_OTLDefinition::getIndexFile(). These method correspond to the script operators (such as SHOPsurface) that are defined in external index files rather than in OTLS and they are separate and unrelated to the OP_OTLDefinition::getPath().
As an example of using the OP_OTLDefinition class, here is a fragment of code that prints the help text for all the operators it contains:
The above code accesses the HDA's FS_IndexFile by directly obtaining the index file from the HDA's section using library's method FS_IndexFile::getIndexFileFromSection(). However, when working in Houdini HDK, it is often possible to use a simpler way of accessing operator's index file, using OP_Operator::getIndexFile(). This method is recommended for obtaining an index file from an operator, because it does some extra processing, such as making sure the correct OTL file is used and that all necessary sections have been properly initialized.
The FS_IndexFile pointer returned by OP_Operator::getOTLIndexFile() should not be deleted because it is owned by the OP_Operator class. This class deletes the pointer when you call OP_Operator::clearOTLIndexFile() or in the destructor. Invoking OP_Operator::clearOTLIndexFile() forces the operator to clear the cached information and reload it from the OTL. This is done, for example, when OTL has changed and there is a new HDA definition for this operator. Usually, though, there is not need for you to invoke it explicitly.
In general, an OP_OTLDefinition defines and corresponds to an operator known to Houdini HDK as OP_Operator. In the previous section, Houdini Digital Asset (HDA), the code examples demonstrated how to obtain FS_IndexFile from OP_Operator. But, there is more explicit and more direct relation between OP_Operator and OP_OTLDefinition, namely OP_Operator has own OP_OTLDefinition instance as a member data, which you can obtain with OP_Operator::getOTLDefinition(). This is a different instance of OP_OTLDefinition from that owned by OP_OTLLibrary, but they both describe essentially the same entity. Except that OP_OTLDefinition owned by OP_Operator represents an operator registered with Houdini, whereas OP_OTLDefinition owned by OP_OTLLibrary represents the operator definition available in a library. When an operator is defined by an HDA, the OP_OTLLibrary's definition describes exactly the same thing as OP_Operator's definition. Or, if there are several libraries defining the same HDA, it is the designated (i.e., current) OP_OTLDefinition that corresponds to the OP_Operator. In addition, if an OP_Operator is defined by an HDA stored in an OTL library, you can obtain the OP_OTLLibrary using OP_Operator::getOTLLibrary().
To rehash the relationship described so far, OP_OTLManager has a list of OP_OTLLibraries, and OP_OTLLibrary, in turn, has a list of OP_OTLDefinitions. When an OP_OTLDefinition is current, it corresponds directly to OP_Operator. The OP_Operator is owned by an OP_OperatorTable, which you can obtain from the global list of tables using OP_Network::getOperatorTable() given the table type. Each table corresponds to the network context, so there is a table for object, sops, pops, etc. Please, refer to OP_Node.h file for a complete list of the table type names that you can use. The OP_Operator, in turn defines all nodes of its type that exist in the current Houdini session, which you can obtain with OP_Operator::getNumActiveNodes() and OP_Operator::getActiveNode(). The following code makes use of these methods to calculate the total number of nodes created from the definitions contained in a given OTL library:
To go the other direction and to obtain the operator from a node, you can use OP_Parameters::getOperator() method, remembering that OP_Node derives from OP_Parameters. Then, you can use OP_Operator::getLibrary() and OP_Operator::getOTLDefinition() to obtain the information about the OTL and the HDA. The code below demonstrates how to use these methods. It checks which library defines the given node:
A node in Houdini can be synchronized or unsynchronized (locked or unlocked). A node is synchronized (locked) when it should exactly match the HDA definition, and it is unsynchronized (unlocked) when it can deviate from the current HDA definition. The unsynchronized nodes are used for modifying the HDA definition, by editing the network of the unsynchronized node first and then saving it as the contents section of the HDA. You can check the status of a node with OP_Node::getMatchesOTLDefinition(), which returns true if the node is synchronized and false when it is not. It is worth mentioning, that it is meaningful to speak of node being synchronized or unsynchronized only for the operators that have contents section. That is, for the operators that return true in OP_Operator::hasContentsSection() to indicate that there is an HDA section that describes a node network defining what the HDA does (ie, its procedural operation). Otherwise, OP_Node::getMatchesOTLDefinition() is always false.
Note, however that the node network stored in the contents section is not equivalent to the OP_Operator::getDefiningNetwork(). This method refers to situations when an operator is defined by a non-HDA network in the current Houdini session, for example, that method would return a specific VOP network that defines some VOP operator.
In general terms, if OP_Operator::getOTLLibrary() returns non-NULL, that operator is defined in an OTL library. In addition, if OP_Operator::hasContentsSection() returns true, that operator is defined by a network that can be edited and saved. On the other hand, if OP_Operator::getScriptIsPython() is true, then the HDA is a custom Python operator, etc. GLSL shaders are flagged as VEX shaders, with OP_Operator::getScriptIsVex().
When Houdini starts up, it needs to find out which OTL files to read in. The original mechanism for this was to supply a set of OPlibrary configuration files that would specify the list of OTLs to load. On startup, Houdini would look for OPlibrary files in standard directories (specified by an environment variable HOUDINI_OPLIBRARIES_PATH which is by default equal to HOUDINI_PATH), and then load the OTLs listed in these OPlibrary files.
A newer, and now default, method is to use an environment variable to search for the OTLs themselves. The variable name is HOUDINI_OTLSCAN_PATH and by default is equal to HOUDINI_PATH with "/otls" appended to each directory in that path. In Operator Type Manager dialog, you can configure which of these two methods Houdini should use.
However, even if Houdini is to scan for OTL files, the OPlibrary is still indirectly useful for manually loading HDAs into Houdini session, which is then saved in .hip files. In File > Install Digital Asset Library... dialog, the user specifies whether the OTL is to be installed to the current hip file. And if so, Houdini will write an .OPlibrary CPIO packet that lists the OTL file being installed. Then, when Houdini reopens the hip file in the future, it knows which additional OTL files to load, because, the hip file most likely contains instances of these manually loaded HDAs.
These sources that tell Houdini which OTL files to load are called metasources in the HDK. They usually represent a concrete OPlibrarary files, but there are a few special metasources, listed in OP_OTLLibrary.h. As mentioned, OTL_INTERNAL_META refers to the OPlibrary saved within a .hip file. Additionally, there is OTL_OTLSCAN_META which is an abstract metasource referring to the fact that the OTL library was loaded because it appeared in the specified scan path. Finally, OTL_FALLBACK is an abstract metasource for HDAs whose OTL files could not be located, for which only the dialog script is available because it was saved inside a .hip file.
All that has relevance to the OP_OTLLibrary::getMetaSource() method, which returns the OPlibrary, or an abstract metasrource, that describes how the library got loaded into Houdini. To get a descriptive name of a metasource for a particular library, you can use OP_OTLManager::getMetaSourceName() methods. Removing a metasrouce with OP_OTLManager::removeMetaSource() will uninstall the libraries it was responsible for.
Similarly to the .OPlibraries CPIO packet, there is also an .defotl CPIO packet that save an OTL file known as OTL_INTERNAL. If you create a new HDA ans specify "Embedded" as the library file, it will be placed in this internal library. This is convenient for distributing HDAs with a .hip file, but note that only OTLs that have instances in the current session will be saved to the OTL_INTERNAL library.
Because there may be more that one library that contains and defines a particular HDA, there is a notion of a preferred HDA. This is the HDA that is currently used for all the synchronized nodes in the session. For a given operator, you can invoke OP_OTLManager::getPreferredLibrary() to find out which library is currently governing the behavior of the nodes. This method is similar to OP_OTLManager::getLibraryPreference(), which returns the path of the current library for a particular operator. You can also programmatically set which library should be the current one, with OP_OTLManager::setLibraryPreference(). And to see which other library defines this operator, you can invoke OP_OTLManager::getNextLibrary().
You can also find out the library that has the newest modification time. You can do that with OP_OTLManager::getLatestLibrary(), that takes an OP_OTLLibrary pointer that gets returned if it turns out to be the latest library among a few other contenders that have the exactly same modification time.
The following example lists all the libraries that define an operator:
It may be sometimes useful to monitor the OTL manager for changes, such as when a new library has been added or an old library is about to be removed. The OP_OTLManagerSink class allows to receive the notification about these events. If, for example, you wanted to know when a new HDA definition is added to any of the OTLs, you would implement own class deriving from OP_OTLManagerSink, then you would register it with the manager calling the base class OP_OTLManagerSink::addManagerSink(), and finally you would overload OP_OTLManagerSink::definitionsAdded() virtual method, which would be invoked by the OP_OTLManager whenever it added a new HDA to any of the OTLs. The arguments of that method would specify which library and which definition within that library has been added.
An index file, FS_IndexFile, is a file that contains a header followed by sections. The header contains information about how many sections there are in the file, and about their sizes and offsets within the index file.
In Houdini, you can specify a section as part of the file path. Usually, the directories are separated with a slash, '/', but to separate an index file from its section you need to use a question mark, '?'. So, "my_directory/my_index_file.idx?my_data", refers to the "my_data" section within "my_directory/my_index_file.idx" file. You can use that convention throughout Houdini, and it will correctly access data in the specified section. There are some utility functions in UT_OpUtils to combine and split index file paths and also to manipulate the operator type names.
The following code prints out the contents of "my_data" section:
As the above example illustrates, most of the time, you can also use an index file section wherever a method expects a file.
The OP_OTLLibrary class actually derives from the FS_IndexFile. Each section of that file contains the data for a single HDA. There is also a special section called "INDEX_SECTION" that contains some basic info about the HDAs in that library. This avoids the need to search and parse the HDA section to find out this information.
As illustrated in the above diagram, an HDA is saved in the section named "table_name/operator_name". If the operator specifies a namespace or a version tag, for example "ArtistX::operator_name::2.0", this information is also included in the section name, for example @ "ArtistX::table_name/operator_name::2.0". The section name can be reliably obtained either with OP_Operator::getDefinitionSectionName() or with OP_HDADefinition::getDefinitionSectionName(). Since both OTL and HDA are saved as index files, we are dealing with nesting, and to get to an HDA section, we need to specify two sections, one for the OTL and the other for the HDA, for example, "directory/file.otl?table_name/operator_name?HelpUrl".
You can easily obtain the index file associated with OTL or an HDA by specifying the file path:
However, if you hold a pointer to the OP_Operator it is preferable to use OP_Operator::getOTLIndexFile(), because this method does some extra processing and error checking. The following example lists the section names in an HDA:
Adding arbitrary data to a digital asset can be achieved by adding a new section to the HDA definition. Similarly, removing a section will remove the data it contains. However, after the changes are made to the HDA file, they have to be committed to disk using OP_OTLManager::addToLibrary() which replaces the old HDA definition with the new one and notifies the interested parties of the change, or with OP_OTLManager::writeOutDefinitionContents() which is intended for maintenance of HDAs that are not the current definition of the operator.
There are a few methods in OP_OTLManager that allow manipulation of the OTL files. For example, OP_OTLManager::addToLibrary() adds a new HDA definition to the library. But it also can be used to update the definition by specifying a new definition that will replace the old one. This method is very versatile because it performs many tasks. It updates the current definition, it saves out the data on disk, and notifies other object of the change event. As illustrated in HDK_HDAIntro_EditingHDA, this method is very versatile and can be used in variaty of cases for various purposes of modifying OTLs and HDAs.
A corresponding method, OP_OTLManager::removeFromLibrary() removes an HDA definition from the library. And another, OP_OTLManager::mergeLibraries(), combines two libraries into one, and deletes the secondary library instance.
This section contains sample code that demonstrates how to do some simple OTL and HDA management.
This code lists the content HDAs of an OTL.
This example scans all the operators in Houdini, checks if that operator comes from an HDA inside an OTL and if so, saves it in the embedded library within the hip file, but only if there is any node instantiated for this operator (ie, the hip file needs that operator).
This example converts the binary ExtraFileOptions section (extracted using "hotl -x") to a readable json equivalent and vice versa.
1.8.6 
