I am testing Solaris on a very heavy scene. I am thinking of is there a way that I can display the geo in the viewpoint using a low res proxy for the purpose of set dressing. Then switching to a high res, when render in Karma ?

Scott talks about it in the H18 launch presentation, the workflow is shown on-screen:
https://youtu.be/Aql6SIOIyXs?t=730 [youtu.be]

I've also attached an example file.

You might also be interested in seeing if the Create LOD LOP can help you manage complexity.

USD accomplishes this with Purposes - Render, Proxy, Guide, and default. Set the dense meshes to Render and your proxy representation to Proxy using the Configure Primitive LOP. GL renderers draw Proxy, Guide, and default, while final renderers draw Render and default.

This is great ! Works like charm. Thanks.

Hey all, bringing this topic back up…

Just to confirm, this creates essentially two primitives in the scenegraph… one for the proxy and one for the render. How are people managing “redundant” primitives in the Scenegraph?

And secondly, if the renderable purpose is hidden from GL, is it still loaded into memory, or is only the proxy loaded into memory (until render time)?

Tim Crowson
Hey all, bringing this topic back up...

Just to confirm, this creates essentially two primitives in the scenegraph... one for the proxy and one for the render. How are people managing "redundant" primitives in the Scenegraph?

And secondly, if the renderable purpose is hidden from GL, is it still loaded into memory, or is only the proxy loaded into memory (until render time)?

interesting questions!

Actually its quite simple:

there a a few attributes that you can create at SOP level that will help you to keep your Solaris clean. In this case you can create s@usdpurpose to set your objects up.

You can create a usdpurpose primitive string SOP attribute to control the purpose of the corresponding USD prim. The value should be either default, render, proxy, or guide.

https://www.sidefx.com/docs/houdini/solaris/sop_import.html [www.sidefx.com]

also very helpful:

https://www.youtube.com/watch?v=Mu8bT3zaUsA [www.youtube.com]

This topic covers the render and proxy settings for geo which can be hold in Houdini'S memory. What about if the proxy doesn't have a render equivalent in the scene, but a file path to a .bgeo or .usd file on disk. How is this handled? Especially if we are talking about hundreds of different high res meshes.

Proxy/Render purpose are USD concepts and so mostly unrelated to what Houdini might have in memory. If you save meshes as binary USD and reference those into lops, the heavy data (typically points, primvars etc) won't be read from the file until the renderer needs it or you display the values in the UI. Put another way, reading the file to construct the scene graph hierarchy (irrespective of purpose) doesn't require reading any of the attribute data from the prims. For bgeo I'm less sure but I suspect it's less optimized than binary USD.

You could test that yourself if you wanted. Try loading a binary USD file with 100 meshes with a million faces each, versus a file with 100 meshes with only 1 face each. You should find that both load into lops and shows you the scene graph tree pretty much instantly because 100 prims is a very simple scene. Hitting render in Karma (or any other renderer) however will be a lot slower for the 100 meshes with a million faces because at that point there will be a lot more data for the renderer to deal with.

Loading the scene graph independently from any prim data is one of the things that makes USD super scaleable. USD itself doesn't really care about any of the attribute data, it's pulled out of the file attribute by attribute as consumers (typically renderers) need it.

Thanks so much for your input, which is really helpful to get the general idea.

But from a practical standpoint, how would I approach this?

I'd import the low res versions of the files (really low res on memory) into lops, reduce them in a sop modify node with a custom node depending on the camera view (delete prims which are out of view), pack them to get a point attribute with the USD reference file name, and then? At this point I'd like to establish that Houdini uses the file path to load the geo for rendering purposes while displaying the proxy version packed one).

How do I use the file path info to make it a reference. I'd like to choose a wrangle and convert it to a reference. Is this the way to go? The amount of different meshes requires that I use the file path attribute. I can't set up anything by hand per object.


Cheers
Tom

Hey Tom, here's an example of what I think you're perhaps shooting for. If not, it might serve as a starting point.

The hip file has a section that generates two usd files and a bgeo - file1.usd, file2.usd, file3.bgeo. Just poke the render button on the /stage/generate_files ropnet to create all three. Each one defines two sets of geometry - proxy and render. By default the proxy geo will only be visible in the gl/vulkan viewport and the render geo will only be visible in karma viewport (that can be changed using the 'sunglasses' icon on the viewer righthand sidebar).

All three files are 'loaded' simultaneously using a reference lop. Like I said in my earlier post though, the data in the usd files isn't really loaded, only the prim hierarchy. I'm less sure about the bgeo file but I suspect the entire contents are loaded into memory at the time the file is referenced. Anyway I included a bgeo file mostly to show that the reference lop can load bgeo just fine (which isn't always obvious), and that the usd purpose attribute can be authored into bgeo files as well. If you really care about performance though I'd definitely recommend running some tests with your data to see how the two formats compare.

Following the reference there's two sopmodify ops, one that modifies the proxy geometry and another that modifies the render geometry. Obviously when the render geometry is loaded into the sopmodify the points and topology for the sphere (file1.usd) and cube (file2.usd) must be loaded into houdini. But if the sopmodify were disabled the data would not be read into houdini, only karma at render time.

Thanks so much for your example!

My workflow is slightly different and my problem unfortunately remains. : )

In the folder with my high res geo are about 20.000files, all .usd with a name attribute.

In SOPs I do have several low res primitives which then get transfered to LOPs.

These low res prims do carry a name and a path attribute. Name is identical to the high res version, path attribute is the path to the high res version.

In order to to see the low res prims there are two prims for each of them. One with the purpose "proxy" and "_proxy" attached to its name and one with "render" as the purpose.

If I use Houdini's sublayer LOP node and manually choose one of the high res files which is present on the stage as a low res file, then it replaces the low res "render" prim.

Everything as I'd like it to be. It displays the low res prim and renders the high res prim. BINGO.

Since I can't do this manually for all the prims I want to sublayer these files in the python lop.


https://docs.omniverse.nvidia.com/dev-guide/latest/programmer_ref/usd/layers/add-sublayer.html [docs.omniverse.nvidia.com]


That's the only help I can find on this subject. Unfortunately the code doesn't work. It throws an error "A layer already exists with identifier". Guess that's the fault of the CreateNew()

It's really tricky to find ANY documentation on python and sublayers except this one. And the usual search commands will almost always point you to this page. : /


https://docs.omniverse.nvidia.com/dev-guide/latest/programmer_ref/usd/references-payloads/add-reference.html [docs.omniverse.nvidia.com]


I wished I could see the python code within the sublayer LOP (knowing that it's most likely not python based).

I managed to get hold of all the low res prims I need to exchange and also the path attribute in python. I just struggle with the python code to override the layer with the sublayer or add the sublayer with the higher opinion (whatever the right terminology might be).

Anybody who knows the right syntax?


P.S.: I also exchanged stage with stage = node.editableStage() in the following code, still throws the error.



from pxr import Sdf


def add_sub_layer(sub_layer_path: str, root_layer) -> Sdf.Layer:
sub_layer: Sdf.Layer = Sdf.Layer.CreateNew(sub_layer_path)
# You can use standard python list.insert to add the subLayer to any position in the list
root_layer.subLayerPaths.append(sub_layer.identifier)
return sub_layer

#############
# Full Usage
#############

from pxr import Usd

# Get the root layer
stage: Usd.Stage = Usd.Stage.CreateInMemory()
root_layer: Sdf.Layer = stage.GetRootLayer()

# Add the sub layer to the root layer
sub_layer = add_sub_layer(r"C:/path/to/sublayer.usd", root_layer)

usda = stage.GetRootLayer().ExportToString()
print(usda)

# Check to see if the sublayer is loaded
loaded_layers = root_layer.GetLoadedLayers()
assert sub_layer in loaded_layers

To add a sublayer from python you'll likely have to do so via hou.LopNode.addSublayer(), rather than the regular USD API (which the Nvidia example is using). Another option is to use a foreach loop in "For Each String in Parameter" mode.

Here's and updated the example file from earlier with both of those approaches.

Thanks so much! Again! : )

The only thing which stays unresolved yet is that my sublayer mesh doesn't replace the imported sop mesh. I thought by adding a sublayer Houdini/USD would check for another layer with the same name and replace it.

My stage looks a bit like this:

/
____/parking_lot (xform)
________/cars (xform)
____________/car_a__proxy (mesh) -> low res mesh, purpose "proxy"
____________/car_a (mesh) -> low res mesh, purpose "render"
____________/car_b__proxy (mesh) -> low res mesh, purpose "proxy"
____________/car_b (mesh) -> low res mesh, purpose "render"
____________/car_c__proxy (mesh) -> low res mesh, purpose "proxy"
____________/...

This got created by using a sopimport LOP node with import path prefix set to "/parking lot/cars". The meshes do carry the name attributes "car_a" and so on.

Now I use the sublayer LOP and import the high res mesh with the name attribute "car_a". Or the python code you posted. Same result.


I thought that it would replace my low poly mesh since it got the same name attribute. But it doesn't. It just adds it to the stage.

/
____/Geometry
________/car_a
____/parking_lot (xform)
________/cars (xform)
____________/car_a__proxy (mesh) -> low res mesh, purpose "proxy"
____________/car_a (mesh) -> low res mesh, purpose "render"
____________/car_b__proxy (mesh) -> low res mesh, purpose "proxy"
____________/car_b (mesh) -> low res mesh, purpose "render"
____________/car_c__proxy (mesh) -> low res mesh, purpose "proxy"
____________/...


BUT if I save the high res mesh with the name "/parking_lot/cars/car_a" it replaces the low res car like desired.

The problem is that I might want to place "car_a" in another scene not in a parking lot between other cars, but in a landscape.

/
____/landscape
________/car_a

So saving my high res asset with an absolute path is obviously not the best choice, since the car doesn't really now yet where it's going to be placed.

What I expect the tree to look like is

/
____/parking_lot (xform)
________/cars (xform)
____________/car_a__proxy (mesh) -> low res mesh, purpose "proxy"
____________/car_a (mesh) -> high res mesh, purpose "render"
____________/car_b__proxy (mesh) -> low res mesh, purpose "proxy"
____________/car_b (mesh) -> high res mesh, purpose "render"
____________/car_c__proxy (mesh) -> low res mesh, purpose "proxy"
____________/...

An alternative would be to already pass on an ".usd" path to the low res prims as a reference, but this seems to be not possible as far as I can tell.

Sorry for the underscores, but the spaces were removed.

P.S.: Can my problem be that I'm trying to sublayer a mesh which is probably not a layer in terms of the USD definitions? I also recreated the stage with an inline sop loading the .usd high res files. Just without the proxy meshes.

/
____/parking_lot (xform)
________/cars (xform)
____________/car_a (mesh) -> high res mesh, purpose "render"
____________/car_b (mesh) -> high res mesh, purpose "render"
____________/...

Using a sublayer node with the stage tree as seen in my former post in the 1st input and this tree in the second input also does nothing, EXCEPT from coloring the "cars" xform" green.

I though it would override the car_a and additional cars as well. : (

USD sublayering just combines whatever scene graph is in each layer. It doesn't do any kind of object or name based replacement. Also, the combining happens at the attribute level not prim level. So it's not possible to replace something entirely just by using a stronger sublayer. If you need to completely erase the contents of a prim (points, topology etc), the robust way to do that is to place the attributes in a variant which can be switched off thus creating an 'empty' prim. Then, a stronger layer that wants to replace a prim entirely would toggle the variant, guaranteeing that it's starting with an empty state.

Anyway, for your current structure take a look at the attached hip file. The trick is to change the /Geometry scope to a class specifier (rather than def) so that it doesn't render, and then have /parking_lot/cars inherit from /Geometry, which will place all the hero cars below /parking_lot/cars. Another detail is that the sopimport with the proxy and semi-hero cars must be brought into lops with the 'Load as Reference' checkbox enabled. Without that the inherit that comes later won't work due to composition order (inherit is stronger than reference, which is what we want in this case).

Note that because there's no name matching happening, the render geometry coming from the sopimport isn't needed either. It would be simpler to remove those and just have the sopimport pull in the proxy geometry.

Here's another version that's likely more useful in practice because it places the high res geometry on every occurrence of a low res instance. It requires a slight tweak to the sop network though to retain the instancing (via packedprims)

I highly appreciate your help!

Will still have to check the files though.

I made some progress in the meantime and I guess it's been your advice to "import as references" which made the difference. Suddenly it works. The python code with "AddReference()" does the job, "overwriting" the current low res mesh with the link to the high res one. Will post my code/scene later on so that hopefully somebody can profit from it too or tell me that this is the utterly wrong way to do it! : )

Hey Tom, there's many ways to approach these things for many different reasons, so it's hard to say if there is a more optimal approach. What I'm not quire sure of is what the advantages are of having proxy geo in SOPs. I've worked on a few parking lots and typically we have a bunch of vehicle assets (that may or may not have proxy geometry), and we just instance them around, either in a pointinstancer or standalone models. If there are huge numbers of instances we typically rely on texture cards to keep the viewport quick (our pipeline renders out front/side/top images of every asset and so usd texture cards work-out-the box, which admittedly takes a bit of setup and can be a pain to get working while knee deep in project work).