Excited about H13, i was wondering if there was some documentation on the new Packed primitives concept/workflow.
I can't find any in the help files yet.

RudiNieuwenhuis
Excited about H13, i was wondering if there was some documentation on the new Packed primitives concept/workflow.
I can't find any in the help files yet.

http://www.sidefx.com/docs/houdini13.0/news/13/ [sidefx.com]

and in particular

http://www.sidefx.com/docs/houdini13.0/news/13/geometry [sidefx.com]

Though, I'm not sure how much of the documentation touches on new workflows.

Packed primitives are a generalization on the Alembic primitives.

You can pack any Houdini geometry in to a packed primitive: volumes, NURBs, vdb's, ploys, polysoups, etc. Including Alembic Primitives and other packed primitives. This means you have nested instancing.

So Alembic primitives and packed primitives are deferred rendered in to the viewport and Mantra and can be treated as instanced geometry.

The Copy SOP can now do real instancing when you pack primitives feeding in to the left. The COPY SOP even has a new stamp option to pack the incoming primitives in-line. Along with a ton of override parms now.

Packed primitives don't support animation baking, but we use Alembic primitives for that.

You can use the Pack SOP to pack any incoming geometry.

One key SOP in the workflow is the Assemble SOP now. It can create packed primitives for you. This is the one I turn to 99% when turning assemblies in to a packed primitive hierarchy.

The Packed primitive can be saved to memory or to disk. When in memory, the viewport and Mantra (ipr) will access the packed geometry from there. You can also save the packed primitives along with any other geometry in to a .bgeo file.

There is the Packed Edit SOP that works in conjunction with the new Data Tree's two options: geometry and light modes. When you move the display flag to each Packed Edit SOP, you can inspect it's settings in the Data Tree panes. You can have a few Packed Edit SOPs. Moving the display flag around will allow you to inspect what each one of those has done: display settings, material assignments, light assignments.

That's pretty much it. Pretty simple. Designed for Look Development and lighting, same problem Alembic is solving.

Oh and Bullet now supports packed geometry making the Assemble SOP even more important. If you bring in fractured geometry from Houdini or another app, just put the Assemble SOP in there. The new RBD Fracture shelf tool does that for you anyway.

Hope that helps.

They're pretty easy to wrap your head around if you get Alembic primitives in Houdini.

Thanks for the information guys.
I will try soon to get this to work.

I put together a description of some possible examples, but it was too colloquial, incomplete, and way too last-minute to get into the docs in any acceptable form. In case you're interested, here's what I wrote:

A packed primitive provides a suite of mechanisms for encapsulating an entire detail that may or may not even exist in memory until the primitive is unpacked. To avoid having to load and display everything while using it, it provides options for displaying as just a bounding box, or just the points of the geometry. It also enables very significant memory sharing, to the point of providing SOP-level instancing and much more, while being much simpler to set up than object-level instancing. This is recommended as the way to do instancing in the future.

For example, suppose that you have 5 intricate oak tree models, stored as polygon soups in files, with which to make a forest of 10,000 trees. Simply copying the oak tree model 10,000 times would take 10,000 times as much memory, so unless you make lightweight sprites to stand-in for most of the trees, copying the model is out of the question. Instancing the models onto points allows you to place them in the scene without loading the 5 trees, and when viewing, the tree models only need to be loaded once. However, it can be awkward visualizing the placement and transformations of the trees as just one point for each. Packed primitives would enable you to, in SOPs, have 10,000 primitives, each referring to one of the 5 files, and adjust the placement and transformation of each primitive, selectively viewing just one or a few of the trees as full models, or as point clouds, still only loading up to the 5 files, as needed. This can be easily set up using the pack options on the File SOP and then copy stamping.

Perhaps more interestingly, you could procedurally generate 100 models of branches and trunks for the trees, each using 3 polygon soups, and use the “Pack Geometry Before Copying” option on the Copy SOP to pack up each of these details before copy stamping to procedurally generate 50 tree models, each with 40 branches and a trunk, using packed primitives referring to the 100 details containing the branches and trunks. When a packed primitive is copied, it just refers to the same geometry data as the original, so there will only be 100 branch and trunk models and a set of packed primitives referring to these models. These 50 tree models can then also be packed up, and copy stamped to make the forest, and the result will then be:

• 1 detail, for the forest, containing 10,000 packed primitives, only referring to
• 50 unique details, one for each unique tree model, each containing 41 packed primitives, only referring to
• 100 unique details, one for each unique branch or trunk model, each containing 3 polygon soups, and the corresponding points and vertices.

The forest detail on its own takes up a bit more than 2 MB, the 50 tree details take up a total of a bit less than 1 MB, and supposing that each branch or trunk model takes up about 70 KB, the 100 branch and trunk details take up a total of about 7 MB, for a grand total of 10 MB for the entire forest. If you were to unpack just the first level of packing, you would have a forest containing 410,000 packed primitives referring to the branch and trunk models, which would take up about 89 MB, for a total of 96 MB. If you were to unpack just the second level of packing, you would have 50 tree models of 123 polygon soups each, which would take up about 144 MB, for a total of 146 MB. Unpacking both levels would result in a forest detail containing 1,230,000 polygon soups, taking up a staggering 28.7 GB.

This is why it’s important that packed primitives aren’t accidentally unpacked without a user specifically indicating to do so. This means that packed primitives don’t support most operations that one might want to perform on the underlying geometry. It would be a rude surprise if one put down a Divide SOP and everything was unpacked in the process.

If you want to perform some operations on the packed geometry, you have to unpack the part of the geometry that’s of interest, perform the operations, and then optionally repack that geometry. For example, supposing that a branch on one of the trees in the forest is encroaching on a path, you can unpack that tree, replacing its packed primitive with 41 branch and trunk packed primitives, translate, rotate, and scale the offending branch packed primitive, then repack the 41 packed primitives for the tree. There will then be 51 unique details for trees.

Mantra treats packed primitives as instanced, though it doesn’t yet support nesting instances, so it will create 410,000 instances internally for this example, which can get memory intensive, but is certainly feasible. This means that for more significant nesting of packed primitives, there can be a tradeoff between the size of the underlying geometry and the number of instances. For example, packing a single polygon and creating 1,000,000 packed primitives referring to it will take up significantly more memory than a single detail with 1,000,000 polygons, but packing 1,000 polygons and referring to it 1,000 times is better than either of those. With some careful balancing of that tradeoff, it is possible to render over 1 trillion cubes with vertex normals using less than 9 GB of memory.

Supposing that a branch needs to be broken as a solid object using a finite element simulation, you can unpack just that tree, then unpack just that branch, and set up the simulation. For RBD simulations, geometry often doesn’t even need to be unpacked. It can use packed fragment primitives to refer to just groups within a detail. It also means that that using the Copy SOP with a group and packing doesn’t need to make a new detail with just the content of the group, unless the primitives are unpacked.

From the HDK, packed primitives are even more flexible, since you can provide your own packed implementation, which could procedurally generate geometry as you choose, using any data from your custom SOPs that created the packed primitives, as well as any data from the detail containing the primitives at render time. For example, you could create a packed primitive implementation that generates, at render time, snow packs and trees for mountain terrain in the detail.

That is amazing!
Thanks Neil for concise explanation.

You're welcome!

I've only got images for the 1 trillion cubes example, though I tried out the forest example with some stand-in L-systems and it seemed to work okay. It took a while to wrap my head around what things to pack and what not to pack when mixing the separate copy stampings, but (I think) I eventually figured it out. I found the key for me was just thinking about when I would end up having packed primitives that each only contain one packed primitive, and not packing in that case.

Warning: In the trillion cubes scene, if you change it to do 1 trillion boxes where each box is a packed primitive referring to the same single box, DO NOT try to render it, since Mantra will try to make 1 trillion instances. That's why I have it as a polysoup up to 1 million boxes; then Mantra only has 1 million instances.

1280x720: http://neildickson.com/images/1TrillionCubes.png [neildickson.com]
19200x10800: http://neildickson.com/images/HighResCubes.png [neildickson.com] (It's 200MB, but you can see individual cubes at the 1 billion level)

Thank you!!!

Thanks for the detailed info on the very same day of the release.

I am getting some amazing result on copy sop with packing before copying.


P

Thanks for the exlanation Neil, very helpful.

I'm glad I could help. If people are also wondering what's up with polysoup primitives, there are now some tips in the docs about when to use them instead of polygon primitives:

http://www.sidefx.com/docs/houdini13.0/model/primitives#polysoup [sidefx.com]

Thanks for the explanation, it's an eye opener.

Since mantra doesn't yet support nested instancing, what about RenderMan support for this? For RenderMan, would it be technically possible to generate a delayed read archive or instance procedural at each packed primitive level?

There was a paper by Pixar describing some of the tradeoffs between many instances vs. fewer larger instances in RenderMan, which sounds similar to what is happening with mantra.

I may be misunderstanding RenderMan's use of instances as I have only used them briefly on a show, and they were generated by in-house software.

So far it's working, even though it wasn't easy. I could use some help though on automatically filling up the packededit material operations.

I've found a way to automatically split incomming geometry in different packages by analizing the material that's assigned per primitive. I do this with a attributecreate sop where i create a name attribute for which i use this expression:
`substr(prims(“../”+opinput(“.”,0),$PR ,“shop_materialpath”),rindex(prims(“../”+opinput(“.”,0),$PR ,“shop_materialpath”),'/')+1,50)`

So now i can use a pack sop where i use the Name Attribute name to define what goes in which package.
You could do this ofcourse straight in the pack sop but it gives me double slashes in the pack paths (for example: opobj/InstancingGeo/pack1//shop/material1 ) and that freaks me out.

For the packed edit sop however, i still need to fill this myself. For each pack i need to assign the material manually. It would be great if there was a way to automatically fill this but i'm afraid i'm going to have to script this myself…
Or does anyone know how to do this easily ?

RudiNieuwenhuis
So far it's working, even though it wasn't easy. I could use some help though on automatically filling up the packededit material operations.

I've found a way to automatically split incomming geometry in different packages by analizing the material that's assigned per primitive. I do this with a attributecreate sop where i create a name attribute for which i use this expression:
`substr(prims(“../”+opinput(“.”,0),$PR ,“shop_materialpath”),rindex(prims(“../”+opinput(“.”,0),$PR ,“shop_materialpath”),'/')+1,50)`

So now i can use a pack sop where i use the Name Attribute name to define what goes in which package.
You could do this ofcourse straight in the pack sop but it gives me double slashes in the pack paths (for example: opobj/InstancingGeo/pack1//shop/material1 ) and that freaks me out.

For the packed edit sop however, i still need to fill this myself. For each pack i need to assign the material manually. It would be great if there was a way to automatically fill this but i'm afraid i'm going to have to script this myself…
Or does anyone know how to do this easily ?

The packed edit SOP is built to work in tandem with the new Data Tree pane. Try out the Object Appearance view in the Data Tree.

Oh, right now, the packed edit needs to be displayed to work in the data tree.

The Data Tree is not showing the materials i set in the packededit nodes. It does render ok, so everything is working, but the ui could use some love imho.

RudiNieuwenhuis
The Data Tree is not showing the materials i set in the packededit nodes. It does render ok, so everything is working, but the ui could use some love imho.

Can you submit a bug for this? There's a limit to what the object appearance view can do if the packed edit sop has been manually edited, but I'd like to cover as many cases as possible, even if only to read them properly.

halfdan
Can you submit a bug for this?

I already sent an email to Jenny about this just a couple of minutes ago.
If you need more info don't hesitate to contact me.

What is the preferred/optimal method for using animated geo to be offset in time? The copy stamping still takes forever when you start going up in numbers, even with the alembic primitive sop vs the timeshift sop. Is there any way to speed this up?

Thanks!
-Steve

I'm not sure if it is optimal but i think using an expression in the file sop to offset the framenumber will prevent unneeded loading of data.
something like: geo_`padzero(4,$FF-100)`.bgeo

Actually I was able to do this using the instance sop, and it is WAY faster than the copy stamping. In fact, I don't know if I'll ever be using the copy sop when needing to do this, because it is so much faster. :shock: