Hi guys,

Building octrees takes a long time for heavy or complex geometry. I don't know much about octrees, are they in world/object space or camera based? If they are in some kind of world/object space it would be really cool to have a kind of delayed load octree which could be baked out before rendering allowing mantra to skip the octree creation step and start rendering more quickly.

I THINK that's the idea with vrmeshes in VRay. They can't be displaced or altered after they're made but renders start immediately.

Would love to hear other peoples thoughts and please correct me if I'm wrong in understanding this.

Yeah I'd love for this to be faster. I think it got a little bit threaded in H12 but it still sucks up a lot of time before you see an image…

Yeah would be greate to have the ability to store the occtree with a delayed load. +1!

Maybe submit an rfe?

Definitely +1 for this.

Before the wave of +1's…..

I guess some questions and caveats. I suspect the Octree's positional data is currently stored in the camera's space and not world. So that would have to be adjusted.

Also if you are saving out the octree its possible that you'll have limitations like not being able to change certain attributes like Subdivision surfaces.

Maybe instead of saving out Mantra's internal octree structure, support was added for a random access geometry format. Like extend the TBF/point cloud files to support full geometry sets. That way when tracing against geo archives you don't need to load the full mesh in, only the relevant section. (This is how a vrmesh (vray proxy) works).

http://www.spot3d.com/vray/help/150SP1/vrayproxy_params.htm#vrmesh [spot3d.com]

Hi everyone!

I've just committed an attempt at parallelizing the construction of k-D trees (what's currently used in place of octrees), so I'd be really interested to see how it works for ya. Tomorrow's daily build should have the change, so long as that build works.

Caveats: For some renders, it temporarily uses a bit more memory. Also, on renders where the total amount of memory used approaches or exceeds the machine's physical memory, the performance may be as poor as before. That one's kind of expected, since there's not much that can be done with just a few weeks work to reduce paging out to disk.

Cool! Let's hope it's not Donut Day

pbowmar
Cool! Let's hope it's not Donut Day

I'll put Tim Horton's on speed dial. :wink:

Wow cant wait to try it out!

Does this mean full threading of octree genetration? Or more threading? Hoping for the first!

ndickson
Hi everyone!

I've just committed an attempt at parallelizing the construction of k-D trees (what's currently used in place of octrees), so I'd be really interested to see how it works for ya. Tomorrow's daily build should have the change, so long as that build works.

Caveats: For some renders, it temporarily uses a bit more memory. Also, on renders where the total amount of memory used approaches or exceeds the machine's physical memory, the performance may be as poor as before. That one's kind of expected, since there's not much that can be done with just a few weeks work to reduce paging out to disk.

Awesome can't wait to try it out!

ndickson
That one's kind of expected, since there's not much that can be done with just a few weeks work to reduce paging out to disk.

KD-Tree stuff aside, these efforts are equally welcomed.

Korhon
Does this mean full threading of octree genetration? Or more threading? Hoping for the first!

It's just the building of the geometry k-D tree structure used by mantra (presumably, that's most of what it's doing when it says “Building octree…”). This building was all serial before. Specifically, it:

1. Builds a temporary k-D tree structure (that can be built in parallel), recursing in parallel
   
2. Estimates the amount of memory used, recursing in serial, (fairly fast, so parallelization might not make a difference)
   
3. Builds the usual k-D tree structure from the temporary structure, recursing in parallel
   
4. Deletes the tempoary structure, recursing in parallel
   
5. Reorders the bounding boxes pointed to by the k-D tree, in serial (because parallelization would get messy and might not help)
   
   Of course, all bets are off if memory pages to disk, but when using 12 cores, building the temporary k-D structure seems to take 65-75% of the time and reordering the boxes takes maybe 20% of the time.

This is great, thank you very much.

Can we expect this work to accelerate building pointcloud KD-Trees and those Houdini operations that use KD-Trees too? (eg VolumeFromAttrib?)

jason_iversen
Can we expect this work to accelerate building pointcloud KD-Trees and those Houdini operations that use KD-Trees too? (eg VolumeFromAttrib?)

Once I've cleaned up a few things, I may be working on that next, (unless I'm confusing it with something else). It sounds like it should be easier than parallelizing the k-D tree building in mantra, but I haven't looked at it myself.

Wolfwood

ndickson
That one's kind of expected, since there's not much that can be done with just a few weeks work to reduce paging out to disk.

KD-Tree stuff aside, these efforts are equally welcomed.

I hear ya. Paging to disk can bring a program to its knees. Reducing it usually involves a massive redesign/reorganization of how memory is used, as well as assumptions on how the program is commonly used, and sometimes even a complete change of how the program behaves.