Greetings,

I'm pleased to release the first version of the Houdini/mantra “clusterThis” geometry/volume instancer. The DSO will create instances of primitives for each point of the incoming geometry and be “displaced” around the point based on a number of user parameters and methods.

The project is an open source project and will also be on the source forge odforce site for other developers to contribute if they would like. The current state of the code is just the basic structure to get the project started, very simple algorithms for now … and there's definitely bugs! Use at our own risk! The noise generator is right out of the HDK toolkit, nothing else used yet. Soon I will have the CVEX functionality then you'll be able to use standard VEX code for processing the instanced objects. Truthfully, I've been having so much fun with it I've been a bit tardy in getting the code released, so I know once a few more features are implemented there will be good fun for all.

Keep in mind this is a geometry instancer, it merely off loads the creation of geometry until render time, so keep an eye on memory usage. Future projects will include some shaders that will do similar kinds of things but of course limited to what can be done with just shaders … the two tools together will give an artist a lot of capability for managing complex detail.

Here's the web page where you can download the code, DSO/DLL (and yes, there's a Mac OSX version!), and see some example pics/movies as we test and develop the project:
http://www.digitalcinemaarts.com/dev/clusterThis/index.html [digitalcinemaarts.com]

And, here's a more or less complete list of current and planned features for this project. The docs, web page, examples, and of course the code, will be updated quite regularly, check back often.

Features:
(*) features coming soon

# Instance a variety of geometry primitives (sphere, cube, grid, circle, tube, curve (*), and BGEO file (*))
# Copy point attributes to the instanced geometry
# “Recursive” point generation to displace the instanced geometry “about” the position of the incoming point's position.
# Noise generator for additional displacement data
# Attribute driven behavior (pscale, speed stretching, etc.) and blending (via CVEX (*))
# CVEX processing (*)
# Application of volumetric modulation data. (*)
# Multi render pass optimizations (almost there, will be in next release) (*)
# Motion blur (almost there, will be in next release) (*)
# Volumes instancing
# CVEX processing (*)
# More sophisticated noise/filter functions (fBm, gaussian, bezel, impulse, butterworth, sinc) (*)
# Lacunarity and true fractalization (*)
# “RiPoints” instancing (*)
# RIB output (already there, just need interface change) (*)
# Proper primitive/point attribute blending/processing (CVEX *)
# Read volumetric data files and use for modulation of instanced geometry and attribute processing (DT grids). (*)
# Optimizations of all kinds (especially when generating deep shadow passes) (*)
# Velocity/Force stretching the prims and blending/interpolating the attributes (*)
# Primitive selection from point attribute (prim_type) (*)


Thanks,
Mark

Hi Mark,

Christmas came early! This is great!

Thanks!

Cheers!
steven

stevenong
Christmas came early! This is great!
steven

Hi Steven!

Good to hear from you! Hope you are well!

Yea, I'm finding the time to write something other than file translators :-)


Mark

NOTE: There's a new version of the first DSO with the copy/recursion loop fixed. I'll get the motion blur and a few other things working whilst figuring out the problem with my addProcedural() implementation.

New version available:

Now has metaball instancing (which you can also use to render volumes), simple deformation motion blur, a few odds and ends. The file instancing will kind of work, didn't have to finish everything for it, will have it done in the next few days. The shader strings are still a problem, need to iron that out ASAP as well as the velocity motion blur. I'll have more docs/help/examples in the next release too.

Please report bugs.


Enjoy,
Mark

Hola~

New version (1.3.7) available:
http://digitalcinemaarts.com/dev/clusterThis/index.html [digitalcinemaarts.com]

Added point instancing, fixed the file instancing and implemented shader point assignment if “vm”surface“ attribute is present in the incoming geometry (in deferred instancing mode). For the life of me I can't figure out how to get point based ”vm_surface" attribute with the Material SOP, that's why the Shader SOP is still in the example file.

The example file has a handful of takes for various examples.

Be sure to replace the OTL as well as the DSO/DLL when upgrading.


Take care,
Mark

good job man!

Greetings Earthlings,

New version (1.4.4) available:
http://digitalcinemaarts.com/dev/clusterThis/index.html [digitalcinemaarts.com]

* Added an optimization for when mantra generates deep shadow maps. The DSO has an option to write the instanced data to a temp file which is then read for each subsequent call from mantra. This is only available for the “generate all the geometry at once” method, since in the deferred instancing mode the geo instancing is controlled by mantra.

* Added simple CVEX processing for the instanced geometry. Point positions of the instanced geometry is passed to the VEX engine for further processing.

Notes:

* CVEX: Presently the CVEX hook is just a simple call passing only point positions into the VEX code. The arrays that are used to call the CVEX code is currently a static array as this was a “proof of concept” test for the project. I need to put a little thought into the most efficient way to dynamically build the arrays to pass to the VEX code so as many attributes as desired to be processed in VEX can quickly be assembled and shipped off for some SIMD goodness. This will be coming along quickly as there's much to be gained by leveraging CVEX at render time.

* Deep Shadow passes: To optimize the process of generating deep shadow maps, the user can now choose to have a temporary geometry cache file stored which is then read by the DSO for each additional deep shadow map and the final pass that mantra calls before starting to render. You may also choose to not delete the file in case you would like to use the instanced geometry later in your pipeline.

From Mark Elendt regarding why mantra calls a geo procedural for each light that gens a deep shad map:
“Consider generating an iso-surface. One light
really close to the iso surface, the other really far away. If you
generate a single piece of geometry for both views, then you'll end up
with either way way way too much detail or not enough…

Since the geometry can change based on the view, then we need to
regenerate the geometry each time.”

* Motion Blur: The motion blur is implemented as a deformation blur, duplicating the points and then moving them. For now the velocity based motion blur has to wait. I'm thinking that there should be an option to move the points with CVEX code so the user can adjust the motion blur in any sort of way they would like.

From Mark Elendt:
“That's a little tricky….
Right now, duplicate your geometry, then move one piece based on the velocity attribute.”

Here's the current method for displacing the duplicated geometry to create motion blur for mantra:

myPointAttributes.myMBPos = myPointAttributes.myNewPos - myPointAttributes.v.x();
myPointAttributes.myMBPos = myPointAttributes.myNewPos - myPointAttributes.v.y();
myPointAttributes.myMBPos = myPointAttributes.myNewPos - myPointAttributes.v.z();


The example file shows a few ways of how to use the DSO but because of some pesky license problems, I'm presently not able to save a non-NC hip file (don't want to distribute a .hipnc file); shouldn't be too much longer, than I'll have more examples using the CVEX feature.

Next pass I hope to have better CVEX code, the curve instancing, maybe true volume instancing (though you can currently instance meta balls and render them as volumes), a lot more examples, a few pretty pictures, and …

It time for a cleanup/modular dissection and code review … :roll:

Remember to be sure to replace the OTL as well as the DSO/DLL when upgrading.


Enjoy,
Mark

Hey there Mark - thanks for working on this!

Just a heads up, when I try to use the dso on:

SUSE 10.2 64 bit with H9.5.356

I get:

Houdini DSO - error on ‘/home/jc/houdini9.5/dso/VRAY_clusterThis.so’
/home/jc/houdini9.5/dso/VRAY_clusterThis.so: undefined symbol: _ZNK11VRAY_Volume8getXformER10UT_Matrix4


when starting Houdini. When I compile the dso myself, I get the same thing. Compiling seemed to run OK:

Makefile:54: warning: overriding commands for target `clean'
/opt/hfs95/toolkit/makefiles/Makefile.gnu:39: warning: ignoring old commands for target `clean'
g++ -c -DVERSION=\“9.5.356\” -DDLLEXPORT=“” -DMAKING_DSO -D_GNU_SOURCE -DLINUX -m64 -fPIC -DAMD64 -DSIZEOF_VOID_P=8 -DSESI_LITTLE_ENDIAN -DENABLE_THREADS -DUSE_PTHREADS -DENABLE_UI_THREADS -DGCC3 -DGCC4 -I/opt/hfs95/toolkit/include -I/opt/hfs95/toolkit/include/htools -Wall -W -Wno-parentheses -Wno-sign-compare -Wno-reorder -Wno-uninitialized -Wunused -Wno-unused-parameter -Wno-deprecated -O2 -DMAKING_DSO -DUT_DSO_TAGINFO='“3262197cbf184a342daf06810764cfa2416ddafddf2bd5a137b692560d0ac79e32de0385dd327f34535fb7bd71c0200f4b8b8664efd0e9544a3c3df877ea81ac5bbe45e2a68849dee046c3410cf33b”' VRAY_clusterThis.C -o VRAY_clusterThis.o
/opt/hfs95/toolkit/include/VM/VM_Math.h: In static member function ‘static void VM_Math::set(fpreal32*, fpreal32, int, const uint32*)’:
/opt/hfs95/toolkit/include/VM/VM_Math.h:333: warning: dereferencing type-punned pointer will break strict-aliasing rules
VRAY_clusterThis.C: In member function ‘int VRAY_clusterThis::runCVEX(GU_Detail*, GU_Detail*)’:
VRAY_clusterThis.C:2162: warning: unused variable ‘N_val’
VRAY_clusterThis.C:2162: warning: unused variable ‘v_val’
VRAY_clusterThis.C:2162: warning: unused variable ‘Cd_val’
VRAY_clusterThis.C:2162: warning: unused variable ‘Alpha_val’
VRAY_clusterThis.C:2162: warning: unused variable ‘pscale_val’
VRAY_clusterThis.C:2162: warning: unused variable ‘id_val’
VRAY_clusterThis.C:2162: warning: unused variable ‘weight_val’
VRAY_clusterThis.C:2178: warning: unused variable ‘N_out’
VRAY_clusterThis.C:2178: warning: unused variable ‘Cd_out’
VRAY_clusterThis.C:2178: warning: unused variable ‘v_out’
VRAY_clusterThis.C:2178: warning: unused variable ‘Alpha_out’
VRAY_clusterThis.C:2214: warning: unused variable ‘newN’
g++ -shared VRAY_clusterThis.o -L/usr/X11R6/lib -lGLU -lGL -lX11 -lXext -lXi -ldl -lpthread -oVRAY_clusterThis.so


Running gcc (GCC) 4.1.2 20061115

Any thoughts?

Cheers,

J.C.

Hi John,

I think it's because the DSO needs to be in a location that Houdini doesn't source at startup ($HOME/houdini9.5/dso/mantra).

Try moving it to the above location (mantra looks there by default) and see if that fixes it.

Sorry about all the warning when compiling, I had a bunch of unused vars hanging on that will be used later when I get more extensive CVEX processing.

I'll check back later today to see if it's working OK for you, I am sick in bed with a damn cold. YUCK! SNEEZE! COUGH!


Mark

User fail! Sorry..

Hope you're feeling better soon…

Cheers,

J.C.

JColdrick
User fail! Sorry..

:-)

Have a great weekend John!


Mark

thank you very much.

Could I render the “volume sprite” with cluster this now?

instance the volume objects onto particles.

ykcosmo
Could I render the “volume sprite” with cluster this now?
instance the volume objects onto particles.

No, not quite yet. I will have volume primitive instancing implemented in the next version, sometime over the next month. I wanted to work out the rest of the CVEX idea and am releasing that version this evening.


Mark

Hello,

OK, I have a better mechanism for the CVEX support, and added the velocity based motion blur for point instancing.

Notes:
1) For point instancing P, as well as v, N, Cd, Alpha and pscale are passed to the VEX engine and returned; id is passed in too but not returned. Therefore you can alter any attribute including the point's position.

2) For primitive, metaball and file instancing P is not sent & returned to the VEX engine, (this is because I'm dealing with primitive attributes as opposed to point attributes). I need to figure out a way to handle this, because there can be differing array sizes needed for different types of primitives.

3) For metaball instancing the weight attribute is passed to VEX and returned.

4) The motion blur for point instancing is now calling addVelocityBlurGeometry() instead of using deformation blur. So, as usual, the velocity vectors and shutter time will determine blur.

After working with the CVEX API, it's got some serious promise for all kinds of applications, very nice to have this available in the HDK! I'm thinking that there could be a few places in this DSO that could take advantage of it, e.g. pre-processing incoming geo, processing the instanced geo (as is done now), and further processing of the motion blur copy of geo required for deformation blur. It could be used for filtering and attribute blending as well, not to mention the processing that could be applied to instanced volume primitives. Coolio!

http://www.digitalcinemaarts.com/dev/clusterThis/index.html [digitalcinemaarts.com]

Have fun.


Mark


P.S. The example file is unfortunately a .hipnc file, my apologies, this will be remedied soon.

Greetings,

Update: Version 1.4.6 available at:
http://digitalcinemaarts.com/dev/clusterThis/index.html [digitalcinemaarts.com]

New features and fixes:

* Added geometry velocity blur.
The velocity motion blur now calls addVelocityBlurGeometry() and the deformation motion blur is calculated by clusterThis DSO as before (remember when using deformation blur you will be doubling the amount of geometry generated).

* Added volume instancing.
Simple volume instancing. I started creating extensive Voxel arrays but it became clear that would better executed in a separate project so the volume instanced is a single “volume object” per location generated.

* Added curve instancing.
A NURBS curve is created per incoming point, created from the points generated from the instancing loop.
(Not quite working yet, having issues with part of the curve remaining at world origin)

*Optimized attribute processing.
Cleaned up a bunch of the attribute creation and assignment code, will reduce overhead.

*Added motion blur pass for CVEX processing.
Added a loop for processing deformation motion blur geometry to the CVEX processing code.

*Changed parameter code to use proper functions.
Changed the import() function to the proper getFParm()/getIParm()/getSParm() functions.

*Added verbosity switch for console messages.

*Added randomness for when to instance of objects
There's now a control on the OTL to control the “probability” of an object being instanced.

*Using SYSsin() and SYScos () instead of std C functions.

*Optimized memory usage for CVEX processing, correct memory allocation for attributes and objects.
Now only the exact amount memory is allocated for the CVEX processing based ob geometry size and the user selected attributes to be sent to the CVEX processing engine.

*Added user selectable attributes for CVEX processing.
Allows the user to select which attributes to be sent to the CVEX engine so the VEX code you write will have those data available and only the memory required for that processing is allocated.


There's a number of tips and notes I need to compile that will be helpful for using this clustering tool, I'll post those soon. Much testing is in order now as there's a few odd things I can't figure out related to using the addProcedural() method (“deferred processing”), which I would think would have a number of benefits, doesn't seem the case. Please send any bug reports. And as always, use with caution, it's open source … m'kay?

Enjoy.


Mark

congratulations mark!


i am playing around with this now

great example file too


i will keep you updated on my progress

thanks!

erich

[quote=“sidenimjay”
i am playing around with this now

erich

Hi Erich!

Man, long time!!! Yea, beat the hell out of that DSO, I want it wrung out and stable, it's getting a lot of use already; there's a bit more work to do but it's getting to be very helpful. I'm using it for a number of things not least of which making tons of spray and mist, but also some odd ball fx from just futzin' around with it. I do love happy accidents.

Once I have the remaining features completed, the code will go through a heavy “rinse/repeat as often as needed” for optimization, etc.

Stay in touch!


Take care,
Mark

dear lord baby jeebus…you have given me so much inspiration with those examples…THANKS YOU

For brevity (and less redundancy), the new version announcement may be found at our friends at odforce:
http://forums.odforce.net/index.php?/topic/7916-mantra-clusterthis-procedural-geometry-dso/page__st__50__gopid__66725&#entry66725 [forums.odforce.net]