PHENOMDESIGN
Inlinecpp too that you can reference in the nanovdb headers or a compiled Julia library . There will have to be some clever decomposition, abstraction, or/and multi-res.

I mean I'd rather do full on HDK than this tbh.

There's no need to have compiled Julia Libs or even implement NeuralVDB to have good performance to simulate a "realtime" or at least fast Smoke solver. I just need more control on the memory and avoid data transfer done when using nodes. I believe this is easily achievable.

How do I define performance ?
You can take Embergen as a benchmark as I believe this is peak performance for fluid solving. The guy who's working on it is really smart.


how large is the simulation domain
I'm not working on a specific simulation but on a solver, so with my current hardware I can probably go up to 500M active voxels.
So let's say a 1500^3 grid max with a 0.05 voxelscale

Real-Time Fluid Dynamics Simulation in Julia [www.youtube.com]

I would imagine the HDK is good. The inclinecpp compiles for you and you can grab that library.

Embergen is an optimized abstraction. Like I said, you need clever decomposition, abstraction, or/and multi-res and precomputed components.

"Since we only use specific VDB features at JangaFX we are replacing OpenVDB with a custom writer that we can optimize for our needs." -- VDB: A Deep Dive [jangafx.com]

If you want to catch Embergen you need device specific kernels not OpenCL.

Also, are all the OpenCL as sequential as possible and there is a compile block encapsulating them? That will keep it from returning to the CPU and keep it on the GPU.

Watch the Invoke Compile Blocks | Jeff Wagner | Houdini Illume [vimeo.com] to see if you can then further optimize into a compute graph that is invoked.

I could not see the image as it was too small.

Thanks for the ressources, yes I imagine OpenCL is not suitable for deep optimisations. For me it was a first step before digging deeper .
Also I take Embergen as the optimisations upper limit, I probably won't achieve that.

If anyone reading this thread is interested / want to contribute feel free to contact me !

ZephirFX
Also I take Embergen as the optimisations upper limit, I probably won't achieve that.

Check out Prof. Steve Brunton's Youtube:
https://www.youtube.com/@Eigensteve


You will be surprised. Physics-based AI will make solving physics in real-time trivial and accessible to anyone on any machine. You will not be "simulating" but inferencing the physics-defined space. This will give guaranteed memory draws and provide immediate feedback.

Specifically, Houdini has the foundation for the next class of Neural Networks based on continuous weights such as splines and Reservoir computing.

The compression ratio is just unbelievable. I would love to see this tech implemented in Houdini

Antti1999
The compression ratio is just unbelievable. I would love to see this tech implemented in Houdini

NeuralVDB is still not currently open-sourced yet and has been around since 2022. The OpenVDB version of this seems to be the fVDB branch on the github here: https://github.com/AcademySoftwareFoundation/openvdb/tree/feature/fvdb [github.com]



Unfortunately it is all pretty CUDA specific, a very ecologically damaging GPU framework. It is cool that Nvidia GPUs can simulate but there is not getting away from the noise, size, cost, heat, electricity use, and future bottle necks.

Houdini should implement this with the Anari Renderer to future proof their render for neural renderers.

ANARI - The Industry's First Portable Rendering Engine API - SIGGRAPH 2023 BOF Session [www.youtube.com]

Including the VTK-m filters VTK toolkit of scientific visualization algorithms for emerging processor architectures. [m.vtk.org]

I also wonder if there is a better way that does not have to be bound to the implementation of CUDA or specific for rendering like SuiteSparseGraphBlas:

SPAA Keynote Talk: Large Scale Parallel Sparse Matrix Streaming Graph/Network Analysis

On a less speculative note, I believe this person is working on some Differentiable Fluid Sim in Houdini : https://github.com/HinaPE [github.com]
Also there's some realllly interesting stuff on his profile.

I do want to note that fVDB is released already and you can use it in Houdini. Specifically for Houdini, Sparse GraphBLAS enriches the whole paradigm and are packages that are OpenSource.

Maybe none of this is speculative? Could be that it is already done and new to you.

ZephirFX
Differentiable Fluid Sim in Houdini

Nice, from what I can tell it is Taichi that is backing the Differentiability?

PHENOMDESIGN
Maybe none of this is speculative? Could be that it is already done and new to you.

Yes true, but I'm afraid the complexity is absolutely overwhelming for someone who's not a specialist in the field :/

PHENOMDESIGN
Nice, from what I can tell it is Taichi that is backing the Differentiability?

yess looks like it.

I started a few thing to get myself familiar with the HDK and openvdb :
https://github.com/ZephirFXEC/HNanoSolver [github.com]

ZephirFX
I started a few thing to get myself familiar with the HDK and openvdb :
https://github.com/ZephirFXEC/HNanoSolver [github.com]

Awesome! I am excited to see how this develops!

You inspired me to look further into PhiFlow with your references and saw that the new version released.
Good looking out. I like the work this group is doing with this and Mantaflow.



You may be aware of this but this project is an example of bringing Mantaflow into Houdini with Tensorflow.

https://github.com/Ozeuth/Houdini-Plugin-for-Tensorflow-Smoke-Stylization [github.com]