Some of you may have seen Nvidia's new NueralVDB tech [developer.nvidia.com] (100x memory footprint improvements over standard VDB). Obviously, I would love to see this make it's way into Houdini. I wouldn't be surprised if some larger studious have already managed to integrate this with Houdini, as it's apparently already being used in feature films.

From what I understand, it seems like NueralVDB files can work in existing of VDB piplelines, so should be a pretty simple update right?

Honestly I wonder why Houdini hasn't really embraced some of the new AI powered simulation approaches, as described in some of the videos on To Minute Papers [www.youtube.com]. I know the space is moving quickly, but isn't simulation one of Houdini's greatest strengths, so why not embrace these advances?

In a couple of years I think AI simulation tech will make working with (modest) simulations in anything less than real time feel antiquated. I really don't think you should need outrageous hardware to do fluid and volume simulations. I have a desktop with a decent CPU and GPU but the latency I get when working with volumes and fluids in Houdini is kind of a big put off from embracing these workflows.

I agree, it would be nice for Houdini not to lose its eye on innovation. Other than implementing most recent methods in neural graphics I would love to see Houdini enable users to make tools of their own. A Tensor Operator Context (TEOps) with automatic differentiation or a differentiable operators context (DIFOps) where every operator can compute gradients for backpropagation (like NVIDIAs Warp) would be great. If you look at Neural Fields, Differentiable Simulations, PINNs there is a lot out there that Houdinis toolset is not adapted to yet but will soon become relevant for vfx.

I would love to see something similar to Nuke's CopyCAT node. So i could train and then use models inside Houdini without worring about export, TensorFlow or whatever relatet to programming. Lates say i create character with muscles and skin sliding, then run simulation of different poses over night, connect this AI node so it could learn all the deformations related to bones/KineFX and finally animate using trained model with realtime deformations! That would ROX of course it would be nice to train points against pixels (animated textures or video input) basicly train every possible data type inside Houdini against anything. To be honest iam learning Houdini becouse i strongly belive it will be possible in near future. Houdini is made for AI

This breakdown the recent advances in the tech very well.

I'd love to see someone implement a basic example in Houdini.

Some relevant github repos:
https://github.com/maziarraissi/PINNs
[github.com]
https://github.com/alexpapados/Physics-Informed-Deep-Learning-Solid-and-Fluid-Mechanics [github.com]
https://github.com/PredictiveIntelligenceLab/USNCCM15-Short-Course-Recent-Advances-in-Physics-Informed-Deep-Learning [github.com]
https://github.com/tensordiffeq/TensorDiffEq
[github.com]

on twitter SideFX are indicating they are searching for devs dedicated to ML workflows
so, good things coming our way sooner or later

Honestly I might give applying a basic PINN a go.

ChatGPT gave me this nice example of the general steps on how to go about using a PINN to speed up a physics solve, btw!

I think the tough part of using the PINN, from my very basic understanding, is that unless you spend the time to build a very large data set, the PINN you train will have a fairly narrow set of applicable use cases.

But lets say you need to refine and iterate a simulation many times, or create many instances of a certain kind of simulation, then the PINN could greatly accelerate that.

Step 1: Define the geometry of the system that you want to simulate.

In Houdini, you can create a 3D model of the system that you want to simulate using its built-in modeling tools. For example, you could create a simple geometry, such as a ball falling onto a plane.

Step 2: Use a physics engine to simulate the system and generate observational data.

In Houdini, you can use its built-in physics engine to simulate the system and generate observational data. For example, you could set up a rigid body simulation where the ball falls onto the plane and bounces off.

Step 3: Train a PINN using the observational data and the physical laws that govern the system's behavior.

To train the PINN, you need to provide it with both the observational data and the physical laws that govern the system's behavior. For example, in the case of the ball falling onto a plane, you could provide the PINN with the initial position and velocity of the ball, as well as the laws of gravity and collision.

In Houdini, you could use a machine learning framework, such as TensorFlow or PyTorch, to create and train the PINN. You would need to define the PINN architecture and loss function, and then train the network using the observational data and physical laws as inputs.

Step 4: Use the trained PINN to predict the system's behavior under different conditions, instead of simulating it using the physics engine.

Once you have trained the PINN, you can use it to predict the system's behavior under different conditions. For example, you could use the PINN to predict how the ball would behave if it was dropped from a different height or if it was made of a different material.

In Houdini, you could create a new simulation where you input the different conditions and use the trained PINN to generate the simulation results.

To summarize, in this basic example, you could use Houdini to create a simple 3D model, simulate it using its built-in physics engine, train a PINN using the observational data and physical laws, and use the trained PINN to predict the system's behavior under different conditions. This approach can significantly reduce the computational cost of the simulation, as the PINN can make predictions much faster than the physics engine can simulate the system.

orr
I agree, it would be nice for Houdini not to lose its eye on innovation.

hahaha, You made my day. Oh, wait, You mean that seriously.

wyhinton1
Honestly I might give applying a basic PINN a go.

ChatGPT gave me this nice example of the general steps on how to go about using a PINN to speed up a physics solve, btw!

I think the tough part of using the PINN, from my very basic understanding, is that unless you spend the time to build a very large data set, the PINN you train will have a fairly narrow set of applicable use cases.

But lets say you need to refine and iterate a simulation many times, or create many instances of a certain kind of simulation, then the PINN could greatly accelerate that.

Step 1: Define the geometry of the system that you want to simulate.

In Houdini, you can create a 3D model of the system that you want to simulate using its built-in modeling tools. For example, you could create a simple geometry, such as a ball falling onto a plane.

Step 2: Use a physics engine to simulate the system and generate observational data.

In Houdini, you can use its built-in physics engine to simulate the system and generate observational data. For example, you could set up a rigid body simulation where the ball falls onto the plane and bounces off.

Step 3: Train a PINN using the observational data and the physical laws that govern the system's behavior.

To train the PINN, you need to provide it with both the observational data and the physical laws that govern the system's behavior. For example, in the case of the ball falling onto a plane, you could provide the PINN with the initial position and velocity of the ball, as well as the laws of gravity and collision.

In Houdini, you could use a machine learning framework, such as TensorFlow or PyTorch, to create and train the PINN. You would need to define the PINN architecture and loss function, and then train the network using the observational data and physical laws as inputs.

Step 4: Use the trained PINN to predict the system's behavior under different conditions, instead of simulating it using the physics engine.

Once you have trained the PINN, you can use it to predict the system's behavior under different conditions. For example, you could use the PINN to predict how the ball would behave if it was dropped from a different height or if it was made of a different material.

In Houdini, you could create a new simulation where you input the different conditions and use the trained PINN to generate the simulation results.

To summarize, in this basic example, you could use Houdini to create a simple 3D model, simulate it using its built-in physics engine, train a PINN using the observational data and physical laws, and use the trained PINN to predict the system's behavior under different conditions. This approach can significantly reduce the computational cost of the simulation, as the PINN can make predictions much faster than the physics engine can simulate the system.

Besides PINNs, differentiable simulations are a very interesting field right now. You can check out nvidias wrap for example here [developer.nvidia.com]. One of the ideas is that you can compute the gradient of your input parameters through the simulation. This would in theory allow you to optimize for the initial parameters of your simulation from a given final state.
For example, and was shown in a talk by Miles Macklin, you want a piece of cloth to land on a ball, and by using the automatically computed gradient you can optimize for the initial velocity and force that will cause the cloth to land on the ball. This is a very simplistic example, but I think it gives an idea of where this could go, and it is already used in the deep RL community.
There is a houdini wrap plugin, but it does not allow you to perform gradient computations yet as far as I know.