I always love to convolve audio files together. not just the typical audio convolution of an impulse response of sistene chapel convolved with my shredding guitar solo(jklol), but I also like to play around with convolution as a sound design tool…

it's so interesting to me to hear one sound get totally grokked with another.

Now, i've always wondered what it would look like if an analogous operation could be performed on video.

I know that image filter blurs and stuff like that use a small convolution kernel to work their magic, but I feel like it's not really the same thing… it isn't really operating in the time domain..

So, i was playing around with COPs today (do COPs play around irl?) and I noticed that there is a convolve COP, and that it lets you specify an image file as the convolution kernel…

What I'm wondering is, how could I set up something that convolves 2 image sequences in the time domain, (would i have to use a foreach (?) and convolve each image of one sequence with each image of the 2nd sequence?)

and also, how would you deal with color channels? would you convolve each channel separately and then just recombine the output? Is there a proper way to extend convolution to multiple dimensions? is a normal blur effect (or whatever) considered a 3 dimensional operation? (i'm thinking the image is 2D and then it has a dimension that is like the intensity(?) of each pixel in the kernel?) Is that how it is really processed, or is the kernel just an single array of values? Is the single intensity(is that the right word?) channel of the convolution kernel applied to each color channel of the image, separately?

does any of this make sense? i have always imagined video convolution being a super trippy effect if I set it up right.

Although, i feel like what i'm talking about would be painfully slow in houdini, unless maybe it was run an a monster cpu with many cores…

I'm guessing houdini does this stuff with the cpu, right?


If anybody can clear the fog from around my head with respect to this topic, i would be stoked! If anybody could help me set something like this up that works and doesn't take a year to compute, I would be super stoked! If it actually looks cool once computed, I would be ultimately stoked.

Until then, i'm just gonna hook a camera to a projector and point them both at the same wall… lol

If you mean convolve the same (x,y) pixel in multiple frames over time, there is the TimeFilter COP. It is essentially doing a 1D window convolve, though you can't customize it, just select from some premade windowing filters. There's also the TimeMachine COP, which samples a range of frames on a per-pixel basis, based on the values in the 2nd image. Beyond that you'd have to write a VEX filter COP to do the frame sampling you'd like.

Convolve itself is just a NxN kernel filter over a single image.

Do you have an example of the audio convolution you are talking about? I am interested in these kinds of stuff.

Twod, thanks i will play with those nodes. For some reason my brain has been on 2D effects!

Ani, one time i built a hackintosh osx86 machine and i was using some freeware/abandonware audio editor that allowed me to load up two audio files and convolve them. It also let me draw a graph which would weight between the two files as a function of time.

You could effectively use the technique to morph one sound into another one.

I am no mac guy and unfortunately, i cant remember for the life of me what that program is called, and i never encountered that exact function again. Although i am guessing that the graph was just acting as a crossfader between the two files before they were fed into the convolution algorithm.

Umm anyway, these days you always see realtime convolution reverb plugins. They aren't exactly the same thing but close. (And if u havent tried one for its intended purpose, you should, they sound a billion times better than a typical digital reverb)

There have also been a handful of plugins recently that will interpolate between a number of “impulse responses”. This is cool because it can emulate vintage hardware complete with controls…

Of course, it would prolly be cooler if i just loaded up a bunch of random weird samples and interpolated through those. Haha

Anyhow, the claim to fame of this new crop of plugins is that they can give you very nice copies of analog hardware. So of course they come with jigabytes and jigabytes and cost $300 or so. I am thinking of acustica audio nebula n4 and liquidsonics seventh heaven.


If, like me to you dont really wanna spend that kinda dough at the moment. You can find some freeware convolution reverbs and a number of DAWs come with one…

But what about convolving 2 arbitrary audio files? Well, i found this java program (which i have yet to try)

https://code.google.com/archive/p/convolve-j/ [code.google.com]

Also, you could use matlab or max or csound or puredata or python, etc.

I wonder if there is a way in CHOPs…

TimeMachine COP makes me think of kung fury. I can hack you back in time!

@animatric3d: there is a sound designer called Diego Stocco who uses convolution processing to manipulate audio.
Here are a few links to is technique:

https://www.youtube.com/watch?v=LvJWlKBW0Kk [www.youtube.com]

https://www.youtube.com/watch?v=ABMBcstWZqU&t=3s [www.youtube.com]

https://www.youtube.com/watch?v=ek_ugqI7Qf8&t=96s [www.youtube.com]

Oh, i came across another audio plugin software today that does convolution. It is called Izotope Trash 2. It is mostly geared towards making weird, distorted sounds. i kind of find the distortions to be a bit harsh, but the convolution section sounds nice (i've never heard convolution that doesn't sound nice, tho..). It doesn't have all the features i would like to have though. Like it is awkward to edit the length of the audio file used for the convolution.

Maybe I should dust off one of those signals and systems books i have collecting dust and see if i can figure out how to implement some kind of convolution asset in VEX.. because I don't spend enough time making pointless weird things with Houdini!

I mean, if you think about it, it's just signal processing, so why not convolve your favorite music with your favorite artwork? maybe i could code it up so that the amplitude of each sample becomes the intensity of each cell of the convolution kernel.

Or vice versa, maybe try extracting an audio file from a black and white picture?

or maybe you could use the chroma and luma values somehow or just one color channel… or just convolove the Red channel with the blue panel and play it back for the green channel on your hifi?


12++