The video [www.youtube.com] showcasing distributed particle sims has been around since 2015, but only recently it occurred to me that this technique only applies to DOPs?

Am I correct in saying that there is no general solution to distribute SOP cooks? For example, slicing and aggregating (map-reducing) VDB clouds or any VEX snippets on large meshes?

PS Duplicate of this post [forums.odforce.net].

For loops+compile blocks are designed for this.

Ah, but not across multiple machines though out of the box, that's getting into tops which is a whole other can of worms.

mestela
Ah, but not across multiple machines though out of the box, that's getting into tops which is a whole other can of worms.

You don't really need TOPs. A simple global variable to wedge the scene across multiple jobs on a render farm is all you need. It's a pretty standard workflow that predates TOPs.

dankray
Am I correct in saying that there is no general solution to distribute SOP cooks? For example, slicing and aggregating (map-reducing) VDB clouds or any VEX snippets on large meshes?

yes, you are correct, there is no mehnaism in sops that would handle that for you on a single connected geo or volume

however if you are willing to take care of splitting the computation to bitesized chunks on separated/sliced geos/volumes and handle combining those together (if needed) yourself, then above mentioned methods like TOPs would be the way to go if your goal is to compute them separately on the farm

Thanks for the replies. Yes, I realized that the reason DOPs need their own distributed solution is because the slices need to COMMUNICATE with each other in the runtime. They also need to run in lockstep. I'm sure there are plenty of other reasons, as well.

But, but - I think that it still would be great if the SOP geometries could be easily sliced and distributed in one nifty node. Maybe that's an idea for another day. There was a GameDevTools once - maybe it's time for SciDevTools now

I think that this problem must have been documented somewhere else - surely various games must have come across it, it's not too different from multithreading. I mean what problems come up when distributing stateless geometry processing tasks.

well, if they are the same geo or volume then that's pretty much impossible, there is so many sops that need to be aware of the whole geo, what you would lose any advantage of it being split up and any sort of communication among machines would create too big overhead

only true independent geometry pieces can be split and that's super easy with TOPs, so in that sense the solution is already there