Houdini allows you to set up the networks of a simulated object so that it can be solved in parallel on multiple machines using HQueue. HQueue is a render and simulation job management system, consisting of a HQueue Server, HQueue Clients, workstations, and a Shared Folder.

• The server is responsible for distributing your task to different client machines. The server also provides a web interface, where you can monitor and manage your jobs. The server is normally installed on a single computer.

• A client receives jobs from the servers and performs the incoming simulation and render tasks. Sometimes, clients are also called nodes. The machine with the server running can also be configured as client. The client must be installed on every computer you want to use for a job.

• Users, working in Houdini, submit jobs from a workstation to the HQueue Server which are then distributed to the HQueue Clients.

• The Shared Folder is mounted as a shared drive on the network. Workstations transfer input files to the folder to make them accessible for the HQueue Clients. The HQueue Clients, on the other hand, write output files to the Shared Folder. The workstations are then able to read the output files.

Before you can start distributing simulation jobs, you have to install the HQueue Server and Clients. To do this, download the Houdini installer from the SideFX website and execute it. Note that you normally need administrator privileges to install the software. Specify, which computer you want to use as

• HQueue Server. The HQueue Server must be installed on one machine only.

• HQueue Client. The HQueue Client must be installed on all machines that should contribute to the job.

• Shared Folder. The Shared Folder is defined in the HQueue Server’s web interface. Make sure that all machines have appropriate privileges to access this folder.

Most of Houdini’s simulation networks can also be created inside SOPs, e.g. Vellum, fluids or Rigid Body Dynamics. However, distributed simulations are currently only possible with DOP-based workflows. To be more precise: it’s possible to send a SOP-based simulation job to a single HQueue Client, but you can’t distribute the job over several machines.

You can distribute POP particles, non-adaptive particle fluids and dense pyro simulations using slicing. There, the space in which the effect happens is partitioned and each slice is solved on a separate machine. You can slice grid-based fluid containers using a certain number of divisions per side, but for particle fluids simulations that do not have boundaries, you have to slice them manually. For sliced simulations, the nodes on the network talk to each other to communicate information across the boundaries between slices.

Dense pyro simulations, on the other hand, can also be clustered through a Cluster SOP to subdivide the pyro source particles into segments. Each cluster container solves independently, even if they overlap with other cluster containers. For smoke and pyro simulations this works well since their output volumes blend together visually.

Particle fluids are certainly the most interesting solver type for distributed simulations, because they often require long simulation times. To ease the task of preparing a project for distributed simulations, you can use various shelf tools. They're all located under the Particle Fluids shelf tool.

1. Open the Particle Fluids shelf tab.

2. ⌃ Ctrl click the FLIP Tank shelf tool to add the tank at the scene’s origin.

3. From the same shelf, click the Distribute Particle Fluid shelf tool.

4. You must select a particle fluid to run the tool. From the viewport’s left icon bar, choose the Select tool and draw a rectangle around the fluid in the viewport. Then, press Enter to confirm your selection.

5. Houdini jumps into the out network you can see a distributedsim_fliptank node. This node submits the job to the HQueue for distribution on the farm.

6. In the Output Driver you can see a path. If you have followed the steps above, the path is /obj/distribute_fliptank, while save is the node you're referencing to.

7. Open the path in the Network Editor. The save object is a Geometry Output ROP and performs the simulation by clicking Save to Disk.

You can select the distribution method you want to use from the Partitioning Type drop-down menu located in the distributedsim_fliptank HQueue Simulation render node parameters.

Slicing the space containing a particle fluid allows different machines on the HQueue farm to simulate different regions of the fluid. You can send a particle fluid simulation to the farm without slicing, but it will just simulate on one machine instead of multiple machines in parallel.

There are two shelf tools on the Particle Fluids tab for slicing a particle fluid simulation into separate regions for distribution:

• Slice along line tool

  Creates a specified number of equally spaced slices along a certain direction. This is the simplest way to slice up space.

  

• Slice tool

  Slices the space using a hierarchy of intersecting planes which arbitrarily divides the space in complex ways.

  

Each fluid region requires an individual machine on the HQueue client network, so you can only set up as many regions as you have machines. For example, using the Slice along line tool with Number of regions set to 8 would require 8 machines on the HQueue network.

1. Create a clustered smoke or pyro simulation from point sources using the Smoke Cluster or Pyro Cluster tools on the Pyro FX shelf tab (see clustering).

2. On the Container Tools shelf tab, click the Distribute tool.

3. Select the container you want to distribute and press Enter.

   If you are distributing a clustered simulation with multiple containers created by the smoke/pyro cluster tool, you can just select one of them.

   (Alternatively you can select the DOP object in the network editor and then press Enter in the viewport.)

4. If the object has multiple containers (for example, it was created using clustering), a dialog box appears asking if you want to distribute the simulation using slices or clustering.

   (If your simulation is so massive even a single clustered container at a time is too much to run on one machine, you can click Slicing to distribute separate slices of each clustered container.)

5. If for some reason you have more than one object importing the results of the simulation, a dialog box will ask you which object should be set up for distribution (see how the tool modifies the geometry network below).

6. The tool will create and select a HQueue simulation render node. Set the node’s HQueue server parameter to the name of the HQueue server machine and port name (for example hq.company.com:5000).

   If you are slicing, you can set the Slice Divisions parameter. The default 1, 1, 1 slices does not actually divide the container into multiple subdomains. Increasing these values to 2, 2, 2, for example, will slice the container into 8 smaller boxes for distribution.

Once the HQueue network has generated the geometry, you should stop the original simulation network from cooking since you now have the end-result cached.

Do one of the following:

• Turn off the visibility of any scene-level objects that require the simulation to cook.

• Turn off simulation in Houdini: right click (or pressing and holding on) the simulation menu in the bottom right corner of the main window and turn off Enable simulation.

• Start a fresh new .hip file that simply loads the generated particle fluid geometry files using a File surface node.

1. In the main menu, choose Render ▸ Edit render node ▸ distributedsim. In the parameter editor:

   • Set the frame range you want to simulate.

   • Check the Slice divisions parameter. The product of the three components should equal the number of regions in your particle fluid simulation (for example, 3, 1, 1 for a simulation with 3 regions).

     This number is set automatically when the Distribute tool creates the render node. However, if you then go back and change the number of slices (for example, by changing the Number of regions parameter on a Slice Along Line node), you need to update this parameter on the render node.

   • Set the HQ Server to the hostname and port of the HQueue server, for example render.example.com:5000.

2. In the parameter editor, click Render to send the job to HQueue.

   The render node sends its information to the HQueue server. The HQueue server sends commands to its client machines to load the .hip file and simulate and save a slice of the particle fluid simulation per-frame. The individual client machines save their generated geometry cache files to the shared network directory.