Houdini 20.5 HQueue

Installation

How to set up a basic HQueue farm.

On this page

Before you begin

Note

This guide covers the setup of an HQueue farm where the server and clients all share the same operating system (Linux, macOS, or Windows).

Although you can setup HQueue farm with mixed operating systems, this does not cover it.

The following items are required before setting up an HQueue farm:

Network connectivity

  • Machines must see or ping each other by their domain (DNS) names or IP addresses.

  • Client machines must connect to the server machine through port 5000. The server machine must connect to client machines through port 5001. This may require opening ports in any local firewalls.

Shared network folder

  • The server and clients must access a shared network filesystem, for example an NFS mount or Samba (SMB) share.

  • The shared folder must be mounted at the same location on every machine, for example /mnt/storage/ on Linux, or H:/ on Windows.

Houdini

  • Houdini must be installed on every client machine and in the same install directory.

    Linux

    On Linux, the default Houdini installation path is /opt/hfsX.Y.ZZZ.

    Mac

    On macOS, the default Houdini installation path is /Applications/Houdini/HoudiniX.Y.ZZZ.

    Windows

    On Windows, the default Houdini installation path is C:/Program Files/Side Effects Software/Houdini X.Y.ZZZ.

  • Houdini licensing must also be set up on each client machine so the client software can launch licensed executables such as hython, mantra and karma.

Installing HQueue

Setting up an HQueue farm involves:

  1. Installing the HQueue server,

  2. Registering a network folder with HQueue,

  3. Adding client machines to HQueue.

Testing the Farm

To test that the HQueue farm is configured correctly you can submit a simple Houdini render job:

  1. Launch Houdini.

  2. Create a simple scene. At a minimum, create some geometry, a light, a camera and a Mantra ROP for rendering.

  3. Navigate to the /out node network and create an HQueue Render ROP.

  4. In the Parameters pane, set the Output Driver parameter to your Mantra ROP.

  5. Point the HQueue Server parameter to your HQueue farm.

    For example, http://serverMachine:5000.

    Note

    If DNS is not configured for serverMachine then you can alternatively use the server machine’s IP address in the URL. For example, http://X.X.X.X:5000.

  6. Then set Target HFS to where Houdini is installed on the client machines.

    Linux

    For Linux, set Target HFS to:

    /opt/hfsX.Y.ZZZ
    

    Mac

    For macOS, set Target HFS to:

    /Applications/Houdini/HoudiniX.Y.ZZZ/Frameworks/Houdini.framework/Versions/Current/Resources
    

    Windows

    For Windows, set Target HFS to:

    C:/Program Files/Side Effects Software/Houdini X.Y.ZZZ
    
  7. Save the .hip file in your (HQROOT) network folder. You can find the HQROOT network folder settings on the HQueue webs interface in the Network Folders web page.

    Linux

    On Linux, the network folder is defined by the Linux Mount field for the HQROOT network folder. For example, save the .hip file to /mnt/myShare/test.hip.

    Mac

    On macOS, the network folder is defined by the macOS Mount field for the HQROOT network folder. For example, save the .hip file to /Volumes/myShare/test.hip.

    Windows

    On Windows, the network folder is defined by the Windows Mount field for the HQROOT network folder. For example, save the .hip file to H:/test.hip.

  8. At the top of the Parameters pane, select Submit Job.

    • A new job appears in the HQueue web interface.

    • You can select the job’s id to view more details.

Troubleshooting

When a job fails, you can troubleshoot by selecting the job’s id in the HQueue web interface. This takes you inside the job to view more details. If any of the job’s Child Jobs failed, then select on the child job’s id to further troubleshoot it. The Output Log contains an error message explaining why the job failed.

You can learn about common job failure errors and possible solutions in the Troubleshooting FAQs.

Scripted Installations

Note

Use the old Houdini installer to do scripted HQueue installations.

The Houdini Launcher currently does not support scripted HQueue installations.

You can perform the HQueue installation from the command-line allowing for scripted installations. This is beneficial if you want to automate the installation and configuration of an HQueue farm.

Upgrading HQueue

Upgrading an HQueue farm involves

  1. Upgrading the HQueue server

  2. Upgrading the HQueue clients

To...Do this

Upgrade the HQueue server

  1. In the Houdini Launcher, select HQueue in the product list.

  2. Select the for HQueue Server and choose Modify.

  3. Select a newer HQueue version in the HQueue Server Modification Dialog.

  4. Select Modify to continue with the upgrade.

  5. When the upgrade completes, the server files are updated but the server configuration files and server database are preserved.

Upgrade the HQueue clients

On each client machine:

  1. Open the Houdini launcher and select HQueue in the product list.

  2. Select the for HQueue Client and choose Modify.

  3. Select a newer HQueue version in the HQueue Client Modification Dialog.

  4. Select Modify to continue with the upgrade.

  5. When the upgrade completes, the client files are updated but the client configuration files and client database are preserved.

Upgrading an HQueue farm previously installed by the old Houdini installer

Use the Houdini Launcher (recommended) or the old Houdini installer to upgrading an existing HQueue farm that was previously installed by the old Houdini installer.

Houdini Launcher
Old Houdini Installer
  • To upgrade the HQueue server, proceed with the regular server installation process in the Houdini Launcher. The Launcher will upgrade the server files but preserve the existing server database and configuration files.

  • To upgrade an HQueue client, proceed with the regular client installation process in the Houdini Launcher. Ensure that the Client User field is set to the same user account that currently runs the HQueue client. The Launcher will upgrade the client files but preserve the existing client configuration file.

  • To upgrade the HQueue server, proceed with the server installation process in the old Houdini installer. Depending on the operating system, the installer may prompt you to install or upgrade the existing HQueue server installation. Choosing to upgrade overwrites the server files but preserves the existing server database and configuration files. Choosing to install overwrites the server files, database, and configuration files. If the installer does not prompt you to install or upgrade, the installer performs an upgrade.

  • Once the HQueue server upgrades to a newer version, the HQueue clients automatically upgrade by pulling the new client code from the HQueue server. You may see Upgrade <machine> jobs appear in the HQueue website that indicate the clients auto-upgrading themselves. If auto-upgrading fails or does not occur, then you can manually request to upgrade a client from the clients web page on the HQueue website (see Managing clients), or install the new client version using the old Houdini installer. Installing via the old Houdini installer will upgrade and preserve the client’s existing configuration.

Upgrading an old HQueue Python 2 farm

Note

Use the old Houdini installer to upgrade an old HQueue Python 2 farm.

The Houdini Launcher does not support upgrading an HQueue Python 2 farm.

Use the old Houdini installer to upgrade the HQueue Server. The installer detects the old HQueue Python 2 server and backs up the old server before proceeding with a new installation/upgrade.

The old server is backed up to:

Linux

/opt/hqueue.backup.YYYY-MM-DD_HHMMSS

Mac

/Library/HQueueServer.YYYY-MM-DD_HHMMSS.bak

Windows

C:/HQueueServer_bk.MM-DD-YYYY.HHMMSS

The installer also converts the old server database and configuration files to the new formats expected by the new HQueue server.

After the installation and the server has been upgraded, the old Python 2 HQueue clients can be upgraded. You can only upgrade old Python 2 clients using the old Houdini installer. Upgrading Python 2 clients from the Houdini Launcher or the HQueue website is not possible.

The installer detects the old HQueue Python 2 client and backs up the old client before proceeding with a new installation/upgrade.

The old client backs up to

Linux

/home/myUser/hqclient.backup.YYYY-MM-DD_HHMMSS

Mac

/Library/HQueueClient.YYYY-MM-DD_HHMMSS.bak

Windows

C:/HQueueClient_bk.MM-DD-YYYY.HHMMSS

The installer also converts the old configuration files to the new formats expected by the new HQueue client.

After the client installation, your new Python 3 HQueue farm is ready for use. It is up to you to remove the backup server and client folders if desired.

Optional: hquser account

On client machines you can optionally set up an hquser user and hqgroup user group to run HQueue as. By convention the user and group ID are both 601, and the user’s password is hqpass, but you can change these.

Linux

Your Linux distribution may come with a user interface for creating user accounts, otherwise use the adduser command.

Mac

You can create users and groups in the Users and Groups panel of System Preferences.

Having consistent user accounts across clients has benefits:

  • Do not have to keep track of different user accounts on different machines when you need to log into a particular client.

  • A single consistent user owns all files created by the client software.

Once you have created the user and group, you should start the HQueue server/client as hquser.

HQueue

Getting started

  • About HQueue

    HQueue is a general-purpose job scheduling system. You can use it to distribute renders, simulations, and other work to remote clients.

  • Installation

    How to set up a basic HQueue farm.

  • Configuration

    How to set configuration options for the HQueue server and clients.

  • How to submit jobs

    How to put work on the farm.

Managing the farm

  • Managing jobs

    How to view and manage jobs on the farm.

  • Managing clients

    How to use the web interface or local logins to add, remove, restart, and manage client machines.

  • Managing client groups

    How to use the web interface or local logins to create and manage groups of client machines.

  • Network Folders

    How to use the Network Folder management page.

  • Resources

    You can specify what resources (such as licenses) are available on each client, so jobs can be scheduled on clients where they can run.

  • Notes

    Each client and job can have informational notes attached.

Next steps

  • Logging

    HQueue stores separate logs for server errors and scheduling events, and each client also generates a log.

  • Uninstalling

    How to uninstall the HQueue server or client software.

  • FAQs

    Answers to frequently asked questions.

Guru level