Houdini 20.5 HQueue

Switching the Database Engine

How to change the HQueue farm SQLite database backend to another database engine.

On this page

Overview

The HQueue farm uses SQLite for its database backend. SQLite is lightweight and portable but not scalable. It is unsuitable for high concurrency as it limits writing to only one writer at a time. HQueue provides the option to change the database backend to more performant database engines, MySQL and PostgreSQL.

Note

SQLite is sufficient for most HQueue farms. Switching to MySQL or PostgreSQL is beneficial only for large farms where freezes and slowdowns are noticeable in the HQueue web interface.

Note

Switching the database backend to MySQL or PostgreSQL does not migrate the data from the existing SQLite database. Job, client, group and resource information will be lost, however, the new database will automatically repopulate with client information when the HQueue clients ping the server.

Switching to MySQL

Prerequisites

The HQueue server requires the following

  • MySQL and the database management tool MySQL Workbench on the HQueue server.

  • The mysqlclient Python package.

    This must be installed into the HQueue server installation

    Linux

    In a terminal, run:

    sudo /opt/hqueue/bin/pip3 install mysqlclient

    Mac

    In a terminal, run:

    sudo /Library/HQueueServer/Frameworkds/Python.framework/Versions/X.Y/bin/pip3 install mysqlclient

    Windows

    In a Windows command prompt terminal, run as administrator:

    cd C:/HQueueServer/pythonXY/Scripts
pip3.exe install mysqlclient

    Note

    You may need to install the pkg-config and libmysqlclient-dev packages before installing mysqlclient.

Setup Instructions

  1. Stop the HQueue Server.

    See the FAQs for instructions on how to stop the server.

  2. Access the MySQL database shell.

    In a terminal or Windows command prompt, run:

    mysql -u root -p

  3. Create the HQueue database.

    In the MySQL database shell, run:

    mysql> CREATE DATABASE hqueue;

    The database should be visible in MySQL Workbench.

  4. Create a database user account. (optional)

    Note

    Skip this step if you already have a database user account that you want to connect to the HQueue database with.

    In the MySQL database shell, run:

    mysql> CREATE USER '<username>'@'localhost' IDENTIFIED BY '<password>';

    where <username> and <password> are the new user account’s username and password respectively.

  5. Grant HQueue database access to the database user account.

    In the MySQL database shell, run:

    mysql> GRANT ALL ON hqueue.* TO '<username>'@'localhost';

    where <username> is the database user account.

  6. Configure the HQueue server to connect to the MySQL database.

    Edit the HQueue server configuration file, hqserver.ini in a text editor and add the following to the end of the file:

    [database]
engine = django.db.backends.mysql
name = hqueue
user = <username>
password = <password>

    where <username> and <password> are the user credentials required to connect to the MySQL database.

    See HQueue Configuration for further details on the database configuration options.

  7. Create the HQueue database tables.

    Linux

    In a terminal, run:

    cd /opt/hqueue
sudo ./bin/python3 ./hqserver/manage.py migrate

    Mac

    In a terminal, run:

    cd /Library/HQueueServer
sudo ./Frameworks/Python.framework/Versions/X.Y/bin/python3 ./hqserver/manage.py migrate

    Windows

    In a Windows command prompt terminal, run as administrator:

    cd C:/HQueueServer
pythonXY/bin/pythonX.Y.exe hqserver/manage.py migrate

  8. Restart the HQueue Server.

    See the FAQs for instructions on how to start the server.

Switching to PostgreSQL

Prerequisites

The HQueue server requires the following

  • PostgreSQL and the database management tool pgAdmin.

  • The psycopg Python package.

    This must be installed into the HQueue Server installation:

    Linux

    In a terminal, run:

    sudo /opt/hqueue/bin/pip3 install psycopg2-binary

    Mac

    In a terminal, run:

    sudo /Library/HQueueServer/Frameworkds/Python.framework/Versions/X.Y/bin/pip3 install psycopg2-binary

    Windows

    In a Windows command prompt terminal, run as administrator:

    cd C:/HQueueServer/pythonXY/Scripts
pip3.exe install psycopg2-binary

Setup Instructions

  1. Stop the HQueue Server.

    See the FAQs for instructions on how to stop the server.

  2. Access the PostgreSQL database shell.

    In a terminal or Windows command prompt, run:

    psql -U postgres

    You will be prompted to enter the postgres password that was set during the PostgreSQL installation.

  3. Create the HQueue database.

    In the PostgreSQL database shell, run:

    postgres=# CREATE DATABASE hqueue;

    The database should be visible in pgAdmin.

  4. Create a database user account. (optional)

    Note

    Skip this step if you already have a database user account that you want to connect to the HQueue database with.

    In the PostgreSQL database shell, run:

    postgres=# CREATE USER <username> WITH PASSWORD '<password>';

    where <username> and <password> are the new user account’s username and password respectively.

  5. Grant HQueue database access to the database user account.

    In the PostgreSQL database shell, run:

    postgres=# GRANT CONNECT ON DATABASE hqueue TO <username>;
postgres=# \c hqueue
postgres=# GRANT ALL PRIVILEGES ON ALL TABLES IN SCHEMA public TO <username>;
postgres=# GRANT ALL PRIVILEGES ON ALL SEQUENCES IN SCHEMA PUBLIC TO <username>;

    where <username> is the database user account.

  6. Register a server in pgAdmin.

    Populate the name and host name/address fields accordingly. Then populate the username and password fields with the chosen username and password from the previous step respectively.

  7. Configure the HQueue server to connect to the PostgreSQL database.

    Edit the HQueue server configuration file, hqserver.ini in a text editor and add the following to the end of the file:

    [database]
engine = django.db.backends.postgresql
name = hqueue
user = <username>
password = <password>

    where <username> and <password> are the user credentials required to connect to the PostgreSQL database.

    See HQueue Configuration for further details on the database configuration options.

  8. Create the HQueue database tables.

    Linux

    In a terminal, run:

    cd /opt/hqueue
sudo ./bin/python3 ./hqserver/manage.py migrate

    Mac

    In a terminal, run:

    cd /Library/HQueueServer
sudo ./Frameworks/Python.framework/Versions/X.Y/bin/python3 ./hqserver/manage.py migrate

    Windows

    In a Windows command prompt terminal, run as administrator:

    cd C:/HQueueServer
pythonXY/bin/pythonX.Y.exe hqserver/manage.py migrate

  9. Restart the HQueue Server.

    See the FAQs for instructions on how to start the server.

HQueue

Getting started

  • About HQueue

    HQueue is a general-purpose job scheduling system. You can use it to distribute renders, simulations, and other work to remote clients.

  • Installation

    How to set up a basic HQueue farm.

  • Configuration

    How to set configuration options for the HQueue server and clients.

  • How to submit jobs

    How to put work on the farm.

Managing the farm

  • Managing jobs

    How to view and manage jobs on the farm.

  • Managing clients

    How to use the web interface or local logins to add, remove, restart, and manage client machines.

  • Managing client groups

    How to use the web interface or local logins to create and manage groups of client machines.

  • Network Folders

    How to use the Network Folder management page.

  • Resources

    You can specify what resources (such as licenses) are available on each client, so jobs can be scheduled on clients where they can run.

  • Notes

    Each client and job can have informational notes attached.

Next steps

  • Logging

    HQueue stores separate logs for server errors and scheduling events, and each client also generates a log.

  • Uninstalling

    How to uninstall the HQueue server or client software.

  • FAQs

    Answers to frequently asked questions.

Guru level