Paul Golter

paulgolter

About Me

EXPERTISE
Technical Director
INDUSTRY
Film/TV

Connect

LOCATION
Stuttgart, Germany
WEBSITE
http://www.paulgolter.com/

Houdini Skills

Availability

Not Specified

Recent Forum Posts

The proper way of doing asyncio in Houdini 20's python node? June 27, 2024, 9:08 a.m.

Hey all,

so I found a way to make it work both for houdini 19.5 and Houdini 20 and above.

You can control this with the CREATE_THREAD variable. In 19.5 the event loop needs to run in a separate thread otherwise it's blocking the UI.

The scripts tries to include all sorts of asyncio stuff, like endless coroutines, async generators, executing a last coroutine before houdini shuts down as well as gracefully ending the loop.

Save this in a pythonrc.py file and include it in HOUDINI_PATH so the loop gets started when houdini boots up.

Note: sheduling a coroutine in the atexit event leads to seg fault in houdini 20 and above.

import threading
import asyncio
import atexit
import logging

logging.basicConfig(level=logging.DEBUG)
logger = logging.getLogger("hou-asyncio")
logger.setLevel(logging.DEBUG)

print("Hello from pythonrc")

CREATE_THREAD = False

def get_event_loop() -> asyncio.AbstractEventLoop:
    try:
        loop = asyncio.get_running_loop()
        logger.info("Take existing event loop: %s", loop)
    except RuntimeError:
        loop = asyncio.new_event_loop()
        logger.info("Created new event loop: %s", loop)
    return loop


async def async_start_heartbeat():
    while True:
        logger.info("Heartbeat")
        await asyncio.sleep(1.5)


async def async_generator():
    number = 0
    while True:
        yield number
        number += 1
        await asyncio.sleep(1)


async def async_use_async_generator():
    async for number in async_generator():
        logger.info("Received number: %i", number)


async def async_cancel_all_tasks(loop: asyncio.AbstractEventLoop):
    tasks = [
        t
        for t in asyncio.all_tasks(loop=loop)
        if t is not asyncio.current_task(loop=loop)
    ]
    for task in tasks:
        task.cancel()
    await asyncio.gather(*tasks, return_exceptions=True)


async def async_shutdown_tasks(loop: asyncio.AbstractEventLoop):
    logger.info("Cancelling all tasks...")
    await async_cancel_all_tasks(loop)

    logger.info("Shutting down async generators...")
    await loop.shutdown_asyncgens()


async def async_last_coroutine():
    logger.info("Houdini about to exit, executing last coroutine...")
    for i in range(3):
        logger.info("Last couroutine done in: %i", 3 - i)
        await asyncio.sleep(1)
    logger.info("Last coroutine done")


def on_exit(loop: asyncio.AbstractEventLoop, thread: threading.Thread = None):
    logger.info("Detected houdini exit event")

    future = asyncio.run_coroutine_threadsafe(async_last_coroutine(), loop)
    future.result()

    future = asyncio.run_coroutine_threadsafe(async_shutdown_tasks(loop), loop)
    future.result()

    logger.info("Stopping event loop...")
    loop.call_soon_threadsafe(loop.stop)

    logger.info("Closing event loop...")
    loop.call_soon_threadsafe(loop.close)

    if thread:
        thread.join()
        logger.info("Joined thread: %s", thread.native_id)


def _start_loop_target(loop: asyncio.AbstractEventLoop):
    asyncio.set_event_loop(loop)
    loop.run_forever()

def main():
    loop = get_event_loop()
    asyncio.set_event_loop(loop)

    # No need to start the event loop in Houdini?
    thread = None
    if not loop.is_running():
        if CREATE_THREAD:
            thread = threading.Thread(target=_start_loop_target, args=(loop,))
            thread.start()
            logger.info("Starting event loop: %s in thread: %s", loop, thread.native_id)
        else:
            logger.info("Starting event loop: %s", loop)
            loop.run_forever()

    logger.info("Scheduling main coroutines...")
    asyncio.run_coroutine_threadsafe(async_start_heartbeat(), loop)
    asyncio.run_coroutine_threadsafe(async_use_async_generator(), loop)

    atexit.register(on_exit, loop, thread)
    logger.info("Registered atexit callback")

if __name__ == "__main__":
    main()

The proper way of doing asyncio in Houdini 20's python node? June 19, 2024, 9:54 a.m.

Hey @iiif

Just found something.

Apparently Houdini 20 ships with their own implementation of the asyncio event loop.

In Houdini 19.5 this feature could be enabled by setting the env variable: 

HOUDINI_ASYNCIO=1

In Houdini 20 this seems to be enabled by default but can actually be turned off by setting the variable to 0.
After this my asyncio code worked again.

Please note that I don't know if this is a good idea...

The asyncio patch in the changelog for reference:
https://www.sidefx.com/changelog/?journal=&categories=&body=asyncio&version=&build_min=&build_max=&show_versions=on&show_compatibility=on&items_per_page= [www.sidefx.com]

The env variable:
https://www.sidefx.com/docs/houdini/ref/env.html [www.sidefx.com]

Apparently the nvidia omniverse connector also had issues with H20 and asyncio, but they managed to fix it, maybe worth looking in to what they did:
https://forums.developer.nvidia.com/t/houdini-20-and-connector-200-0-0-issue-with-node-parameter-menu/283025/4 [forums.developer.nvidia.com]

The proper way of doing asyncio in Houdini 20's python node? June 19, 2024, 9:07 a.m.

Hello

I would be interested in that too! I have issues running the same asyncio code (that worked in Houdini 19.5) in Houdini 20.

In my case i was running the asyncio loop inside of a thread.

But in H20 this does not seem to be allowed anyomore? 


Traceback (most recent call last):
  File "C:\PROGRA~1\SIDEEF~1\HOUDIN~1.724\python310\lib\threading.py", line 1016, in _bootstrap_inner
15:00:26 DEBUG:hatchet.base: Data: Start timers
    self.run()
  File "my_script.py", line 170, in run
    self._loop.create_task(self._async_start_heartbeat())
  File "C:\PROGRA~1/SIDEEF~1/HOUDIN~1.724/houdini/python3.10libs\haio.py", line 2165, in create_task
    check_thread()
  File "C:\PROGRA~1/SIDEEF~1/HOUDIN~1.724/houdini/python3.10libs\haio.py", line 112, in check_thread
    raise RuntimeError("Current thread is not the main thread")
RuntimeError: Current thread is not the main thread
VIEW ALL POSTS