Forums Technical Discussion Python comand line scripting import hou generates double free/corruption error

Python comand line scripting import hou generates double free/corruption error

   4522   4   1
User Avatar
knork
Member
5 posts
Joined: Dec. 2017
Offline
 
Hello Houdini Gurus,

following the guide on command line scripting [www.sidefx.com] has worked very well for me in the past. However, with the recent Houdini 17 update, changing the HFS variable from 16.5 to 17.0 as indicated below, results in the following error:

WARNING: JE Malloc not configured.
         Excessive memory fragmentation and poor performance may result.
         If this is intentional, set
             HOUDINI_DISABLE_JEMALLOCTEST=1
         to remove this warning.
double free or corruption (out)
Aborted

I am running Debian Buster and I have libjemalloc installed. Any suggestions what might cause the error or how I can load Houdini-17 files in an external python script?

Thanks in advance,

Knork

#!/usr/bin/python
def enableHouModule():
    '''Set up the environment so that "import hou" works.'''
    import sys, os

    #import platform
    # Importing hou will load in Houdini's libraries and initialize Houdini.
    # In turn, Houdini will load any HDK extensions written in C++.  These
    # extensions need to link against Houdini's libraries, so we need to
    # make sure that the symbols from Houdini's libraries are visible to
    # other libraries that Houdini loads.  So, we adjust Python's dlopen
    # flags before importing hou.
    if hasattr(sys, "setdlopenflags"):
        old_dlopen_flags = sys.getdlopenflags()
        import DLFCN
        sys.setdlopenflags(old_dlopen_flags | DLFCN.RTLD_GLOBAL)

    try:
        import hou
    except ImportError:
        # Add $HFS/houdini/python2.7libs to sys.path so Python can find the
        # hou module.
        # works for 16.5, fails for 17.0 or 17.0.352 
        HFS="/opt/hfs17.0"
        sys.path.append(HFS + "/houdini/python%d.%dlibs" % sys.version_info[:2])
        import hou
    finally:
        if hasattr(sys, "setdlopenflags"):
            sys.setdlopenflags(old_dlopen_flags)

enableHouModule()
 Report
User Avatar
Aeoll
Member
8 posts
Joined: Dec. 2014
Offline
 
I'm also seeing this error when importing hou into an external python script (Linux Mint).
Nick Taylor
http://nicholas-taylor.com/ [nicholas-taylor.com]
 Report
User Avatar
Stalkerx777
Member
183 posts
Joined: Nov. 2008
Offline
 
Starting from version 17 Houdini doesn't use systems Python anymore. I'd imagine that trying to use Houdini's libraries from the external interpreter could lead to all sorts of issues. Unless you absolutely must using system's Python for some reason, stick to Hython instead.
Aleksei Rusev
Sr. Graphics Tools Engineer @ Nvidia
 Report
User Avatar
gemof
Member
9 posts
Joined: Sept. 2019
Offline
 
I'm seeing this as well on Houdini 19 / Ubuntu 20.04.

In my case I have to user an external interpreter (Py 3.7.12 from deadsnake)since gRPC won't import into Hython with ctype errors.
 Report
User Avatar
gemof
Member
9 posts
Joined: Sept. 2019
Offline
 
OK actually just solved this. Follow this guide to setup jemalloc: https://zapier.com/engineering/celery-python-jemalloc/ [zapier.com]

Then if you run external python as suggested importing Hou module won't error about jemalloc. As of writing this latest jemalloc is 5.2.1 so worth getting latest and changing bash commands appropriately: 

# installing jemalloc
wget https://github.com/jemalloc/jemalloc/releases/download/5.1.0/jemalloc-5.1.0.tar.bz2
tar xvjf jemalloc-5.1.0.tar.bz2
cd jemalloc-5.1.0
./configure
make
sudo make install

# running your app
LD_PRELOAD=/usr/local/lib/libjemalloc.so python your_app.py

FYI this is the doc showing how to load hou in external py interpreter: https://www.sidefx.com/docs/houdini/hom/commandline.html [www.sidefx.com]

However DLFCN was deprecated in py 3.6, so just change:
import DLFCN
sys.setdlopenflags(old_dlopen_flags | DLFCN.RTLD_GLOBAL)

to:
import ctypes
sys.setdlopenflags(old_dlopen_flags | ctypes.RTLD_GLOBAL)
 Report
  • Quick Links