'HFS' environment variable not set?

Forums Houdini Indie and Apprentice 'HFS' environment variable not set?

13243 22 2


chris wells: Member; 57 posts; Joined: Jan. 2015; Offline

Dec. 19, 2016 3:38 p.m.

so I'm trying to setup my python for outside Houdini communication. and found this page.

http://www.sidefx.com/docs/houdini15.0/hom/commandline [sidefx.com]

great, makes sense, However, digging into it. the ‘HFS’ environment variable doesn't exist. I would have thought this was set when houdini was installed. I could set it myself, but that would mean everytime I update houdini, my python would not be updated?

is this typically set when houdini is installed? and if mine isn't, what can I do?

I tried running the installer as admin, and that didn't help.

thanks
Chris


goldfarb: Staff; 3464 posts; Joined: July 2005; Offline

Dec. 19, 2016 3:58 p.m.

it's right there on the page:
“Note
If you do not run source houdini_setup, you must set the HFS environment variable manually. Also, on Windows, you must append the value of $HFS/bin to the PATH environment variable.”
and yes, you'll have to set it up when you update Houdini (I have 56 builds of Houdini currently installed…yikes I should clean this up), so I have a script that I run that will set up the version/build of Houdini that I want to run.

Michael Goldfarb | www.odforce.net
Training Lead
SideFX
www.sidefx.com


chris wells: Member; 57 posts; Joined: Jan. 2015; Offline

Dec. 19, 2016 6:06 p.m.

okay, sorry for the silly-ness, I thought that must have been talking about the install, there are many aspects of houdini that I'm still just figuring out

looking into source houdini_setup, That seems to be something that sets the commandline options, but again, the how is illusive. Maybe I need to do a feature request, have a checkbox option on the install, “setup Python” Fusion actually just does this for you. It's pretty awesome. want to talk to fusion or generation from the outside with python, go into your python and type “import BlackmagicFusion”, that's it. on install it does it all for you.

So currently I'm trying to figure out how to modify that script so I can get this working.

def enableHouModule():
    '''Set up the environment so that "import hou" works.'''
    import sys, os
    # Importing hou will load in Houdini's libraries and initialize Houdini.
    # In turn, Houdini will load any HDK extensions written in C++.  These
    # extensions need to link against Houdini's libraries, so we need to
    # make sure that the symbols from Houdini's libraries are visible to
    # other libraries that Houdini loads.  So, we adjust Python's dlopen
    # flags before importing hou.
    if hasattr(sys, "setdlopenflags"):
        old_dlopen_flags = sys.getdlopenflags()
        import DLFCN
        sys.setdlopenflags(old_dlopen_flags | DLFCN.RTLD_GLOBAL)
    try:
        import hou
    except ImportError:
        # Add $HFS/houdini/python2.7libs to sys.path so Python can find the
        # hou module.
        sys.path.append(r'C:\Program Files\Side Effects Software\Houdini 15.5.673\bin')
        sys.path.append(r'C:\Program Files\Side Effects Software\Houdini 15.5.673\houdini\python2.7libs')
        import hou
    finally:
        if hasattr(sys, "setdlopenflags"):
            sys.setdlopenflags(old_dlopen_flags)
enableHouModule()
import hou

However I get an error, DLL load failed. I thought appending the bin would add all those dll's, So again I'm stumped at this. Any ideas?

thanks
Chris

Edited by chris wells - Dec. 19, 2016 18:07:52


rvinluan: Staff; 1283 posts; Joined: July 2005; Offline

Dec. 20, 2016 9:29 a.m.

Hi Chris,

What does the DLL load failed message say exactly?

It might be error'ing out because it can't find any of the Houdini libraries. Can you confirm that you appended $HFS/bin to the PATH environment variable before starting Python?

For your setup, I would expect to set PATH like so:


    set PATH=%PATH%;C:\Program Files\Side Effects Software\Houdini 15.5.673\bin

Cheers,
Rob


chris wells: Member; 57 posts; Joined: Jan. 2015; Offline

Dec. 20, 2016 5:13 p.m.

the error is

Traceback (most recent call last):
  File "Z:\Z_transfer\chris\scripts\HoudiniPytonSetup.py", line 28, in <module>
    enableHouModule()
  File "Z:\Z_transfer\chris\scripts\HoudiniPytonSetup.py", line 23, in enableHouModule
    import hou
  File "C:\Program Files\Side Effects Software\Houdini 15.5.673\houdini\python2.7libs\hou.py", line 19, in <module>
    import _hou
ImportError: DLL load failed: The specified module could not be found.

I thought I had the path set, But I must not have? the import hou starts, but inside the hou, is the import _hou which apparently calls some dll's


rvinluan: Staff; 1283 posts; Joined: July 2005; Offline

Dec. 21, 2016 9:54 a.m.

chris wells
the error is

ImportError: DLL load failed: The specified module could not be found.

Ah. That indicates that Python cannot locate one of _hou's library dependencies. Unfortunately the Windows OS never tells us which library that is but I think it's safe to assume its one of the Houdini libraries from Houdini's bin directory.

So I would double-check that C:\Program Files\Side Effects Software\Houdini 15.5.673\bin is added to the PATH environment variable before Python starts up.

And if you are still running into issues, then can you post your PATH value here?


chris wells: Member; 57 posts; Joined: Jan. 2015; Offline

Dec. 21, 2016 3:31 p.m.

okay, there is something I am just not getting. So let me go over the steps I have done so you can tell me, where I'm am being silly

I have run

set PATH=%PATH%;C:\Program Files\Side Effects Software\Houdini 15.5.673\bin

in the command prompt. and it still doesn't work.

so for a test I did.

import sys
for f in sys.path:
….print f

it's not on the list. so I did.
sys.path.append(r'C:\Program Files\Side Effects Software\Houdini 15.5.673\bin')
sys.path.append(r'C:\Program Files\Side Effects Software\Houdini 15.5.673\houdini\python2.7libs')

great when I print f in sys.path it's on the list. but when I then

import hou

it recognizes hou but gives me the same error about DLL load failed:

and in addition if I close IDLE and reopen they are off the list. I apologize for the not getting this, I just haven't had to do this before, so am lost.


rvinluan: Staff; 1283 posts; Joined: July 2005; Offline

Dec. 21, 2016 5:51 p.m.

Hi Chris,

No worries. I think there is some confusion though between PATH and sys.path so hopefully I can shed some light.

The PATH environment variable is for locating executables and libraries (i.e. .exe and .dll files respectively) on Windows.

sys.path on the other hand is for locating Python modules (i.e. .py or .pyd files).

So the two are separate and handle different things which is why when you add a path to PATH it does not show up in sys.path.

Now we add the ‘python2.7libs’ path to sys.path so that Python can locate the hou module (and also its related _hou module) and import it.

But the hou module is linked to several Houdini libraries that are located in %HFS%/bin which is why we have to additionally add the ‘bin’ path to PATH.

Now everything that you are doing looks correct to me so I'm not sure why you are still seeing those DLL errors.

I'll have access to a Windows box tomorrow so let me see if I can get this working myself with the latest 15.5 build and I'll get back to you on this.

Cheers,
Rob


pezetko: Member; 392 posts; Joined: Nov. 2008; Offline

Dec. 22, 2016 10:25 a.m.

To see difference between PATH and sys.path and loading hou module:

1. Create this bat file (Don't forget to change version number):

@rem add houdini path to the PATH variable
@set HFS=C:\Program Files\Side Effects Software\Houdini 15.5.632
@set PATH=%PATH%;%HFS%\bin
@rem set current working directory
pushd %~dp0
@rem run script in current working directory
@python .\HoudiniPythonSetup.py
pause

Then in same directory create HoudiniPythonSetup.py

#!/usr/bin/python
def enableHouModule():
    '''Set up the environment so that "import hou" works.'''
    import sys
    import os
    # Importing hou will load in Houdini's libraries and initialize Houdini.
    # In turn, Houdini will load any HDK extensions written in C++.  These
    # extensions need to link against Houdini's libraries, so we need to
    # make sure that the symbols from Houdini's libraries are visible to
    # other libraries that Houdini loads.  So, we adjust Python's dlopen
    # flags before importing hou.
    if hasattr(sys, "setdlopenflags"):
        old_dlopen_flags = sys.getdlopenflags()
        import DLFCN
        sys.setdlopenflags(old_dlopen_flags | DLFCN.RTLD_GLOBAL)
    try:
        import hou
    except ImportError:
        # Add $HFS/houdini/python2.7libs to sys.path so Python can find the
        # hou module.
        sys.path.append(os.environ['HFS'] + "/houdini/python%d.%dlibs" % sys.version_info[:2])
        # import hou
    finally:
        if hasattr(sys, "setdlopenflags"):
            sys.setdlopenflags(old_dlopen_flags)
# TEST THIS: #
def main():
    import sys
    import os
    print("Python Version: {}\n".format(sys.version))
    print("PATH variable: {}\n".format(os.environ['PATH']))
    print("sys.path variable: {}\n".format(sys.path))
    try:
        enableHouModule()   # without including import hou
        import hou
        print("Houdini version: {} module imported".format(hou.applicationVersionString()))
    except:
        print("Error: cannot load houdini module")
if __name__ == '__main__':
    main()

Then run bat file and check its output.


rvinluan: Staff; 1283 posts; Joined: July 2005; Offline

Dec. 22, 2016 10:43 a.m.

Hi Chris,

I gave this a try on my Windows box and was able to import the hou module into a standalone Python session.

Here's what I did (in case it helps):
- Installed Houdini 15.5.673.
- Installed Python 2.7.9 into the default C:/Python27 location. Note that I actually installed this Python version a while ago onto my system which is why its a relatively older Python version.
- Opened a Windows command prompt and ran these commands to launch Python:

set PATH=C:\Program Files\Side Effects Software\Houdini 15.5.673\bin;%PATH%
cd C:\Python27
python.exe

- Then in Python, I ran these commands to import the hou module:

>>> import sys
>>> sys.path.append("C:/Program Files/Side Effects Software/Houdini 15.5.673/houdini/python2.7libs")
>>> import hou

And that's it.

So I'm not sure why you are hitting the DLL error. It must have something to do with the PATH environment variable. Maybe double-check that PATH still has the Houdini bin path in it when python starts up? Maybe something is overwriting PATH later on after you've made the change to it?

In python, you can inspect PATH with this bit of code:

import os
print os.environ["PATH"]

Also, try prepending the Houdini bin path to PATH instead of appending. Maybe appending the path is causing the hou module to pick up Houdini library dependencies from another location instead of from bin.

Cheers,
Rob


chris wells: Member; 57 posts; Joined: Jan. 2015; Offline

Dec. 22, 2016 11:51 a.m.

Thanks for the help everyone! That is working! That set PATH worked, and now it's working like I would expect.

I somehow was just not setting the PATH correctly before.

thanks again!
Chris


axelsp: Member; 100 posts; Joined: Sept. 2006; Offline

Feb. 28, 2017 8:10 a.m.

Hello Gang,
Im having the same error. I followed the thread, and helped > a lot < but no luck figuring out what's going on.
Python version I have seems to be 2.7.12.
PATH and sys.path seem to be correct. But I keep bumping on that Import Error on line 19.
Any clues? Im on windows 8.1

c:\Python27>python
Python 2.7.12 (v2.7.12:xxxxxxxxxx, Jun 27 2016, 15:24:40) [MSC v.1500 64 bit (AMD64)] on win32Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> sys.path.append("C:\\Program Files\\Side Effects Software\\Houdini 15.5.523\\houdini\\python2.7libs")
>>> import hou
Traceback (most recent call last):  File "<stdin>", line 1, in <module>  File "C:\Program Files\Side Effects Software\Houdini 15.5.523\houdini\python2.7libs\hou.py", line 19, in <module>    import _houImportError: DLL load failed: The specified module could not be found.
>>> sys.path['', 'C:\\Windows\\SYSTEM32\\python27.zip', 'c:\\Python27\\DLLs', 'c:\\Python27\\lib', 'c:\\Python27\\lib\\plat-win', 'c:\\Python27\\lib\\lib-tk', 'c:\\Python27', 'c:\\Python27\\lib\\site-packages', 'C:\\Program Files\\Side Effects Software\\Houdini 15.5.523\\houdini\\python2.7libs']
>>> import os
>>> os.environ["PATH"]
'"C:\\Program Files\\Side Effects Software\\Houdini 15.5.523\\bin";C:\\Program Files (x86)\\Intel\\iCLS Client\\;C:\\Program Files\\Intel\\iCLS Client\\;C:\\Windows\\system32;C:\\Windows;C:\\Windows\\System32\\Wbem;C:\\Windows\\System32\\WindowsPowerShell\\v1.0\\;C:\\Program Files (x86)\\QuickTime\\QTSystem\\;C:\\Program Files (x86)\\Skype\\Phone\\;C:\\Program Files\\Intel\\Intel(R) Management Engine Components\\DAL;C:\\Program Files\\Intel\\Intel(R) Management Engine Components\\IPT;C:\\Program Files (x86)\\Intel\\Intel(R) Management Engine Components\\DAL;C:\\Program Files (x86)\\Intel\\Intel(R) Management Engine Components\\IPT;C:\\Program Files (x86)\\NVIDIA Corporation\\PhysX\\Common;C:\\Python27'
>>> sys.path['', 'C:\\Windows\\SYSTEM32\\python27.zip', 'c:\\Python27\\DLLs', 'c:\\Python27\\lib', 'c:\\Python27\\lib\\plat-win', 'c:\\Python27\\lib\\lib-tk', 'c:\\Python27', 'c:\\Python27\\lib\\site-packages', 'C:\\Program Files\\Side Effects Software\\Houdini 15.5.523\\houdini\\python2.7libs']
>>> import hou
Traceback (most recent call last):  File "<stdin>", line 1, in <module>  File "C:\Program Files\Side Effects Software\Houdini 15.5.523\houdini\python2.7libs\hou.py", line 19, in <module>    import _houImportError: DLL load failed: The specified module could not be found.

Edited by axelsp - Feb. 28, 2017 08:13:07


rvinluan: Staff; 1283 posts; Joined: July 2005; Offline

Feb. 28, 2017 10:19 a.m.

Hello,

The paths look right.

I think the problem might be the Python interpreter itself. It looks like it is built with an older compiler version so it probably cannot import the compiled _hou module or its library dependencies because they are built with MSVC 2015.

I bet if you launch the Python interpreter shipped with Houdini instead then you will be able to import the hou module.

So try launching “C:\Program Files\Side Effects Software\Houdini 15.523\python27\python.exe” instead and import the hou module there. When you launch that Python you will notice a different compiler version outputted in the header string (i.e. “MSC v.1900 64 bit (AMD64)”).

Cheers,
Rob


axelsp: Member; 100 posts; Joined: Sept. 2006; Offline

March 1, 2017 6:23 a.m.

YES !! EVERYTHING CHECKS! From the Header to the module import! Thanks so much!
What would be the best solution for this tho, should I replace the “C:\Python27” entry on the os Path to point to houdini's? Bit reluctant on that, perhaps other things will start crashing (?)

C:\Program Files\Side Effects Software\Houdini 15.5.523\python27>set PATH=C:\Program Files\Side Effects Software\Houdini 15.5.523\bin;%PATH%
C:\Program Files\Side Effects Software\Houdini 15.5.523\python27>python.exe
Python 2.7.5 (default, Oct 15 2015, 15:22:58) [MSC v.1900 64 bit (AMD64)] on win32Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> sys.path['', 'C:\\Program Files\\Side Effects Software\\Houdini 15.5.523\\python27\\python27.zip', 'C:\\Program Files\\Side Effects Software\\Houdini 15.5.523\\python27\\DLLs', 'C:\\Program Files\\Side Effects Software\\Houdini 15.5.523\\python27\\lib', 'C:\\Program Files\\Side Effects Software\\Houdini 15.5.523\\python27\\lib\\plat-win', 'C:\\Program Files\\Side Effects Software\\Houdini 15.5.523\\python27\\lib\\lib-tk', 'C:\\Program Files\\Side Effects Software\\Houdini 15.5.523\\python27', 'C:\\Program Files\\Side Effects Software\\Houdini 15.5.523\\python27\\lib\\site-packages']
>>> sys.path.append("C:\\Program Files\\Side Effects Software\\Houdini 15.5.523\\houdini\\python2.7libs")
>>> import hou
[Redshift][core] Redshift for Houdini plugin version 2.0.86
[Redshift][core] Plugin compile time HDK version: 15.5.565
[Redshift][core] Houdini host version: 15.5.523
[Redshift][core] Plugin dso/dll and config path: CProgramData/Redshift/Plugins/Houdini/15.5.565/dso
[Redshift][core] Core data path: C:\ProgramData\Redshift
[Redshift][core] Local data path: C:\ProgramData\Redshift
[Redshift][core] Procedurals path: C:\ProgramData\Redshift\Procedurals
[Redshift][core] Preferences file path: C:\ProgramData\Redshift\preferences.xml
[Redshift][core] License path: C:\ProgramData\Redshift
[Robert Vinluan] GENIUS !
>>>

Edited by axelsp - March 1, 2017 06:27:16


rvinluan: Staff; 1283 posts; Joined: July 2005; Offline

March 1, 2017 9:38 a.m.

axelsp
YES !! EVERYTHING CHECKS! From the Header to the module import! Thanks so much!
What would be the best solution for this tho, should I replace the “C:\Python27” entry on the os Path to point to houdini's? Bit reluctant on that, perhaps other things will start crashing (?)

That would be the simplest solution.

I would set PATH like so:

set PATH=C:\Program Files\Side Effects Software\Houdini 15.5.523\bin;C:\Program Files\Side Effects Software\Houdini 15.5.523\python27;%PATH%

I wouldn't worry about other things crashing because of this change. There's not a lot of libraries in the python27 folder that can cause library conflicts.

But switching to Houdini's python can potentially cause any 3rd-party Python modules that you've installed to stop working if those modules are compiled (i.e. are .pyd files) and compiled by a different compiler version. But that's a problem you would eventually hit anyway with or without setting the PATH variable.

Cheers,
Rob


axelsp: Member; 100 posts; Joined: Sept. 2006; Offline

March 2, 2017 8:40 a.m.

Thanks a lot Rob!


axelsp: Member; 100 posts; Joined: Sept. 2006; Offline

March 4, 2017 2:45 a.m.

rvinluan
axelsp
YES !! EVERYTHING CHECKS! From the Header to the module import! Thanks so much!
What would be the best solution for this tho, should I replace the “C:\Python27” entry on the os Path to point to houdini's? Bit reluctant on that, perhaps other things will start crashing (?)

That would be the simplest solution.

I would set PATH like so:
set PATH=C:\Program Files\Side Effects Software\Houdini 15.5.523\bin;C:\Program Files\Side Effects Software\Houdini 15.5.523\python27;%PATH%
I wouldn't worry about other things crashing because of this change. There's not a lot of libraries in the python27 folder that can cause library conflicts.

But switching to Houdini's python can potentially cause any 3rd-party Python modules that you've installed to stop working if those modules are compiled (i.e. are .pyd files) and compiled by a different compiler version. But that's a problem you would eventually hit anyway with or without setting the PATH variable.

Cheers,
Rob

How Could I set the path so Hqueue sees that lib ? It throws that same error even if I set in on the OS env. variable.


axelsp: Member; 100 posts; Joined: Sept. 2006; Offline

March 5, 2017 4:17 a.m.

For the sake of testing on ‘cmd.exe’ I have successfully set up the following:

PATH=C:\Program Files\Side Effects Software\Houdini 15.5.523\bin;C:\Program Files\Side Effects Software\Houdini 15.5.523\python27;%PATH%
PYTHONPATH=C:\Program Files\Side Effects Software\Houdini 15.5.523\houdini\python2.7libs

- Opening Python.exe goes straight for Houdini version.
- ‘import hou’ it just works.

With that set, Hqueue keeps failing with “import hou line 19 error”

Cleared those env vars and tried setting them on the ‘job_environment’ session of the client hqnode.ini, no luck.

Finally tried adding Environment var from the hq_render submit node from the scene with those 2 variables, also doesnt seem to affect it.

Checking the Diagnostic info, the paths seems to be going thru correctly:

HQCOMMANDS:
{
    "pythonCommandsWindows": "(set HFS=C:\\Program Files\\Side Effects Software\\Houdini 15.5.523) && \"!HFS!\\python27\\python2.7.exe\"", 
    "hythonCommandsWindows": "(set HOUDINI_PYTHON_VERSION=2.7) && (set HFS=C:\\Program Files\\Side Effects Software\\Houdini 15.5.523) && (set PATH=C:\\Program Files\\Side Effects Software\\Houdini 15.5.523\\bin;!PATH!) && \"!HFS!\\bin\\hython\" -u", 
    "mantraCommandsWindows": "(set HFS=C:\\Program Files\\Side Effects Software\\Houdini 15.5.523) && \"!HFS!\\python27\\python2.7.exe\" \"!HFS!\\houdini\\scripts\\hqueue\\hq_mantra.py\""
}

PYTHONPATH:
C:\Program Files\Side Effects Software\Houdini 15.5.523\houdini\python2.7libs

HQ_PRESERVE_ENV_VARS:
PYTHONPATH,PATH

PATH:
C:\Program Files\Side Effects Software\Houdini 15.5.523\bin;!PATH!

Job Conditions and Requirements:
================================
hostname any BLACK

Executed Client Job Commands:
=============================
Windows Command:
(set HOUDINI_PYTHON_VERSION=2.7) && (set HFS=C:\Program Files\Side Effects Software\Houdini 15.5.523) && (set PATH=C:\Program Files\Side Effects Software\Houdini 15.5.523\bin;!PATH!) && "!HFS!\bin\hython" -u "!HFS!\houdini\scripts\hqueue\hq_submit_renders.py"

Im restarting HQueue server and the client each time.
Im guessing that its only about setting sys.path with houdini\python2.7libs. That seems to be the only thing missing.

I have a feeling PYTHONPATH doesnt not affect Hqueue. Any ideas on how to go about it ?

Edited by axelsp - March 5, 2017 04:40:20


rvinluan: Staff; 1283 posts; Joined: July 2005; Offline

March 6, 2017 10:12 a.m.

You should be able to set PATH and PYTHONPATH using the Environment parameter in the HQueue Render ROP.

Looking at the diagnostics info it appears that C:\Program Files\Side Effects Software\Houdini 15.5.523\python27 is missing from PATH.

Try adding that path back in and see how that goes.

Cheers,
Rob


Stocks: Member; 6 posts; Joined: Sept. 2016; Offline

April 4, 2021 10:10 a.m.

Hay there everyone, I know this Thread is old but I am trying to get my Hou Module to work within Maya.
I am trying to get Maya to load a houdini hip file and run a render ROP.
If I am understanding correctly, setting this up should do the trick but I am not sure on that ?

I have followed this thread and get the following error after doing the import hou after launching python from inside the command Prompt and setting all the paths.

Traceback (most recent call last):
  File "", line 1, in 
  File "C:/Program Files/Side Effects Software/Houdini 18.5.351/houdini/python2.7libs\hou.py", line 32, in 
    import _hou
ImportError: DLL load failed: %1 is not a valid Win32 application.

I have also tried Manually editing the Environment variables in Windows with no luck.
Any help Would be Greatly Appreciated.

Edited by Stocks - April 4, 2021 10:10:57

Attachments:
Screenshot_54.png (16.9 KB)

Quick Links

                    
                        Search links
                        Show recent posts
                        Show unanswered posts