Hi all,
I'm tryng to install HDF5 library on my python houdini windows PC, but I have many problems… I did it with succesfull on my ImacPro (osx) but in windows (window 10 pro 64 bit) when I try to “import h5py”, python give me many errors.
On my iMac was very easy: I copied the h5py library (installed with “pip” in my system python) in “site-packages” folder of python inside houdini.
What kind of version I've to install (32 bit or 64 bit)?. I've tried both, but no one works.
There is anyone that works with hdf5 file in houdini that teach me what I've to do?
Any suggest?
Thank you.
Paolo
HDF5 Python library
5174 8 3-
- Punkt
- Member
- 10 posts
- Joined: 10月 2018
- Offline
-
- bradcarvey
- Member
- 52 posts
- Joined: 7月 2013
- Offline
-
- silkey
- Member
- 13 posts
- Joined: 9月 2016
- Offline
-
- shadesoforange
- Member
- 201 posts
- Joined: 7月 2015
- Offline
I recently had a similar issue.
The problem is because of the uncommon version of HDF5 Houdini ships with, which is HDF5-1.8.11 [www.sidefx.com]. There are no h5py versions available on package management systems like pip or conda which are built with HDF5-1.8.11.
Additionally this prevents working with TensorFlow within Houdini, since TF also requires h5py.
I've recently logged a RFE for this: Shipping Houdini with a HDF5 version that has a matching & available h5py version [www.sidefx.com]
Feel free to add another RFE and mention RFE#109905 from above if this affects you as well.
The problem is because of the uncommon version of HDF5 Houdini ships with, which is HDF5-1.8.11 [www.sidefx.com]. There are no h5py versions available on package management systems like pip or conda which are built with HDF5-1.8.11.
Additionally this prevents working with TensorFlow within Houdini, since TF also requires h5py.
I've recently logged a RFE for this: Shipping Houdini with a HDF5 version that has a matching & available h5py version [www.sidefx.com]
Feel free to add another RFE and mention RFE#109905 from above if this affects you as well.
#RFE109905
As we now have stable Python 3 builds of Houdini, machine learning within Houdini becomes more and more attractive. The most popular Python machine learning frameworks are Tensorflow and Pytorch.
Both Tensorflow and Pytorch (with torch-geometric) depend on h5py (a HDF5 Python wrapper).
Houdini ships with HDF5-1.8.11. There are no builds of h5py available via any package manager (pip, conda..) that are built upon HDF5-1.8.11, which makes the usage of Tensorflow/Pytorch within Houdini hard since there are no Houdini compatible versions of h5py available. The only remaining option is to build h5py from source yourself, which is very hard (especially on Windows).
My proposal is to ship Houdini with a HDF5 version that is used by some h5py distributions. One of these versions would be HDF5-1.8.18. This would make the implementation of ML processes within Houdini a lot easier/possible.
I do not know what implications this brings for you as a lot of other things that you guys ship are probably built with this exact HDF5 version.
Any feedback would be appreciated on how we could better integrate ML processes within Houdini.
Edited by shadesoforange - 2020年12月16日 13:33:29
Manuel Köster - Senior Technical Artist @Remedy Entertainment
https://www.remedygames.com/ [www.remedygames.com]
http://shadesoforange.de/ [shadesoforange.de]
https://twitter.com/ShadesOfOrange_ [twitter.com]
https://www.remedygames.com/ [www.remedygames.com]
http://shadesoforange.de/ [shadesoforange.de]
https://twitter.com/ShadesOfOrange_ [twitter.com]
-
- silkey
- Member
- 13 posts
- Joined: 9月 2016
- Offline
-
- silkey
- Member
- 13 posts
- Joined: 9月 2016
- Offline
-
- nuclear_polygon
- Member
- 21 posts
- Joined: 11月 2018
- Offline
Hi there! I had figured out some workaround on this issue. now i can import tensorflow inside houdini without any errors. not tried it in work yet though. anyway these are steps I made:
hope this helps
- download h5py [github.com] from source
- build it
#update pip hython -m pip install --upgrade pip sudo apt install python3-pip #set environment export HDF5_INCLUDEDIR=/opt/hfs18.5/toolkit/include/ export HDF5_LIBDIR=/opt/hfs18.5/dsolib/ #unzip h5py-3.6.0 cd ~/Downloads 7z x ./h5py-3.6.0.zip cd ./h5py-3.6.0 #build and install h5py hython setup.py build pip3 install --target=~/.local/lib/python3.7/site-packages -v .
- that's it
hope this helps
-
- ajp_wyss
- Member
- 1 posts
- Joined: 12月 2021
- Offline
Finally figured out how to get h5py installed in Houdini's python on Windows.
As is recommended here: http://wordpress.discretization.de/houdini/home/advanced-2/installing-and-using-scipy-in-houdini/, [wordpress.discretization.de]
you'll want to get pip. However, because Houdini uses a custom version of HDF5, the normal `python3.7.exe -m pip install h5py` will NOT work- you will get "DLL load failed" errors when importing.
0. This process requires Visual Studio C++ desktop development tools, since we are compiling h5py. If you execute the `... pip install --no-binary ...` step without it, the error message will have a link of where to download it. (Probably here [visualstudio.microsoft.com])
1. Run Houdini as Administrator (so it has permissions to edit the Program Files), and open a terminal (in the menu: Windows > Shell).
2. As is suggested here: https://docs.h5py.org/en/stable/build.html#custom-installation, [docs.h5py.org] we need to set environment variables to tell pip where to find Houdini's existing HDF5 .lib and .h files.
On my system, this was achieved by the following commands, but you will have to update your paths based on your installation:
```
set HDF5_LIBDIR=C:/Program Files/Side Effects Software/Houdini 19.0.455/custom/houdini/dsolib/
set HDF5_INCLUDEDIR=C:/Program Files/Side Effects Software/Houdini 19.0.455/toolkit/include/
```
3. Run `hython -m pip install --no-binary=h5py h5py`.
4. Test that it worked with `hython -c "import h5py"` (no error message means success).
5. Don't forget to quit Houdini and re-open it without Admin permissions when you're done!
As is recommended here: http://wordpress.discretization.de/houdini/home/advanced-2/installing-and-using-scipy-in-houdini/, [wordpress.discretization.de]
you'll want to get pip. However, because Houdini uses a custom version of HDF5, the normal `python3.7.exe -m pip install h5py` will NOT work- you will get "DLL load failed" errors when importing.
0. This process requires Visual Studio C++ desktop development tools, since we are compiling h5py. If you execute the `... pip install --no-binary ...` step without it, the error message will have a link of where to download it. (Probably here [visualstudio.microsoft.com])
1. Run Houdini as Administrator (so it has permissions to edit the Program Files), and open a terminal (in the menu: Windows > Shell).
2. As is suggested here: https://docs.h5py.org/en/stable/build.html#custom-installation, [docs.h5py.org] we need to set environment variables to tell pip where to find Houdini's existing HDF5 .lib and .h files.
On my system, this was achieved by the following commands, but you will have to update your paths based on your installation:
```
set HDF5_LIBDIR=C:/Program Files/Side Effects Software/Houdini 19.0.455/custom/houdini/dsolib/
set HDF5_INCLUDEDIR=C:/Program Files/Side Effects Software/Houdini 19.0.455/toolkit/include/
```
3. Run `hython -m pip install --no-binary=h5py h5py`.
4. Test that it worked with `hython -c "import h5py"` (no error message means success).
5. Don't forget to quit Houdini and re-open it without Admin permissions when you're done!
-
- Eche
- Member
- 39 posts
- Joined: 2月 2018
- Online
Adding another resource - here Paul Ambrosiussen talks about installing python libraries in Houdini, specifically SciPy and Tensorflow:
https://youtu.be/gOdRQsAjQWk?t=1302 [youtu.be]
https://youtu.be/gOdRQsAjQWk?t=1302 [youtu.be]
-
- Quick Links