Hello!

before Houdini 17 this link showed how to use scipy with Houdini even for windows http://wordpress.discretization.de/houdini/home/advanced-2/installing-and-using-scipy-in-houdini/ [wordpress.discretization.de]

However, this trick does not work anymore. I really need it for my work.

Overwriting too many files using the anaconda distribution gets Houdini stuck at startup.

Copying only the scipy and/or numpy packages into C:\Program Files\Side Effects Software\Houdini 17.0.352\python27\lib\site-packages results in errors at the node.
Error
Python error: Traceback (most recent call last):
File “”, line 3, in
File “C:\PROGRA~1\SIDEEF~1\HOUDIN~1.352\python27\lib\site-packages\scipy\__init__.py”, line 119, in
from scipy._lib._ccallback import LowLevelCallable
File “C:\PROGRA~1\SIDEEF~1\HOUDIN~1.352\python27\lib\site-packages\scipy\_lib\_ccallback.py”, line 1, in
from . import _ccallback_c
ImportError: DLL load failed: The specified module could not be found.

When overwriting the C:\Program Files\Side Effects Software\Houdini 17.0.352\python27\dlls with the anaconda distribution, _ctypes.pyd and _socket.pyd seem to cause errors at startup.

I hope that someone here has more knowledge on setting up python libraries and can help out.

Has anyone managed to use scipy in Houdini and can please share how to do that here?

I will update the above website with that information.

thanks!

also ask

That error looks like you will probably have to build scipy with Visual Studio 17 (C++14)
Houdini 17.0 has already numpy 1.12.1

Yes you might be right and I wish I could try it out.

According to this site https://docs.scipy.org/doc/scipy/reference/building/windows.html [docs.scipy.org] I need to go through several steps in order to try it. I have not worked with Visual Studio yet. Has anyone tried this successfully?

I've spent a few weeks on this and asked more clever people for guidance. After setting up mingwpy and going through various compilation guides I gave up and told Sidefx to drop the rfe I've posted for including it to Houdini as I understood the scale of issue.

kahuna031
I gave up

Me too. Scipy on Houdini 17 also did not work on my opensuse but I did not experiment enough why.

I just made the rfe too.

What are you after from scipy? I'm thinking a.more realistic expectation would be for sidefx to support specific features rather than the whole package.

kahuna031
What are you after from scipy? I'm thinking a.more realistic expectation would be for sidefx to support specific features rather than the whole package.

We do mathematics and computer graphics research and need the linear algebra packages because of its sparse matrix solvers. Python is the goto language for many algorithms we need and it would be great to stick to it.(examples [page.math.tu-berlin.de])

I was also after the sparse solvers but ended up modifying a git project that was wrapping libigl that in turn is a Eigen wrapper where you got sparse solvers:
https://github.com/bhenriksson031/EdgyEggs [github.com]

Any news on this?
It's actually of major importance for us as well.

No sorry still no news. We just keep using 16.5 and hope for change.

I am trying this again with Houdini 17.5

When installing a version of scipy, I copy over the folders from anaconda2

scipy
scipy-1.1.0-py2.7.egg-info

into C:\Program Files\Side Effects Software\Houdini 17.5.173\python27\lib

then get this error when trying to import scipy in a python node e.g. from scipy.sparse import csr_matrix

Python error: Traceback (most recent call last):
File “”, line 1, in
File “C:\PROGRA~1\SIDEEF~1\HOUDIN~1.173\python27\lib\site-packages\scipy\__init__.py”, line 77, in
from . import _distributor_init
ImportError: cannot import name _distributor_init

according to this source: https://stackoverflow.com/questions/48423782/importerror-cannot-import-name-distributor-init [stackoverflow.com]
I can comment out the import _distributor_init and in the __init__.py so after that I get this message:

Error
Python error: Traceback (most recent call last):
File “”, line 1, in
File “C:\PROGRA~1\SIDEEF~1\HOUDIN~1.173\python27\lib\site-packages\scipy\__init__.py”, line 119, in
from scipy._lib._ccallback import LowLevelCallable
File “C:\PROGRA~1\SIDEEF~1\HOUDIN~1.173\python27\lib\site-packages\scipy\_lib\_ccallback.py”, line 1, in
from . import _ccallback_c
ImportError: DLL load failed: The specified module could not be found.

I think this error goes back to the _ctypes.pyd in C:\Program Files\Side Effects Software\Houdini 17.5.173\python27\dlls
placing the_ctypes.pyd into that folder makes Houdini not start up anymore.

What now? Can anyone help out? Especially if you have windows, please write us if anything worked for you.

thanks in advance.

As far as I know the python distribution that comes with Houdini on Windows was compiled with Visual Studio 2017. You can read this of from the first line in the python shell

Python 2.7.15 (default, Jan 13 2019, 12:40:45) on win32


So whatever you want to add to Houdini on Windows needs to be compiled with the same version ( MSC v.1916 ). Unfortunately, I wasn't able to find any scipy binaries compiled with this particular compiler for python 2.7. So I guess if you don't want to compile it yourself, and sesi cannot do that for you, it might challenging to get it to work.


Depending on your usecase there might be certain workarounds possible. But all of them are not ideal.
Please let me know if you find any solution working for you.

The same compilers is simply not available on windows and it would be a massive job for sidefx supporting a workaround. Reading the history around the project behind the old win-scipy has convinced me of this.

As I mentioned before my suggestion would be to find the specific feature you want from scipy and implement this. Think that would also be a more reasonable rfe for sidefx.

siavash82ir
What are you after from scipy?
I'm thinking a.more realistic expectation would be for sidefx to support specific features rather than the whole package.

We need the linear algebra packages together with the sparse matrix handlers and solvers. That would be a ton of work to re-implement considering that this usage of scipy works well in pre Houdini 17.0 versions.

python is full of packages anyway and many of them are supported by Houdini. Before we managed to insert scipy through the back door but now this is not working anymore. I was thinking that the entire issue could be solved if SideFx includes this one more package in its installation.

Fingers crossed sesi will sort it out soon!

I'm starting a project like this myself and asking the same questions.

Realistically (and very sadly), I think you'll have more luck doing your scipy work in a modern python like python3.7, and starting hython as a subprocess and communicating with it via sockets: either make it a web service, or use execnet (https://pypi.org/project/execnet/) or something similar. IPC always makes everything harder and slower, but trying to write and support anything significant in python2 at this point is asking for trouble.

Hi everyone,

I have spent some time today trying to see if the new Houdini 18 has changed any of this. I was not successful in trying to manually place the scipy/numpy packages.

Does anyone have any new ideas?

Does anyone maybe know how to locally, inside a python script, load numpy/scipy from an entirely different location?

cheers

Marcel

How do we request scipy being shipped with Houdini 18 and future? I am still using 16.5 if I need scipy.

You could do a request for enhancement. I did that when this thread opened.
https://www.sidefx.com/bugs/submit/ [www.sidefx.com]

On another email in January we know this:
SEV 2 (High Priority), This was on 1/27/20

Given that python 2.7 won't have any support anymore I hope that they work over their python in general.
Link [dev.to]