[Bf-python] Loading shared library in Blender Python

[Pierrick Koch]

Hello,

I'd like to imort python package from ubuntu distribution into blender.
Their shared object extension is .cpython-34m-x86_64-linux-gnu.so,
while Blender look for .cpython-34m.so (without "-x86_64-linux-gnu").

Do you know any way to override this issue,
or a way to let Blender Python look for both extension ?

Best,

[Sybren A. Stüvel]

Hi Pierrick,

Post by Pierrick Koch
I'd like to imort python package from ubuntu distribution into
blender. Their shared object extension is
.cpython-34m-x86_64-linux-gnu.so, while Blender look for
.cpython-34m.so (without "-x86_64-linux-gnu").

How did you gather this knowledge? It helps a lot to see what you're
actually doing.

Post by Pierrick Koch
Do you know any way to override this issue, or a way to let Blender
Python look for both extension ?

What I did is the following:

- Create a director ~/.local/lib/python3.4/site-packages
- Symlink packages from /usr/lib/python3/dist-packages into that dir.

For example, I have the following in that dir:

scipy -> /usr/lib/python3/dist-packages/scipy/
scipy-0.14.0.egg-info -> /usr/lib/python3/dist-packages/scipy-0.14.0.egg-info
shapely -> /usr/lib/python3/dist-packages/shapely/
Shapely-1.3.0.egg-info -> /usr/lib/python3/dist-packages/Shapely-1.3.0.egg-info

This allows me to import shapely and scipy into Blender 2.75-rc1 just

Post by Pierrick Koch

import shapely
shapely.__file__

'/home/sybren/.local/lib/python3.4/site-packages/shapely/__init__.py'

If you run into any issues, don't hesitate to mail back with a nice
error message ;-)

Best,

[Arnaud Degroote]

Post by Sybren A. Stüvel
Hi Pierrick,

Post by Pierrick Koch
I'd like to imort python package from ubuntu distribution into
blender. Their shared object extension is
.cpython-34m-x86_64-linux-gnu.so, while Blender look for
.cpython-34m.so (without "-x86_64-linux-gnu").

How did you gather this knowledge? It helps a lot to see what you're
actually doing.

To be more precise, what we try is doing

import numpy

which fails with

ImportError: cannot import name 'multiarray'

(whole discussion is here : https://github.com/morse-simulator/morse/pull/630)

It basically fails because python from Ubuntu is built with multiarch
option and so the libraries have extension cpython-34m-x86_64-linux-gnu.so.
It is not the case of the python embedded in binary blender.

Of course, it is always possible, in this case, to use the numpy copy
from blender (but it is not a generic solution), or to hack the
filesystem (but it is cumbersome for users).

Best regards,

[Sybren A. Stüvel]

Post by Arnaud Degroote
To be more precise, what we try is doing
import numpy
which fails with
ImportError: cannot import name 'multiarray'
(whole discussion is here : https://github.com/morse-simulator/morse/pull/630)
It basically fails because python from Ubuntu is built with
multiarch option and so the libraries have extension
cpython-34m-x86_64-linux-gnu.so. It is not the case of the python
embedded in binary blender.
Of course, it is always possible, in this case, to use the numpy
copy from blender (but it is not a generic solution), or to hack the
filesystem (but it is cumbersome for users).

Alternatively, you could recompile Blender yourself, it then works

Post by Arnaud Degroote

import sip
sip.__file__

'/usr/lib/python3/dist-packages/sip.cpython-34m-x86_64-linux-gnu.so'

However, that won't help your users if you want to distribute a binary
extension.

I don't know how much effort it would be to compile Blender with
multiarch-Python by default, and what kind of incompatibilities it
would give on other platforms/distributions. Maybe someone else can
comment on this?

[Arnaud Degroote]

Post by Sybren A. Stüvel

Post by Arnaud Degroote
To be more precise, what we try is doing
import numpy
which fails with
ImportError: cannot import name 'multiarray'
(whole discussion is here : https://github.com/morse-simulator/morse/pull/630)
It basically fails because python from Ubuntu is built with
multiarch option and so the libraries have extension
cpython-34m-x86_64-linux-gnu.so. It is not the case of the python
embedded in binary blender.
Of course, it is always possible, in this case, to use the numpy
copy from blender (but it is not a generic solution), or to hack the
filesystem (but it is cumbersome for users).

Alternatively, you could recompile Blender yourself, it then works

It is what I do, and why I do not encounter the issue myself. Through,
it is not a really good alternative for our users.

Post by Sybren A. Stüvel

Post by Arnaud Degroote

import sip
sip.__file__

'/usr/lib/python3/dist-packages/sip.cpython-34m-x86_64-linux-gnu.so'
However, that won't help your users if you want to distribute a binary
extension.
I don't know how much effort it would be to compile Blender with
multiarch-Python by default, and what kind of incompatibilities it
would give on other platforms/distributions. Maybe someone else can
comment on this?
--
Sybren A. Stüvel
http://stuvelfoto.nl/
http://stuvel.eu/
_______________________________________________
Bf-python mailing list
http://lists.blender.org/mailman/listinfo/bf-python

[Christopher Barry]

On Wed, 17 Jun 2015 16:53:56 +0200

Post by Arnaud Degroote

Post by Sybren A. Stüvel
Hi Pierrick,

Post by Pierrick Koch
I'd like to imort python package from ubuntu distribution into
blender. Their shared object extension is
.cpython-34m-x86_64-linux-gnu.so, while Blender look for
.cpython-34m.so (without "-x86_64-linux-gnu").

How did you gather this knowledge? It helps a lot to see what you're
actually doing.

To be more precise, what we try is doing
import numpy
which fails with
ImportError: cannot import name 'multiarray'
https://github.com/morse-simulator/morse/pull/630)
It basically fails because python from Ubuntu is built with multiarch
option and so the libraries have extension
cpython-34m-x86_64-linux-gnu.so. It is not the case of the python
embedded in binary blender.
Of course, it is always possible, in this case, to use the numpy copy
from blender (but it is not a generic solution), or to hack the
filesystem (but it is cumbersome for users).
Best regards,

Actually the symlink solution presented in the text you removed on
reply is the typical 'behind the scenes' way this kind of thing usually
gets done by the package manager at install time, using the packages'
postinst script (in debian-derived distributions anyway).

In a terminal run:

$ ls -la /{usr/,}lib{,64}

on your ubuntu box to see a good example of how prevalent this method
is with system libraries.

also:

$ man update-alternatives

for how this methodology is used also for command aliasing and
prioritizing.

try:

$ update-alternatives --display java

to see how java is configured, and

$ for a in $(update-alternatives --get-selections \
| awk '{ print $1 }'); do
update-alternatives --display "$a";
echo;
done | less

to see how all commands with alternatives are setup.


So, it's really not a filesystem 'hack', it's a convenient and simple
way that works for all programs and libs.

(heh, and i swear i did not pick this fortune either, but it's very
apropos... :)
--
Regards,
Christopher Barry

Random geeky fortune:
Those who do not understand Unix are condemned to reinvent it, poorly.
-- Henry Spencer, University of Toronto Unix hack

[Arnaud Degroote]

Post by Christopher Barry
On Wed, 17 Jun 2015 16:53:56 +0200

Post by Arnaud Degroote

Post by Sybren A. Stüvel
Hi Pierrick,

Post by Pierrick Koch
I'd like to imort python package from ubuntu distribution into
blender. Their shared object extension is
.cpython-34m-x86_64-linux-gnu.so, while Blender look for
.cpython-34m.so (without "-x86_64-linux-gnu").

How did you gather this knowledge? It helps a lot to see what you're
actually doing.

To be more precise, what we try is doing
import numpy
which fails with
ImportError: cannot import name 'multiarray'
https://github.com/morse-simulator/morse/pull/630)
It basically fails because python from Ubuntu is built with multiarch
option and so the libraries have extension
cpython-34m-x86_64-linux-gnu.so. It is not the case of the python
embedded in binary blender.
Of course, it is always possible, in this case, to use the numpy copy
from blender (but it is not a generic solution), or to hack the
filesystem (but it is cumbersome for users).
Best regards,

Actually the symlink solution presented in the text you removed on
reply is the typical 'behind the scenes' way this kind of thing usually
gets done by the package manager at install time, using the packages'
postinst script (in debian-derived distributions anyway).
$ ls -la /{usr/,}lib{,64}
on your ubuntu box to see a good example of how prevalent this method
is with system libraries.
$ man update-alternatives
for how this methodology is used also for command aliasing and
prioritizing.
$ update-alternatives --display java
to see how java is configured, and
$ for a in $(update-alternatives --get-selections \
| awk '{ print $1 }'); do
update-alternatives --display "$a";
echo;
done | less
to see how all commands with alternatives are setup.
So, it's really not a filesystem 'hack', it's a convenient and simple
way that works for all programs and libs.

Well, for system-use, I agree.
The easy solution would be to link

multiarray.cpython-34m-x86_64-linux-gnu.so -> multiarray.cpython-34m.so

but it defeats completely the multiarch work (and it requires root access).

Linking only the .egg somewhere in $HOME won't help, because it properly
finds numpy in the system.

The last solution requires to link all the numpy files (with the good
name) one by one somewhere in $HOME, with the right hierarchy (because
of all the relative import). For standard users, it looks hard (of
course, if all the users understand Unix, the World would be so much easier
:)). We can provide of course a "smart enough" script to do that, but
how long will be smart enough to handle differences between
distributions / packages version ?

[Sybren A. Stüvel]

Post by Arnaud Degroote
how long will be smart enough to handle differences between
distributions / packages version ?

Distributions: that's a good point. Ubuntu's Blender package is built with Python in multiarch
configuration, so when you stick to packages of a single distribution things Just Work. The only
time things go wrong, is when someone wants to mix self-downloaded Blender with
distribution-packaged Python modules. Unfortunately, that's probably a common scenario.

If Numpy is really the only package that gives you practical problems right now, I would simply
use Blender's built-in version.

[Dalai Felinto]

Hi,

If it's a one time thing that doesn't need to run in a different computer
you can append the folder of your shared libraries to sys.path.

That's also useful when you run "blender --python myscript.py" and need to
import another module that is in the same folder.

Cheers,
Dalai

Post by Pierrick Koch
Hello,
I'd like to imort python package from ubuntu distribution into blender.
Their shared object extension is .cpython-34m-x86_64-linux-gnu.so,
while Blender look for .cpython-34m.so (without "-x86_64-linux-gnu").
Do you know any way to override this issue,
or a way to let Blender Python look for both extension ?
Best,
--
Pierrick Koch
_______________________________________________
Bf-python mailing list
http://lists.blender.org/mailman/listinfo/bf-python