6

I maintain a Python utility that allows bpy to be installable as a Python module. Due to the hugeness of the spurce code, and the length of time it takes to download the libraries, I have chosen to provide this module as a wheel.

Unfortunately, platform differences and Blender runtime expectations makes support for this tricky at times.

Currently, one of my big goals is to get the Blender addon scripts directory to install into the correct location. The directory (simply named after the version of Blender API) has to exist in the same directory as the Python executable.

Unfortunately the way that setuptools works (or at least the way that I have it configured) the 2.79 directory is not always placed as a sibling to the Python executable. It fails on Windows platforms outside of virtual environments.

However, I noticed in setuptools documentation that you can specify eager_resources that supposedly guarantees the location of extracted files.

https://setuptools.readthedocs.io/en/latest/setuptools.html#automatic-resource-extraction https://setuptools.readthedocs.io/en/latest/pkg_resources.html#resource-extraction

There was a lot of hand waving and jargon in the documentation, and 0 examples. I'm really confused as to how to structure my setup.py file in order to guarantee the resource extraction. Currently, I just label the whole 2.79 directory as "scripts" in my setuptools Extension and ship it.

Is there a way to write my setup.py and package my module so as to guarantee the 2.79 directory's location is the same as the currently running python executable when someone runs

py -3.6.8-32 -m pip install bpy

Besides simply "hacking it in"? I was considering writing a install_requires module that would simply move it if possible but that is mangling with the user's file system and kind of hacky. However it's the route I am going to go if this proves impossible.

Here is the original issue for anyone interested.

https://github.com/TylerGubala/blenderpy/issues/13

My build process is identical to the process descsribed in my answer here

https://stackoverflow.com/a/51575996/6767685

  • "_The directory (simply named after the version of Blender API) has to exist in the same directory as the Python executable._" Can you expand on the reason why this has to be this way? – sinoroc Sep 10 '19 at 15:02
  • @sinoroc because the C code that builds into the `.pyd` module adds the hard-coded path `"./2.79/scripts/modules"` to the top of sys.path, and tries to import `bpy_types` from there, else error. [Related](https://github.com/TylerGubala/blenderpy/issues/13#issuecomment-417777002) –  Sep 10 '19 at 16:17
  • Oh... Hard-coded, not nice. But, does it look for `bpy_types` only in this hard coded directory or in the whole `sys.path`? – sinoroc Sep 10 '19 at 18:14
  • @sinoroc Fairly sure it's fixed function technically; Blender uses the `GHOST` library to find many filesystem paths, but the issue occurs around [the instantiation of the `bpy` Python module object in the C runtime code](https://github.com/sobotka/blender/blob/760dbd1cbf56e13b0a827afb6f7784fa46fff9b4/source/blender/python/intern/bpy.c#L397). Instantiation of the types module (who needs to be in the specific scripts directory) occurs [here](https://github.com/sobotka/blender/blob/760dbd1cbf56e13b0a827afb6f7784fa46fff9b4/source/blender/python/intern/bpy.c#L389) –  Sep 10 '19 at 23:37
  • 1
    Alright... I can't help with the C code, but it feels like the issue has to be solved in the C code and not in the setuptools packaging. Seems weird to me that things are expected to be found there. Anyway, have you tried the `data_files` option of setuptools? This should allow you to install files directly into `sys.prefix` which seems to be what you need. – sinoroc Sep 11 '19 at 09:29
  • @sinroc Allegedly Blender 3 could be a Python module, but that is far, far down the line. I am just trying to support it as extant and document the shortcomings. Thanks for your patience. –  Sep 11 '19 at 12:22

1 Answers1

0

Maybe try the data_files option of distutils/setuptools.

You could start by adding data_files=[('mydata', ['setup.py'],)], to your setuptools.setup function call. Build a wheel, then install it and see if you can find mydata/setup.py somewhere in your sys.prefix.

In your case the difficult part will be to compute the actual target directory (mydata in this example). It will depend on the platform (Linux, Windows, etc.), if it's in a virtual environment or not, if it's a global or local install (not actually feasible with wheels currently, see update below) and so on.

Finally of course, check that everything gets removed cleanly on uninstall. It's a bit unnecessary when working with virtual environments, but very important in case of a global installation.

Update

Looks like your use case requires a custom step at install time of your package (since the location of the binary for the Python interpreter relative to sys.prefix can not be known in advance). This can not be done currently with wheels. You have seen it yourself in this discussion.

Knowing this, my recommendation would be to follow the advice from Jan Vlcinsky in his comment for his answer to this question: Post install script after installing a wheel.

  • Add an extra setuptools console entry point to your package (let's call it bpyconfigure).

  • Instruct the users of your package to run it immediately after installing your package (pip install bpy && bpyconfigure).

  • The purpose of bpyconfigure should be clearly stated (in the documentation and maybe also as a notice shown in the console right after starting bpyconfigure) since it would write into locations of the file system where pip install does not usually write.

  • bpyconfigure should figure out where is the Python interpreter, and where to write the extra data.

  • The extra data to write should be packaged as package_data, so that it can be found with pkg_resources.

  • Of course bpyconfigure --uninstall should be available as well!

sinoroc
  • 18,409
  • 2
  • 39
  • 70
  • Sorry, but if it's not `wheel` compatible it doesn't help me. I've added that to my question. Thanks for your help so far. –  Sep 11 '19 at 12:00
  • I haven't tried it, but if I read the documentation right, only the absolute paths are incompatible with wheels. Relative paths should be fine. They are relative to `sys.prefix` and since `sys.prefix` points usually one level above the Python binary, that should be fine for your use case. Am I missing something here? – sinoroc Sep 11 '19 at 12:44
  • Apologies, I think I was the one missing the point! I'm going to try that and see. –  Sep 11 '19 at 13:19
  • You could start by adding `data_files=[('mydata', ['setup.py'],)],` to your `setuptools.setup` function call, build a wheel, install it and see if you can find `mydata/setup.py` somewhere in `sys.prefix`. In your case the difficult part will be to compute the actual target directory (`mydata` in my example), it will depend on the platform (Linux, Windows, etc.), if it's in a virtual environment or not, if it's a global or local install and so on. Also check what happens on uninstall. – sinoroc Sep 11 '19 at 13:30
  • Oh, I'm not sure that is going to work; since it won't be consistent even on the same platform (Windows `venv` different than system install) and the purpose is to supply a `wheel` –  Sep 12 '19 at 02:19
  • True. Have you had the chance to actually try it? Is there any chance that this folder is being looked up in `sys.prefix` instead of next to Python interpreter binary? Could you eventually provide a [mre]? If `data_files` truly isn't good enough, then I suggest to pack this as `package_data` and provide a custom _post-install_ script or _entry-point_ clever enough to copy the `package_data` to the right place, the users should run this once before actually using your package: https://stackoverflow.com/a/24749871/11138259 - I wouldn't try to pack everything as `setuptools` command. – sinoroc Sep 12 '19 at 08:58
  • Judging by the source code of the GHOST library, this _may_ work if I build it as a portable app from `cmake` but I have yet to attempt this. I'll see if I have enough time to try this tomarro, thanks. –  Sep 15 '19 at 07:10
  • This is exactly what I will have to do. Thank for your time and apologies that I didn't see your update soon enough to award you the bounty –  Sep 17 '19 at 14:23