3

Running pip install seems to create the directory structure + files specified in data_files in /usr/local

However, if I run:

import sys
sys.prefix

I get the string /usr.

Is there any way to figure out where pip installed the data_files for a specific package in a distribution/OS agnostic way ?

Note: I am installing a package from a github repostiroy instead of pypi so maybe this results in the different behavior ?

George
  • 3,521
  • 4
  • 30
  • 75

1 Answers1

4

I believe you should work with sysconfig.

First try:

path/to/pythonX.Y -m sysconfig

And then try its get_path function:

import sysconfig

data_path_str = sysconfig.get_path('data')
print("data_path_str", data_path_str)
sinoroc
  • 18,409
  • 2
  • 39
  • 70
  • This still shows '/usr' rather than '/usr/local' for me :/ – George Jun 19 '20 at 10:45
  • Note: I am installing a package from a github repostiroy instead of pypi so maybe this results in the different behavior ? – George Jun 19 '20 at 10:48
  • You might want to update your question to give more details. Could you point at what project exactly you're installing, and what command you're using to install it? What Python interpreter are you using, and how did you install it? – sinoroc Jun 19 '20 at 12:01
  • The specific thing I'm installing is this: git+https://github.com/mindsdb/mindsdb_server.git@split But it's a rather complex pacakge, I will try to replicate with a simpler example later – George Jun 19 '20 at 20:07
  • The important bit is the `data_files` argument. From what I can see [here](https://github.com/mindsdb/mindsdb_server/blob/split/setup.py) the paths are absolute, so `sys.prefix` or whatever other value shouldn't matter. A [mcve] would be helpful indeed. – sinoroc Jun 19 '20 at 22:58
  • So, upon further digging it seems that data_files and package_files should probably not be used: https://github.com/pypa/sampleproject/issues/30 and the issue I'm running into is likely a bug. (As for the absolute paths, sorry for that, I was just experimenting with stuff and hence why I'd need a minimal example to reproduce, but for now I will give up on the issue) – George Jun 20 '20 at 01:00
  • From my point of view `data_files` should not be part of any Python project. I believe such things should be handled by the operating system's package manager (for example `.msi` or `.exe` installer on Windows, _apt_ / `.deb` or _yum_ / `.rpm`, etc. on Linux). But `package_data` is perfectly fine, I use them a lot, never had any issues with them. I wrote myself a reference on the topic: https://sinoroc.gitlab.io/kb/python/package_data.html – sinoroc Jun 20 '20 at 07:46
  • Noted, will read through it later, for my purpose I think what I need would more so fit under the definition of `package_data` (i.e. files that are generate by the package once it is installed) – George Jun 20 '20 at 11:39
  • Wait. Data that is **generated after the installation** doesn't belong in `package_data`, that's impossible. Such data belongs in something like `~/.local/share/MyApplication/data`, see for example [_appdirs_](https://pypi.org/project/appdirs/) _user data dir_. – sinoroc Jun 20 '20 at 15:01
  • Oh, alright, that was my misunderstanding then, happily enough that's the solution I ended up going with (though I am placing it in the package dir as a last resort if none of the other paths have the desired permissions) – George Jun 21 '20 at 19:46
  • Do you mean that as a last resort you add files to the directory where the packages of the project are installed (usually a `site-packages` directory), at run-time (after the installation has finished)? If yes, that is not OK. This is not something that one should do. This will likely cause issues sooner or later, for example when trying to uninstall the project. – sinoroc Jun 24 '20 at 15:40