0

I'm working on a package that uses data from external git repository. When it's doing its job, it first cloning git repository and then copies files from it to some other location.

Where should I save this repository (and other non-python files) in my filesystem? Is there any standard place for that?

Sure, I could just use site-packages/ directory for my files. But the problem is, git repository could contain python packages too, and I don't want them to be importable.

Is there, maybe, some way to specifically exclude some folder from site-packages/? I think *.dist-info folders are excluded, should I create a fake one for my package?

Thank you very much.

mkurnikov
  • 1,581
  • 2
  • 16
  • 19
  • First, I can't be sure that it will be existed on the system. But in case I don't know smth, I still want that files to survive reboot, so `/tmp` isn't quite a case. – mkurnikov Mar 04 '16 at 14:48
  • `/tmp` will exist on all GNU/Linux systems. What is the OS you use? –  Mar 04 '16 at 14:48
  • If possible, I wanted it to be inside virtualenv directory. Or is it non-common practice? I don't really have much experience with that sort of things. Is it ok to use smth like $HOME for package-specific files? For me, it looks like unnecessary polluting the system. – mkurnikov Mar 04 '16 at 14:50
  • I'm using Debian, and yes, /tmp is there. What about Windows? – mkurnikov Mar 04 '16 at 14:51
  • I'd say be bold, pick somewhere, and then let users override the default location either with an argument or environment variable – Simon Fraser Mar 04 '16 at 15:01
  • Feels bad, but acceptable. Looks like pip using `~/.cache/pip` for caches. I guess I'm just overthinking a problem here. It would be nice, though, to have standard directory for package-specific files inside `virtualenv/` root. – mkurnikov Mar 04 '16 at 15:11

1 Answers1

0

Install them into a subdirectory of your package directory in site-packages. If the subdirectory doesn't have an __init__.py file or if it's name has a dash (-) or other character that isn't valid in a Python identifier it can't be imported using the import statement nor can any Python file located under it.

So for example, if your package name is mypackage, you could use site-packages/mypackage/data-files as the location to store your data.

Ross Ridge
  • 38,414
  • 7
  • 81
  • 112
  • This is why `.dist-info` files are not importable, not because of some weird exclusion somewhere, right? – mkurnikov Mar 05 '16 at 11:07
  • It looks like omitting `__init__.py` on the top-level doesn't solve a problem, because when you're doing that, Python 3 thinks that it's a [namespace package](http://stackoverflow.com/questions/1675734/how-do-i-create-a-namespace-package-in-python/27586272#27586272). Also, I kind of can't be sure that my folder won't have `__init__.py` file on top-level anyway. Although, adding `-` as a part of top-level directory is a perfectly valid and nice way to solve the problem. – mkurnikov Mar 05 '16 at 11:17
  • If my remarks are valid, please remove the part about `__init__.py` from the answer and I'll accept it. – mkurnikov Mar 05 '16 at 11:21
  • @mkurnikov I didn't suggest omitting `__init__.py` from the "top level". I suggested omitting from the subdirectory of your package directory. Your package directory (`site-packages/mypackage` in my example) must have an `__init__.py` file otherwise it's not a package. Any subdirectories of the package directory (eg. `site-packages/mypackage/datafiles`) can't become namespace packages. Only directories in `sys.path` are searched (eg. `site-packages`), not any of their subdirectories. Only direct subdirectories found in the directories in `sys.path` can become namespace packages. – Ross Ridge Mar 05 '16 at 17:34
  • Yes, that definitely makes sense. Sorry for the misunderstanding. Keeping non-python source code files in package directory still feels kind of ugly, but I guess it's my own problems. I'm accepting the answer and lets close it here. Thank you. – mkurnikov Mar 05 '16 at 17:58