3

The goal is to have a universal directory where I can add packages on the network that are automatically added to sys.path without needing to run site.addsitedir or sys.path.append each time I import said packages. Is there a way to do this?

Background: I have a small network of users who all need access to the same scripts. Every time I want to add a new package for them to use, I add the path to their PYTHONPATH environment variable. The user base has started to grow, and so have the number of packages.

If I could set up a master.pth on the network that is loaded when they start any of the scripts (without requiring extra code in all of them), I'd be very grateful. It appears, though, that you can't nest .pth files, so simply adding a pointer .pth to the master.pth directory doesn't seem to work.

Edit: Regarding Comments from @S.Lott (was a little big for a comment): Take, for example, wxPython. Three objects are placed into the site-packages directory: wxversion.py, a directory called wx-2.8-msw-unicode, and a .pth file. The .pth file points to that directory, which contains all of the importable packages (wx, wxPython, etc.). I currently have three other packages that are structured in a similar way, one of which has several base modules. I would need to move all of the importable modules into the same directory to get the desired result.

If, however, .pth files could be "nested", I could add all of these directories to master.pth, keeping a relatively clean folder. I hope that makes sense, and thanks for your help!

TorelTwiddler
  • 5,996
  • 2
  • 32
  • 39
  • Simply adding separate `.pth` files to Python's ordinary site-packages directory works perfectly. What's wrong with doing that? Why add another file and this "indirection"? – S.Lott Jul 06 '11 at 19:22
  • @S.Lott: Mostly because I don't want to update/add the `.pth` files every time I add a new package to everyone on the network. I want to do a one time setup and just update the network directory after that. – TorelTwiddler Jul 06 '11 at 19:26
  • There is no "royal road" to system administration. You should probably reconcile yourself to writing, debugging and support installers for your users to add packages to their various Python installations. Or. Use a single shared directory for all of your users. – S.Lott Jul 07 '11 at 01:20
  • Using a single shared directory is the closest thing to what I'm looking for. I guess I'll just have to live with a messy shared directory (all importable packages must be in the same directory). – TorelTwiddler Jul 07 '11 at 17:57
  • "messy shared directory"? But that's what a lot of people do to keep things simple and neat. I don't get what you're complaining about. Please elaborate. – S.Lott Jul 07 '11 at 18:00
  • I've added a reply to your comment in the original question. – TorelTwiddler Jul 07 '11 at 18:18
  • "I would need to move all of the importable modules into the same directory to get the desired result."? And? What's wrong with that? I don't get the problem. – S.Lott Jul 07 '11 at 18:19
  • I go from having 4, well named, versioned directories and a pth file, to 14 unversioned base modules that require investigation to figure out which one it belongs to. The idea is to get things to work and be easily manageable, rather than just work. – TorelTwiddler Jul 07 '11 at 18:24
  • Where did the directories go? Why did they disappear? My site-packages has a few .PTH files and a few subdirectories. Why did your subdirectories vanish? – S.Lott Jul 07 '11 at 18:38
  • I'm discussing of the theoretical "networked site-packages" directory, not the actual one that is found in the python directory. This "networked site-packages" directory does not read .pth files, even though it is in the PYTHONPATHS environment variable, because it is not the _actual_ site-packages directory. Does that make sense? – TorelTwiddler Jul 07 '11 at 18:49
  • If the network share is on PYTHONPATH, then the packages are directly available and the .PTH files don't matter, do they? – S.Lott Jul 07 '11 at 19:05
  • This, unfortunately, does not work. For example: create a directory outside of the python directory, say `C:\foo`. Download wxPython, and place it in that directory (`C:\foo\wx-2.8-msw-unicode`). Add more if you like. Add the directory `C:\foo` to your `PYTHONPATHS`. Open a new python interactive shell and run `import sys; for i in sys.paths: print i` to verify that the path is there, then `import wx`. You will get an Import Error (assuming you don't have wxPython in your site-packages). – TorelTwiddler Jul 07 '11 at 19:35
  • That's your **real** problem. Please post that as your real problem. It should work. It does for me. I'm hoping that the `PYTHONPATHS` in your comment is just a typing error, since it should be `PYTHONPATH`. Please post a question describing this problem, since solving that problem makes this question unnecessary. – S.Lott Jul 07 '11 at 19:55
  • Yes, sorry, `PYTHONPATHS` was a typo. I will update the original question. – TorelTwiddler Jul 07 '11 at 19:58
  • You might want to open a **new** question with the `PYTHONPATH` issue, completely separate from this. – S.Lott Jul 07 '11 at 20:18
  • Does my answer work for you? If so, please accept it; if not, what problems are you having? – Ethan Furman Mar 16 '12 at 16:11
  • This is especially important in `virtualenv` installations. It took me a very long time to realize that this was the cause of not loading modules from a site-packages folder added to virtualenv by way of the wrapper function `add2virtualenv` (which effectively does what @TorelTwiddler wants to do manually). The behavior is non-intuitive and (in my opinion) is a python bug. At the very least, the error message needs to be changed; Python should know that there's a nested pth file that it refused to import. – mbauman May 21 '12 at 21:31
  • Note that this is [issue #4033](http://bugs.python.org/issue4033) in the Python bug tracker. It's open, but assigned as a documentation issue. – mbauman May 21 '12 at 21:43

1 Answers1

1

You have a couple options:

  • modify your site.py file -- look for the PREFIXES list and add your network path to it:

  • Have one custom .pth file in each machine's site-packages folder with the following line:

    import sys; sys.path.append('/network/path/to/modules_and_packages')

The downside to using site.py is that when you upgrade to a different python you'll have to find and adjust all the site.pys on all the machines. Using the custom .pth file it's a simple matter of copying into the new 'site-packages' folders.

Both solutions allow you to have the network location set up just like the normal site-packages folder. For example:

m:\python_site_packages\   # network drive and folder
   |
   |- dbf.py  # for accessing dBase III and VFP .dbf files
   |
   |- web\
       |- __init__.py
       |- application.py
       |- (etc)

Just a reminder -- if the server becomes unavailable, so will all the network packages.

Ethan Furman
  • 63,992
  • 20
  • 159
  • 237
  • I don't believe that sys.path.append will load `.pth` files in the appended location. At least, it doesn't from within virtualenv on Python 2.7. – mbauman May 21 '12 at 22:33