3

I'm relatively new to Python, and I'm struggling to understand if what I'm trying to do is just not possible, or if I am just doing it wrong. I admit that I come from Perl - where the rules for where you put your packages, and how the directories should look like is a bit looser, so my point of view my be not so pythonic.

Also, this is a different version of a question I've asked a couple of times here and here and I keep getting answers (that I am grateful for) that say I should not, but never "it's impossible" or "here's how you do it, though you should not". If something is possible, but not advisable, I would rather be able to find out why by trying and seeing and if it is impossible by design, I'd like to know (and stop banging my head against a wall).

Apologies in advance for the long intro - here's the question, finally.

I have this nice module - svgpathtools - that contains methods, classes and sits in an install-specific directory (inside the Python install that comes with Blender), directory that looks like this:

/home/simone/blender/blender-2.92.0-linux64/2.92/python/lib/python3.7/site-packages/svgpathtools
├── __init__.py
├── __pycache__
│   ├── __init__.cpython-37.pyc
│   ├── bezier.cpython-37.pyc
│   ├── document.cpython-37.pyc
│   ├── misctools.cpython-37.pyc
│   ├── parser.cpython-37.pyc
│   ├── path.cpython-37.pyc
│   ├── paths2svg.cpython-37.pyc
│   ├── polytools.cpython-37.pyc
│   ├── smoothing.cpython-37.pyc
│   ├── svg_io_sax.cpython-37.pyc
│   └── svg_to_paths.cpython-37.pyc
├── bezier.py
├── document.py
├── misctools.py
├── parser.py
├── path.py
├── paths2svg.py
├── polytools.py
├── smoothing.py
├── svg_io_sax.py
└── svg_to_paths.py

1 directory, 22 files

I want to add functionality to it without messing with that directory and without subclassing. The first for obvious reasons, the second because for now I am only adding a couple of methods max, so it's not worth maintaining a whole subclass (for now at least). So my idea is to have a directory that looks like this:

/home/simone/blender_learning/mesh-20210920
└── svgpathtools
    ├── __init__.py
    └── distance.py

Now: if I keep svgpathtools/__init__.py, python (3.8.10) won't find the original svgpathtools in the Blender install subdirectory, so all its functionality is gone and if I remove svgpathtools/__init__.py, python won't find svgpathtools/distance.py so import svgpathtools.distance throws an error.

Which one is it then:

  • it's impossible: because of the way python looks for packages, you can't have sub-packages in a different place than the main package
  • I am doing it wrong in some way (please tell me why)

Pretty please: don't just tell me I should not do it without having told me which one it is. I know that doable <> advisable (playing with explosives? petting hungry leopards? eating fugu?)

simone
  • 4,667
  • 4
  • 25
  • 47
  • Depends what kind of functionality it is you want to add. In a separate file, you can use reflection stuff to modify the `svgpathtools` module object once you've imported it, which will apply globally throughout the program. Easiest way to make this happen would be to write your own files/module with these ones as dependencies, and then either code passthrough methods or modify what you need to. – Green Cloak Guy Sep 20 '21 at 14:12
  • @GreenCloakGuy - did exaclty that, with a different module name (see the previous questions linked in the post) but the stumbling block was why it's impossible to call this new module ```svgpathtools.distance```, for example – simone Sep 20 '21 at 14:40
  • you already know it does not work that way . it is not "simpler" if it takes you to deal with the inner mechanisms of the language. The correct thing to do there is to have a package with another name - and, if it is the case, to add a "plug-in system" to the original package so that it could find more code that is designet to work along with it, and expose that code to users of the original package, – jsbueno Sep 20 '21 at 14:55
  • @jsbueno - as someone that's picked up python as a hobby a couple months ago, it's not always easy to understand if it doesn't work because of something I do wrong, or because it just doesn't work that way – simone Sep 20 '21 at 14:59
  • it does not work that way. the answers you are getting are pretty comprehensive. Basically a directory is a package, and the first package with the name given on "import" Python finds is used. It does nto magically keep looking for package with the same names and magically "merge" them. That would be a mess (user installs unable to shadow system installs of older versions of the same package, etc...). But there are ways of using "sub-packages" using namespaces sparated by dots. They are complicated to the point most projects phased out this approach. – jsbueno Sep 20 '21 at 22:13

3 Answers3

1

When you import a module import <module> python looks through the paths for the first match

import sys
print(sys.path)

So having a module with the same name ends up shadowing it, rather than what you want which is keep looking and merge them. So you'll have to name your extensions package something else to not conflict with it.

You can probably get the behaviour you want by rolling your own import method with importlib and a wrapper around the module to fallback to the other but it seems hardly worth it.

Steven Summers
  • 5,079
  • 2
  • 20
  • 31
  • thanks - and (a) it stops at the first ```__init__.py``` and (b) if it does not find an ```__init__.py``` it does not look into subdirs, right? – simone Sep 20 '21 at 14:47
  • It would not look into subdirs it only checks the paths in that list. You can have sub packages that you import by making the top level a package. – Steven Summers Sep 20 '21 at 14:50
  • and that's when I run into conflict – simone Sep 20 '21 at 14:52
0

Use e.g. sys.path.append("..") to add the directory above the script to the python modules path.

Janu
  • 44
  • 6
0

It's not impossible but yes: it's explosive. And that's because of the way python stores its references to the imported modules. If you do:

import sys
print(sys.modules)

you'll see that those references are stored in a dictionnary, and that the keys are the modules names.

Although you could find a messy way to overthrow it, the simplest way would probably using a new name for your own lib, something like svgpathtools_ or svgpathtools_extended for example.

olinox14
  • 6,177
  • 2
  • 22
  • 39
  • Aaargh - I was afraid so. I did check . output shows more than I can show here, with svgpathtools in an __init__.py file, and a bunch of submodules to that. I did work around it as you suggested: ```svgpathmoretools``` isn't such a terrible name. But why why why says the perl-er in me? just a design choice that stuck? And would you be so kind and point in the direction of how (I promise not to - just learning) – simone Sep 20 '21 at 14:36
  • you mean perl actually do magically "jsu uses" whateverr files it finds in the same directory name in its search path? for real?? that would be a portal to hell for Python-minded people. :-) – jsbueno Sep 20 '21 at 22:15