3

I'm trying to create an auto_import function which is part of a library: the purpose of this to avoid listing from .x import y many times in __init__ files, only do something this import lib; lib.auto_import(__file__) <- this would search for python files in that folder where the __init__ is present and would import all stuff by exec statement (i.e. exec('from .x import abc')).

My problem is that, somehow the 'from' statement always tries to import .x from lib directory, even if I change the cwd to the directory where the actual __init__ file is placed... How should I solve this? How should I change the search dir for from . statement?

Structure:

$ ls -R
.:
app.py  lib  x

./lib:
__init__.py  auto_import.py

./x:
__init__.py  y

./x/y:
__init__.py  y.py

e.g.: ./x/y/__init__.py contains import lib; lib.auto_import(__file__) auto_import is checking for files in dir of __file__ and import them with exec('from .{} import *') (but this from . is always the lib folder and not the dir of __file__, and that is my question, how to change this to dir of __file__ Of course the whole stuff is imported in app.py like:

import x
print(x.y)

Thanks

EDIT1: final auto_import (globals() / gns cannot be avoided )

import os, sys, inspect

def auto_import(gns):
  current_frame = inspect.currentframe()
  caller_frame = inspect.getouterframes(current_frame)[1]
  src_file = caller_frame[1]
  for item in os.listdir(os.path.dirname(src_file)):
    item = item.split('.py')[0]

    if item in ['__init__', '__pycache__']:
      continue

    gns.update(__import__(item, gns, locals(), ['*'], 1).__dict__)
Zoe
  • 27,060
  • 21
  • 118
  • 148
vpas
  • 513
  • 1
  • 5
  • 18
  • Side note: although I have not seen this till the end yet, you might find some useful things [here](https://www.youtube.com/watch?v=0oTh1CXRaQ0). – quapka May 13 '16 at 21:12
  • Can you give an example of the file/folder structure of your lib? It's quite difficult to follow your question. – gdlmx May 13 '16 at 21:20
  • ok, added the structure – vpas May 13 '16 at 21:22
  • Now the question is clearer. The problem of your approach is that `auto_import` is defined in `lib/auto_import.py` so the context for `exec('from .x import *')` is always `lib/`. Even though you manage to fix the path problem, `lib.auto_import(__file__)` will not import anything to the namespace of `lib.x.y`, because the function locates in another module. – gdlmx May 13 '16 at 21:37
  • So, do I need to always manually list the packages of X in the __init__ ??? Even if there is 1-5-10-20-500 etc?? that's funny – vpas May 13 '16 at 21:40
  • 1
    No, there is a better way to do that, I will test it and post an answer later – gdlmx May 13 '16 at 21:42
  • I also want to avoid to put the auto_import to all package folders end call them in init file (I guess it would solve my problem about listing packages manually, but imho it would be quite ugly) – vpas May 13 '16 at 21:42

2 Answers2

1

The problem of your approach is that auto_import is defined in lib/auto_import.py so the context for exec('from .x import *') is always lib/. Even though you manage to fix the path problem, lib.auto_import(__file__) will not import anything to the namespace of lib.x.y, because the function locates in another module.

Use the built-in function __import__

Here is the auto_import script:

myimporter.py

# myimporter.py
def __import_siblings__(gns, lns={}):
  for name in find_sibling_names(gns['__file__']):
    gns.update((k,v) for k,v in __import__(name, gns,lns).__dict__.items() if not k.startswith('_'))
import re,os
def find_sibling_names(filename):
  pyfp = re.compile(r'([a-zA-Z]\w*)\.py$')
  files = (pyfp.match(f) for f in os.listdir(os.path.dirname(filename)))
  return set(f.group(1) for f in files if f)

Inside your lib/x/y/__init__.py

#lib/x/y/__init__.py
from myimporter import __import_siblings__
__import_siblings__(globals())

Let's say you have a dummy module that need to be imported to y: 

#lib/x/y/dummy.py
def hello():
  print 'hello'

Test it:

import x.y
x.y.hello()

Please be aware that from lib import * is usually a bad habit because of namespace pollution. Use it with caution.

Refs: 1 2

Community
  • 1
  • 1
gdlmx
  • 6,479
  • 1
  • 21
  • 39
  • Working! And almost beautiful, but is it possible to somehow avoid the globals(), locals() pass always to the importer function? I mean, it would be more beautiful if those "code duplications" would not be there always, instead somehow the import function would "calculate" them – vpas May 13 '16 at 22:59
  • As I see the critical is the globals variable as this still gives fine output: globals().update(__import__(item, gns, locals(), ['*'], 1).__dict__) but if i replace that gns with "globals()" then it cannot import – vpas May 13 '16 at 23:01
  • ok... maybe this is also applicable for the __file__ variable, but still, im courius if you have any idea about that :) – vpas May 13 '16 at 23:04
  • `gns` is important for the `__import__` function because it uses this namespace to determine the searching path. There is some tricks to avoid passing `globals()` using CPython stack_frame, but that is not standard and only works with CPython. – gdlmx May 13 '16 at 23:06
0

Use sys module With this folder structure:

RootDir
    |-- module.py
    |--ChildDir
        |-- main.py

Now in main.py you can do

import sys
sys.path.append('..')
import module

A believe there are other hacks, but this is the one I know of and works for my purpose. I am not sure, whether it is the best option to go for some kind of auto_import stuff though.

Park
  • 2,446
  • 1
  • 16
  • 25
quapka
  • 2,799
  • 4
  • 21
  • 35
  • I don't need another solution, I need the correction for my solution, thanks ;) – vpas May 13 '16 at 21:10
  • This is not safe because it affect the python interpreter and may accidentally break other libraries. – gdlmx May 13 '16 at 21:13