Proper way of implementing one function per file in Python

Question

I'm developing a module to help me deal with files, that's the structure:

Archive/
| archive/
  - __init__.py
  - move.py
  - rename.py
  - mime_type.py
| setup.py

I'm designing one file - one function. For me, it's better, i feel more comfortable with this style since i have an Node.js and C/C++ background.

So, my doubt is about how to implement the __init__.py to be able to call the functions using:

from archive import move
from archive import rename
from archive import mime_type

Instead of:

from archive import move.move
from archive import rename.rename
from archive import mime_type.mime_type

I'm doing like this:

__init__.py

from move import move
from rename import rename
from mime_type import mime_type

Is there an easier way to achieve automatically this behaviour? Without the necessity of changing the __init__.py each time i create an file.

Thanks.

I think you are stuck with this. Python generally does not follow single-file for single-function code structure, so there probably isn't a "proper" way. A single file should be a *module*, a collections of functions.You are making it more difficult for yourself doing it that way. — juanpa.arrivillaga, Jan 18 '17 at 19:31
Can't believe in this statement @juanpa.arrivillaga, i feel much more comfortable using this design, and that's all it is, a design decision. I think is so much easier to find functions, maintain, read and extend. Also, if i have one function per file, it's more clear which dependencies the function need to run. — , Jan 18 '17 at 19:34
@FXAMN well, then you will have to do some hacking in your `__init__`. It will require probably `os` and `importlib` as stated in some comments/answers. It will be confusing for someone who reads your code, that's for sure. Sure, it is a design decision, but you are using a tool, Python, that makes that design decision difficult to implement. This is your choice, though. In short, there is no "easy" way, and you will have to implement some hack to achieve it. — juanpa.arrivillaga, Jan 18 '17 at 19:36
Cool, i'm going to search about it. About the confusion, i think this can be solved with a good readme, don't you think? I'm also developing this module that parses modules with this design and creates a 3d graphic showing how functions related to each other. — , Jan 18 '17 at 19:40
Neither C nor C++ has a one-function-per-file structure, and I'm not a Node user, but I'm pretty sure Node doesn't work that way either. If you're carrying these practices from your experience in C, C++, and Node, you were writing really weird and unwieldy code in those languages, too. — user2357112, Jan 18 '17 at 19:44
There are a lot of libs written with this design @user2357112, just search about it. — , Jan 18 '17 at 19:46
@user2357112 - C and C++ can both be written one function or class per file and some coding standards require it. In C++ especially, one .hpp per .cpp with single-class implementation is common. — tdelaney, Jan 18 '17 at 19:47
@tdelaney: You certainly *can* write one function per file in C or C++, just as you *can* write one function per file in Python, but I've never seen any major projects work that way. One class per file isn't that uncommon, but that's not what we're talking about here. — user2357112, Jan 18 '17 at 19:51
Really don't get it why do you see this as a problem.. this won't affect performance, this won't make the code unreadable... is just a design decision... if i expose my code to the world, the api will be exactly the same, the performance also.. isn't Python a flexibly language? @user2357112 — , Jan 18 '17 at 21:25
@FXAMN: For non-opinion-based reasons, you'll get slower imports and more potential for circular import problems. — user2357112, Jan 18 '17 at 21:56
Could you explain me why would be more slower? As pointed here: http://stackoverflow.com/questions/10501724/how-does-python-importing-exactly-work this would not affect the speed. — , Jan 18 '17 at 22:02
@FXAMN: That link says nothing to support your claim. As for why it'd be slower, you're making more filesystem calls and reading more files and directories. — user2357112, Jan 18 '17 at 22:29
Ok, read this one: http://effbot.org/zone/import-confusion.htm This article explain exactly how it works: "This means that it’s fairly cheap to import an already imported module; Python just has to look the module name up in a dictionary.". — , Jan 18 '17 at 22:55
About the circular dependencies i must agree, but this is easy to avoid or fix. — , Jan 18 '17 at 22:57
It's cheap to import an already-imported module, but I'm not talking about already-imported modules. I'm talking about the first time you import your `archive` package, when you'll be searching your directory and reading a whole bunch of individual files. — user2357112, Jan 19 '17 at 03:07
If this is the cost for a more readable code, i don't mine at all since the difference will be minimum. It's a great trade off. — , Jan 19 '17 at 12:04
Maybe this use case makes sense? I'm using a similar structure for a data analysis project where I make several very different figures. It makes sense to group these under a common folder called visualization, and using this format I can run the code as standalone scripts to produce high-quality figures, as well as import the code in an illustrative notebook, for example. — Peter9192, Aug 24 '18 at 14:03

score 2 · Accepted Answer · edited May 23 '17 at 11:45

2

You can use importlib to import the modules and then assign the function names dynamically using globals() and getattr:

from importlib import import_module
for f in ["move","rename","mime_type"]:
    import_module("."+f,"archive") #import archive.f
    globals()[f]=getattr(globals[f],f) #assign the function f.f to variable f

As suggested by tdelaney, you can even get all the module names dynamically using glob:

from os.path import splitext,join,basename,dirname
from glob import glob  
#get all *.py filenames in __file__'s folder that are not __file__.
files=[splitext(f)[0] for f in glob(join(dirname(__file__), '*.py')) 
       if f != basename(__file__)] 
for f in files:
    ...

edited May 23 '17 at 11:45

Community

1
1

answered Jan 18 '17 at 19:32

Saúl Pilatowsky-Cameo

1,224
1
15
20

Could you give me an example? – Jan 18 '17 at 19:35
1

You could add `glob.glob("*.py")` and `os.path.splitext` to take it a step further. – tdelaney Jan 18 '17 at 19:42
1

well... make that `glob.glob(os.path.join(os.path.basename(__file__), '*.py'))` but the general idea was there. – tdelaney Jan 18 '17 at 19:49
The OP wants to import the functions, not just the modules. – Stop harming Monica Jan 18 '17 at 20:29
Yeah, the fourth line of the first code block assigns the functions in the modules to the function names in the current module. – Saúl Pilatowsky-Cameo Jan 18 '17 at 20:31

score 2 · Answer 2 · answered Jan 18 '17 at 19:35

2

Use 'as':

from archive import move.move as move
from archive import rename.rename as rename
from archive import mime_type.mime_type as mime_type

answered Jan 18 '17 at 19:35

ANVGJSENGRDG

153
1
8

What is the difference? – Jan 18 '17 at 19:50
using the as keyword bind the local namespace for the module. so on an imported you can call move(x) rather than move.move(x) – ANVGJSENGRDG Jan 18 '17 at 22:16

Peter9192 · Answer 3 · 2018-08-24T14:26:57.163

Building upon Saulpila's answer I ended up with the following:

import os.path
from glob import glob

# Get all *.py filenames in __init__'s folder
full_paths = glob(os.path.join(os.path.dirname(__file__), '*.py'))

# Remove path prefix and remove __init__.py from the list
file_names = [os.path.split(f)[1] for f in full_paths if f != __file__]

# Remove extension
files = [os.path.splitext(f)[0] for f in file_names]

for f in files:
    # from archive import f (__name__ is 'archive')
    __import__(__name__+'.'+f, globals=globals())

    # assign the function f.main to variable f
    globals()[f]=getattr(globals()[f], 'main')

score 0 · Answer 4 · answered Jan 20 '19 at 21:46

My way is to use `exec()`.

__init__.py

import glob
mod_list = [file_name.split('/')[-1].split('.')[0] for file_name in glob.glob(__path__[0]+'/[!_]*.py')]
sub_pack_list = [folder_name.split('/')[-2] for folder_name in glob.glob(__path__[0]+'/[!_]*/')]

# import functions
for mod in mod_list:
    exec('from . import ' + mod)
    exec('from .' + mod + ' import *')

# import subpackages
for pack in sub_pack_list:
    exec('from . import ' + pack)

# clean up
del glob
try:
    del mod
except:
    pass
try:
    del pack
except:
    pass

After import archive, you will see archive.move() available to you.

As a bonus effect, you have archive.mod_list and archive.sub_pack_list available. If you don't like them, you can also delete them in __init__.py.

One step further

You can still have multiple functions in a file, say you have two functions move_file() and move_folder() in your move.py. You can add a final line

__all__ = ['move_file', 'move_folder']

in move.py to control what functions should be available in the scope of archive.***

Why not put the `del mod` and `del pack` inside of the for loops? you won't get a syntax error since the loop doesn't run if there are no items. — Walter Randomness, Oct 09 '20 at 12:20

Proper way of implementing one function per file in Python

4 Answers4

My way is to use `exec()`.

One step further

Linked

Proper way of implementing one function per file in Python

4 Answers4

My way is to use exec().

One step further

Linked

My way is to use `exec()`.