Best way to structure a larger project

Question

I have many different mini projects / pieces of functionality each in their own folders. I generally create a virtualenv for each folder. I am now going to be starting a larger project which will use several aspects of those different projects.

An example windows file structure might be:

Python_Projects
    Functionality xyz
        several python files
    Functionality abc
        several python files
    New Project
        requires functionality xyz and abc

Is it possible to keep all the different pieces of functionality in their own folders and simply import the relevant python files/functionality into the new project as and when they are needed?

Would this be the recommended way of doing this, or would it be better to copy the relevant python files / functionality into the new project.

Many thanks.

score 0 · Accepted Answer · edited May 23 '17 at 10:24

You could create a package for each project, and then install those packages into the virtualenv of the new project. Creating a package requires that you place an install.py script at the top level of the package directory. This page details the various things a setup script can do, but a basic one looks like this:

from distutils.core import setup

setup(name='xyz',
      py_modules=['xyz']
)

assuming you nest your projects as follows:

Python_Projects
Functionality xyz
    setup.py
    xyz
        __init__.py
        several python files
Functionality abc
    setup.py
    abc
        __init__.py
        several python files

If you don't have __init__.py files already, you can just create blank ones.

From the virtualenv of your new project, you can then run

pip install -e ../abc ../xyz

The -e flag means that a symlink to the abc and xyz packages will be created, so you can continue to edit them without reinstalling the packages each time.

Best way to structure a larger project

1 Answers1