Python 3: importing a function from a python file in a parent directory

Question

I'm new to unit testing and I'm trying to create my first unit test file in python 3. So I have the following really rudimentary folder structure just from imitating an online course.

D:\Projects\Projects\unit_testing_course  # parent dir absolute path

unit_testing_course/                      # root/parent folder
    first_project.py                      # has a function called avg() 
    tests/                                
        test_first_project.py             # I want to import avg() into this file

test_first_project.py is the file I'm using for unit testing, and I want to import the avg() function from first_project.py which exists in a parent directory above the it.

I'm on windows 10, I have git bash open to the following path: '/d/Projects/Projects/unit_testing_course' (i.e. the parent directory containing first_project.py, the module/function I'm trying to import to the test_first_project.py file within the tests child directory). I keep running python tests/test_first_project.py

I quickly found out I couldn't use something like 'from .. import first_project' or 'from .. import avg', which cause the 'ValueError: attempted relative import beyond top-level package' error message.

So from reading documentation I'm getting that both 'from' and 'import' are instructions that are meant to work with modules or packages. https://docs.python.org/3/tutorial/modules.html

"Python has a way to put definitions in a file and use them in a script or in an interactive instance of the interpreter. Such a file is called a module; definitions from a module can be imported into other modules or into the main module (the collection of variables that you have access to in a script executed at the top level and in calculator mode).

A module is a file containing Python definitions and statements. The file name is the module name with the suffix .py appended. Within a module, the module’s name (as a string) is available as the value of the global variable __name__."

So it sounds like as far as python 3 is concerned, every .py file is a module and module name is essentially the .py file name without the .py extension at the end. And it sounds like I'm trying to import a module from the parent directory of the test_first_project.py file.

that documentation link seems to be helpful for explaining the scenario where you have a module within the current directory you're executing a program/python shell from and you're just trying to import a function from that module into that program.

But in this case it seems like we need to actually tell python to look for a module named 'first_project' within a specific directory, the parent directory of the test_first_project.py file. I looked at the section on module search paths, https://docs.python.org/3/tutorial/modules.html#the-module-search-path

When a module named spam is imported, the interpreter first searches for a built-in module with that name. If not found, it then searches for a file named spam.py in a list of directories given by the variable sys.path. After initialization, Python programs can modify sys.path. The directory containing the script being run is placed at the beginning of the search path, ahead of the standard library path.

Then I tried...

import sys
import os
p_dir_path = os.path.abspath('../')
sys.path.append(p_dir_path)
from  first_project import avg

Which still gets me the same error message of 'ValueError: attempted relative import beyond top-level package'. I was hoping that modifying sys.path programatically could change things...

So what do I actually need to do in order to import a module from a parent directory? Or... is it not possible to do that in python? Do I need to be using a package instead of a module (i.e. do I need to create an init.py in my parent directory)?

I've seen other questions and posts on stackoverflow about trying to import a file in a different directory and other cases, but imitating the syntax on those questions hasn't helped me and I think I've spent way too much time on this. I'm also sure I'm not the only person who has wanted to create unit tests and got sidetracked by issues with import statements, modules, and packages.

Answer 1

You should set your PYTHONPATH variable appropriately for your project. In unit_testing_course, run

export PYTHONPATH=$(pwd)

in your shell. And consider the following code:

first_project.py

def avg(l):
    return sum(l)/len(l) if len(l) > 0 else 0

tests/test_first_project.py

import unittest 
from first_project import avg

class Test(unittest.TestCase):
    ...

tests/__init__.py

from .test_first_project import *

running python -m unittest tests will work as desired (provided python points to your desired python version). You can also use a virtualenv and modify its venv/bin/activate to perform this path setting automatically. When your project continues to grow, I recommend having two separate directories in the base - one for the project itself and one for the tests. Then use relative imports in each respectively. This will alow your project to be flexible should you choose to package it or use it as a submodule or something like this.