Unable to find module/folder despite being in the same directory

Question

I am attempting to run a script which calls another python file (copied along with its entire github repo), but I am getting a ModuleNotFoundError: This is despite putting the files into the correct directories.

How I run it

1. Activate my python3.9 virtual environment
2. (newpy39) aevas@aevas-Precision-7560:~/Desktop/Dashboard_2021_12$ python3 dashboard.py where aevas is my username, and ~/Desktop/Dashboard_2021_12 is the folder

No additional arguments required to run it.

Imports for dashboard.py

import sys
sys.path.insert(1, 'targetTrack')

# Qt imports
from PyQt5.QtCore import QThread, pyqtSignal

import argparse
import configparser
import platform
import shutil
import time
import cv2
import torch
import torch.backends.cudnn as cudnn


from yolov5.utils.downloads import attempt_download
from yolov5.models.experimental import attempt_load
from yolov5.models.common import DetectMultiBackend
from yolov5.utils.datasets import LoadImages, LoadStreams
from yolov5.utils.general import LOGGER, check_img_size, non_max_suppression, scale_coords, check_imshow, xyxy2xywh
from yolov5.utils.torch_utils import select_device, time_sync
from yolov5.utils.plots import Annotator, colors
from deep_sort_pytorch.utils.parser import get_config
from deep_sort_pytorch.deep_sort import DeepSort

Part 1: Models not found, despite models being the parent folder.

python3 dashboard.py 
Traceback (most recent call last):
  File "/home/aevas/Desktop/Dashboard_2021_12/dashboard.py", line 25, in <module>
    from trackerThread import trackerDeepSORT
  File "/home/aevas/Desktop/Dashboard_2021_12/trackerThread.py", line 15, in <module>
    from yolov5.models.experimental import attempt_load
  File "/home/aevas/Desktop/Dashboard_2021_12/targetTrack/yolov5/models/experimental.py", line 14, in <module>
    from models.common import Conv
ModuleNotFoundError: No module named 'models'

Part 2: After changing import models.common to import common, it turns out even common cannot be found despite being in the same folder !

python3 dashboard.py 
Traceback (most recent call last):
  File "/home/aevas/Desktop/Dashboard_2021_12/dashboard.py", line 25, in <module>
    from trackerThread import trackerDeepSORT
  File "/home/aevas/Desktop/Dashboard_2021_12/trackerThread.py", line 15, in <module>
    from yolov5.models.experimental import attempt_load
  File "/home/aevas/Desktop/Dashboard_2021_12/targetTrack/yolov5/models/experimental.py", line 14, in <module>
    from common import Conv
ModuleNotFoundError: No module named 'common'

This is how the files are like in the folder models

and this is how the import portion of experimental.py looks like

# YOLOv5  by Ultralytics, GPL-3.0 license
"""
Experimental modules
"""
import math

import numpy as np
import torch
import torch.nn as nn


from common import Conv
from utils.downloads import attempt_download

I have consulted the following links, but to no avail:

1. https://towardsdatascience.com/how-to-fix-modulenotfounderror-and-importerror-248ce5b69b1c
2. Python can't find module in the same folder
3. ModuleNotFoundError: No module named 'models'
4. https://github.com/ultralytics/yolov5/issues/353

I understand that I can change it to import .common and then the module can be successfully imported. However, the next line import utils causes a similar error. utils is on the same level as models. Which means there is going to be a cascade of moduleNotFoundError errors at this rate. I also understand using the folder ()method is inadvisable, hence I did not continue with it.

As I had mentioned that I had copied the entire cloned github repo over, when executed standalone, the github repo works perfectly fine. There are no differences between the experimental.py.

What could be wrong?

Answer 1

The problem is that the models module lives in the /home/aevas/Desktop/Dashboard_2021_12/targetTrack/yolov5 folder, but when you run dashboard.py from /home/aevas/Desktop/Dashboard_2021_12/, the Python interpreter does not look inside the /home/aevas/Desktop/Dashboard_2021_12/targetTrack/yolov5 folder to find modules; it only looks as far as files/folders in /home/aevas/Desktop/Dashboard_2021_12/.

The usual way to fix this would be by installing yolov5 as a package, but it does not appear that yolov5 contains the code necessary to be installed in such a way. To work around this, you need to let Python know that it needs to look inside the yolov5 folder to find the module.

The quick-and-dirty way of doing this is to add the folder to the PYTHONPATH environment variable.

A slightly less quick-and-dirty, but still pretty ugly way of doing it is by dynamically adding the yolov5 folder to sys.path before you try and import the models module. You would do this by adding something like the following to the top of dashboard.py:

from pathlib import Path
import sys

sys.path.append(str(Path(__file__, "..", "targetTrack", "yolov5").resolve()))