0

I've just joined a new company to do an automation project in Python + Behave + Appium. The person before me set up the baseground of the project, amazing job, but I didn't like the way the code was structured. Every important folder was in the root dir of the project (features, steps, pages, utils, etc), so one of the first things that I did was reorganise this in what I thought was a better way.

To start with, I put all the code related to automation into a src folder and then I structured it as follows:

src
  ├── main
  |   ├── resources
  |       ├── devices.json
  |       └── project.conf
  |   ├── utils
  |   ├── device.py
  |   ├── jira_linker.py
  |   └── jira_methods.py
  ├── test
  |   ├── features
  |       ├── environment.py
  |       └── steps
  |   ├── pages
  |   └── resources
  |       ├── strings
  └──

With this structure in mind, the next step was to fix the imports that had broken with the restructure. At first I thought I'd fixed them all, as VSCode didn't complain about them and I could actually navigate to the definitions with ctrl+click.

But when I tried to run a test, to see if everything was OK, then I got a ModuleNotFoundError in environment.py in one of my imports, specifically the first one (although if I fix the first one, the next ones also fail, and so on):

from test.pages.pom_android_player import PlayerPage
from main.utils import utils as utl
from main import device as dvc, jira_methods as jira

Apparently the solution is to put "src." before the rest of the path. So, all the imports within src folder where I don't have the absolute path will fail.

In principle, this is not a big deal. It's only 4 characters more in each import of the project, and here's where I think that maybe it's a silly problem and I'm overthinking this and it isn't even a problem at all, but, on the one hand, having the src folder specified in the imports is something that I think is irrelevant. It's like the root of everything I'm gonna use, so to me, it has no real value and bothers me, and taking that into account, some import lengths are big enough to bother me even more to have to add an extra folder to them.

So I started looking for a solution (and failed every time). Probably, one of the easiest solutions would be to use the python.analysis.extraPaths in the VSCode settings. I don't know if this would work because I haven't tried it, because I'm not gonna be alone in this project in the future and some people might not use VSCode, and I might be wrong, but I think this might be a solution IF everyone in the project would use VSCode.

Beyond this, another solution might be the PYTHONPATH variable, but again, that seems like a solution only to me, and I'm looking for something that anyone who joins the team in the future could have without having to do anything, or at least, with the minimum set up. I mean, what I don't want is something that fixes the problem for me, but then, when someone new joins and clones the project, they find that none of the imports work.

Then, I tried with sys.path.insert. I thought, since I have the environment.py and I have the before_all method, which is the first thing Behave calls every time I run a test, maybe I could insert the src path to PYTHONPATH from here and that'd be it. Wrong. Apparently I did not realise that the imports come first.

So, the last thing I tried, just to see if at least the sys.path.insert solution would work, was to put these statements before the failed imports.

import os
import sys
import allure
import base64
from jira import Issue
from behave.model import Status
from appium.webdriver import Remote

# Insert src directory to the path variable
ROOT_DIR = os.path.dirname(os.path.abspath(__file__)) # This is your Project Root
sys.path.insert(0, ROOT_DIR + "\src")
print(sys.path)

from test.pages.pom_android_player import PlayerPage
from main.utils import utils as utl
from main import device as dvc, jira_methods as jira

Obviously, I don't think it's a good practice at all to do something like that, but it was just to test the solution.... And it didn't work anyway. It does add the src path to sys.path, but it still fails anyway. And this is where I got lost.

I have to say that I'm still a newbie to Python. I have enough knowledge of the language and the previous knowledge of other languages, such as Java, to do the automation coding right, but beyond that, I'm still learning. So maybe some of the things I've tried could be a solution, but I haven't implemented them well, or maybe there is an obvious solution that I can't see due the lack of knowledge of the language.

In this matter, I don't know if for example having a __init__.py in src folder might be a solution for this, but as I said, I don't have enough knowledge of Python yet to know how to do it if it's a solution.

What I'm looking for, and I'm not sure if it's possible, is to find a solution where I can import everything within src folder without actually having to add src to the import path, and this solution has to meet the requirement that it is suitable for everyone working on the project, not just me. FINAL NOTE: The project runs by executing a run.sh which is in the root directory of the project. I also thought that there might be something that could be done inside this .sh file, but I didn't find anything to do that.

Thanks for the help.

EDIT: Apparently I made a mistake when I used the sys.path.insert() statement. I'm guessing that when I write this in environment.py, the path I get in ROOT_DIR = os.path.dirname(os.path.abspath(__file__)) line is the directory where environment.py is, not the root project directory. If I write this instead ROOT_DIR = os.path.dirname(os.path.dirname(os.path.dirname((os.path.abspath(__file__))))) then I get the root dir + /src. This means, ROOT_DIR is the route to src dir, which is kind of what I wanted. However, even with the src folder in my paths when I print sys.path, I get the same ModuleNotFoundError. No module named test.pages in the import from test.pages.pom_android_player import PlayerPage

carlospj87
  • 1
  • 2
  • Without commenting on what would be the "correct" way to handle it, just this: "Apparently I did not realise that the imports come first." The imports come first because you wrote them first. You can import `os` and `sys`, do the `sys.path.insert`, then continue with the rest of your imports. Also, regarding `"\src"`, please see [this](https://stackoverflow.com/questions/16010992/how-to-use-directory-separator-in-both-linux-and-windows-in-python). – Amadan Jul 11 '23 at 18:40
  • Yes, I said I did it to test if it would work. But it didn't anyway. Maybe it is because of what you mention about the link you have shared. I will try to apply what is in that answer. Let's see, although I'm not very confident about it. Having said that, imagine that it works. Putting the ```sys.path.insert``` before the rest of the imports, I understand it would be a bad practice, wouldn't it? – carlospj87 Jul 12 '23 at 07:45
  • It is not the preferred way — I believe a lot of people would recommend against the `src` directory, and use a meaningful package name instead of `main`. See [What is the best project structure for a Python application?](https://stackoverflow.com/questions/193161/what-is-the-best-project-structure-for-a-python-application). Then again, if you _do_ use it... the `sys.path.import` trick is explicitly suggested in the book [The Hitch-Hiker's Guide to Python](https://docs.python-guide.org/writing/structure/#test-suite). – Amadan Jul 12 '23 at 09:28
  • Interesting. But I'm not clear about one thing that is said in the answer to the link about ```src```. At first I understand that what they are proposing is to directly remove ```src```, right? And in my case, ```main``` (or whatever name I give it) and ```test``` would be at the project directory level. If this is the case, it would probably solve the problem directly. If what they are saying is to replace ```src``` with another directory, I think I would be in the same situation. By the way, I've added an edit to the main post because of a bug I saw I was making. – carlospj87 Jul 12 '23 at 11:53
  • The former. Having `main` (or `floozapper`, or whatever your application is called) at top level makes sense. If you inherited the project where `util` was at top level, it makes no sense, as it looks like a support for the `floozapper` app. If `jira_*.py` were at top level, that too makes no sense - if they are independent, `jira_support/*.py` or something makes sense, or even a separate project entirely that is imported by `requirements.txt`; if not, `floozapper/jira/*.py`. Even without `src`-like directory, there is not much reason for the top level to be crowded. – Amadan Jul 12 '23 at 14:36
  • Great! The name ```floozapper``` is made up, isn't it? hahaha One thing I should clarify here is that the project is not about developing an app, but about QA automation of the app. This means that in ```main``` there is no app development, and that the main code where the tests with Behave are is in the ```test``` dir. That's where all the important stuff happens in my project. In ```main``` there is only code for support, or configuration of the environment and test devices, hence the utils dir and the jira files. – carlospj87 Jul 13 '23 at 09:56
  • And yes, all of that and more were at top level when I inherited the project. All jira stuff were inside ```utils``` dir. And everything that is right now inside ```test``` were at top level too – carlospj87 Jul 13 '23 at 09:58

0 Answers0