0

I use Git VC for my Python codes, mostly scripts with scientific calculations. To parse raw data (which are also part of the repository) into Python variables, I do:

import numpy as np
t_x_q_obs = np.genfromtxt('MeasuredAlgebrProductionRate_30min_18h.csv', delimiter=';')

Meanwhile, the repo has grown to contain > 100 files. So I would like to group certain kind of files, especially *.csv files with raw data, in a subfolder. No problem with Git, but Python would not find them anymore. In Matlab, I would just tell people to add the whole folder+subfolders to their pathdef.m, but since Python programmers use a lot of different editors/IDEs, there is no universal how-to. I would like my scripts to be executable OOTB by anyone cloning my repo and running the scripts from the command line or his/her favorite IDE. If I added those subfolders to the PYTHONPATH variable, this would not be universal either. Also, I would not like to hard-code the relative paths in all scripts.

Is there an elegant way to have Python look for external files which are part of the same folder structure?

winkmal
  • 622
  • 1
  • 6
  • 16

1 Answers1

1

You could improve your code by using paths relative to your main python file like this:

import os
base_path = os.path.dirname(os.path.abspath(__file__))
my_file = os.path.join(base_path, 'my_data_file.csv')
# now my_file is referring to a path relative to your python script
André Müller
  • 107
  • 7
  • While this would work, it would coerce me to rewrite all parse commands in all of my scripts! I was hoping to avoid that, if possible. If not, I will probably go down your route. – winkmal Sep 20 '19 at 12:14
  • 1
    What if you would provide a bash script in your repository which sets the Python working path relative to the checkout position of the git repo before calling? Alternatively using `os.chdir(path)` might be a possibility for setting the current working director. Sure - if you have a bunch of absolute paths in your code... then you really need to change that! – André Müller Sep 20 '19 at 12:31
  • Well, I don't have "absolute paths" - I just don't have pathnames at all, just filenames! And in Matlab, this works even when the files are located in subfolders, due to the `pathdef.m`. (Sometimes, I check their existence by `if exist(parametersFilename, 'file')`.) Also, wouldn't "bash script" imply that people used Linux/Mac OS? This will probably not work for the average Windows user. – winkmal Sep 20 '19 at 15:52