4

When starting a project in Python, I want to save some environment variables in a file called environment_variables and source this file in the bashrc.

The file looks something like this: 

username=$(whoami)

# project root path
export PROJECT_DIR='/home/'$username'/nuclei_segmentation/'

# project data path
export DATA_DIR=$PROJECT_DIR"data/"

# location of models 
export MODEL_DIR=$PROJECT_DIR"models/"

# project output data
export OUTPUT_DIR=$PROJECT_DIR"output/"

I would like to change the PROJECT_DIR path so it is platform/name independent. So this environment_variables file will always be in the root directory of the project and I want to set PROJECT_DIR to always be the location of the environment_variables file.

I thought I could do this with PWD but when called from bashrc this creates an error, I also thought of a solution using find to search for the file from the root directory but this seems to complex and think there must be a better way?

William Grimes
  • 675
  • 2
  • 11
  • 29
  • 1
    This sounds very much like an [XY Problem](https://meta.stackexchange.com/questions/66377/what-is-the-xy-problem). *Why* do you want to do these things? Yours is perhaps not a good approach to the underlying problem you are attempting to solve. – tripleee Feb 11 '18 at 15:41
  • Perhaps you are correct! Maybe the X part of my project is, what is a good way to setup environment variables in a python data science project, that is reproducible and platform independent? – William Grimes Feb 11 '18 at 16:14
  • Honestly, I would just edit the file when I first copy it to the project directory to hard-code the value of `PROJECT_DIR`. – chepner Feb 11 '18 at 16:34
  • yes, it works, but I thought there must be an elegant solution for this – William Grimes Feb 11 '18 at 16:40
  • 1
    What do you mean by *"platform/name independent"?* Should this be portable to POSIX platforms, or e.g. Windows too? What exactly is the scenario where `$PWD` doesn't work, and what's the error you get? – tripleee Feb 11 '18 at 18:26
  • @triplee what I meant was that if I give this project to someone else and they put it at a different location (i.e. their path is different) the variables will still be created. So for me it is at `/home/username/project` but maybe theirs is `/mnt/data/all_projects/project` for example. I'm trying to find a way without hard coding the location. – William Grimes Feb 11 '18 at 18:34
  • If I use `$PWD` then I get the location of the .bashrc file, e.g. my home directory, rather than the path to the `environment_variables` file. – William Grimes Feb 11 '18 at 18:36
  • 1
    @WilliamGrimes, if you want to find the path to a source file, that's what `$BASH_SOURCE` is for. We have *lots* of preexisting questions that cover that ground; ie. [getting the source directory of a bash script from within](https://stackoverflow.com/questions/59895/getting-the-source-directory-of-a-bash-script-from-within). Also see [BashFAQ #28](https://mywiki.wooledge.org/BashFAQ/028). – Charles Duffy Feb 12 '18 at 05:03
  • @CharlesDuffy I don't think this is a duplicate, the answer which is similar is downvoted and apparently not at all what the OP wants. – tripleee Feb 12 '18 at 06:18
  • @tripleee, ...if you wanted to edit this question such that it's no longer titled as being principally about bash but is explicitly asking a question about best practices in helping a Python program find its install location (or whatever), I'd agree that it would no longer be duplicate. That said, I've reread the question and still don't see how the OP is "explicitly rejecting" `$BASH_SOURCE` -- they're explicitly rejecting `$PWD`, which is correct, because if you want the source file's location, the shell's current working directory isn't going to give a correct result. – Charles Duffy Feb 12 '18 at 14:37
  • @tripleee, ...as it is, though, the question *asked in the title* is a duplicate, and if what they want to do is set environment variables relative to the location of a `environment_variables` file that's sourced from their dotfiles on login (a reading of the body text that coincides with the plain language of that title), that's clearly a duplicate as well. – Charles Duffy Feb 12 '18 at 14:39
  • @CharlesDuffy Not sure I understand the OP's intentions exactly either, and not sure it's worth the effort. Can you link to the other duplicate you alluded to, though? – tripleee Feb 12 '18 at 14:51
  • @tripleee, I didn't mean to refer to any duplicate other than the currently-linked one; yes, its accepted answer is calling `pwd`, but it does so in a subshell after `cd "$(dirname "${BASH_SOURCE[0]}")"`, making the output different from the case where the OP indicates that `$PWD` doesn't work for them. – Charles Duffy Feb 12 '18 at 15:40
  • @tripleee and @CharlesDuffy thank you both very much for your input. In the end I used `NUC_SEG_DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )"` as suggested in `environment_variables` which works a treat. Then in python I create variables relative to that. I apologise for asking in a somewhat convoluted way, but I suppose as well as the specifics of the question I was also hoping for greater context on best practice which @tripleee kindly provided. Many thanks – William Grimes Feb 12 '18 at 16:38

2 Answers2

2

If I am understanding at all where you want to go with this, my recommendation would be

  • Make the code work out of the current directory as far as possible.
  • Don't litter the user's environment with multiple variables if it can be avoided.
  • If you have to have environment variables, give them a name which clearly communicates what they are related to.

In some more concretion, a single variable which names the project root should be all you really need. Don't require it to be set if the expected files are in the current directory.

export NUC_SEG_DIR=$HOME/nuclei_segmentation

In your Python code, put in reasonable defaults. In this particular case, expect os.path.join(os.environ['NUC_SEG_DIR'], 'data') to point to your data directory, etc. If users want to override this, that's easy enough with symbolic links. Make sure this is parametrized in the code so that it's easy to override in a single place if you should want to make this configurable in the future. Maybe something like

def nuc_seg(root_dir=os.environ['NUC_SEG_DIR'], data_dir='./data', model_dir='./models', output_dir='./output'):
    if root_dir == '':
        root_dir='.'
    ... your code here

and perhaps later on make your command-line interface let the user override these values by way of command-line options or a configuration file.

Again, don't require the user to edit their .bash_profile or similar - what if they want to run multiple experiments in different directories, or whatever? They can figure out how to make the environment variable permanent, or you could even document this as an option if your users can't be expected to be familiar with the basics of the shell.

tripleee
  • 175,061
  • 34
  • 275
  • 318
-1

Use dirname $0 inside script file to find the directory where current script file resides

smilingwang
  • 119
  • 6
  • for example `export PROJECT_DIR=$(dirname $0)` – William Grimes Feb 11 '18 at 16:58
  • not sure exactly of syntax? – William Grimes Feb 11 '18 at 16:58
  • 1
    Use quotes to cope with irregular filenames; `PROJECT_DIR="$(dirname "$0")"` ... Not sure if the `export` is really necessary; many beginners needlessly use it for variables which are not going to be exported to subprocesses. – tripleee Feb 11 '18 at 17:05
  • thanks @tripleee I've just tried that and get the following error : `dirname: invalid option -- 'b'` – William Grimes Feb 11 '18 at 17:39
  • 2
    Then your `$0` is `-bash` (or `-sh`) which means you are sourcing the script. This answer is specifically about finding the source of the currently running script, but if you are not running a script, then of course that is moot. – tripleee Feb 11 '18 at 18:22
  • Oh sorry for the confusion, yes I'm sourcing the script in bashrc. Could I make the `environment_variables` into a bash script and run it then, if i just put in `#!/usr/bin/env bash` – William Grimes Feb 11 '18 at 18:38