I am looking for best practices for maintaining different python installations (or environments) simultaneously, specifically, with regards to achieving the following 3 objectives/requirements:
Note: I am using Anaconda installation for Python
Requirements
Requirement1 - Jupyter Notebook convenience
All python interpreters (either in different installations or environments) should be available in jupyter notebook in the list of kernels.
Requirement2 - Command line convenience
On the command line it should be easy to specify any python interpreter available on my machine to run a py script, something like: 'py2 test.py' or 'py3_env1 test.py'
Requirement3 - Automation convenience
Run the py scripts using the *.bat files, using any python interpreter available on the machine.
Choices
As implicit above, there are 2 broad choices here:
Choice1 - Single installation, multiple environments
...i.e. maintain a 'single' python installation folder but create multiple environments in it using 'conda'.
Choice2 - Mulitple installations, single environment each
... i.e. maintain different python interpreters in separate installation folders with only the root environment.
If I make any any of the two Choices above, I can satisfy some, but not all 3 requirements.
For instance, if I make Choice1, I can satisfy Requirement1 easily, but not 2 and 3; and with Choice2, I can satisfy Requirement2 and Requirement3 only. Details as below.
Satisfying requirements
Satisfying Requirement1 - Jupyter Notebook convenience
If I make Choice1 (single installation, multiple environments), I can satisfy Requirement1 using the solutions posted here. It suggests registering ipykernels in all of the environments.
With Choice2, is it possible for jupyter notebook to see all kernels spread across many python installations? I couldnt make it work.
Satisfing Requirement2 - Command Line convenience
If I make Choice2, then I can satisfy Requirement2 by using Python Launcher or creating symlinks as suggested here
With Choice1, it doesn't work since Python Launcher doesn't recognise different environments (yet!). So, is there any other way to make it possible? Can I, lets say, use some kind of command alias, something like a short-cut command 'py' that refers to the 'root environment' python.exe file, and another one 'py2' that refers to another environment's python.exe. And if I do that, will calling 'py2' be robust in the sense that it will call correct site-packages folder? (Note, we haven't activated the environment here, so the windows 'PATH' may not point to the correct folders). I would assume that activating the environment is necessary, but I am not sure.
Satisfing Requirement3 - Automation Convenience
If I make Choice2, I can probably satisfy Requirement3 using Python Launcher as well (just like above). Also, not to mention, my script can also use shebang operator to mention which python it wants to invoke. However, I don't want to hardcode the python version in the script and keep the flexibility in the command itself.
If I make Choice1, I thought that a *.bat file like below will work: conda activate py3_env1 python test.py deactivate
So, I intend to activate the environment first and then call the py file. However, this doesn't work. Calling this bat file executes just the first line, activates the environment and then nothing happens.
So, in short, I am looking to have a consistent setup that allows me to invoke different python versions from all mediums - notebook, command prompt and automated bat files - very conveniently.
Any guidance or recommendation around best practices are very appreciated.
