Go back and forth between environments in a script (e.g., Jupyter Notebooks)

Question

I'm very confused in regards to whether it's possible to switch between environments in one script. I'm currently working Jupyter Notebooks and realized I do need a package that is in a separate (non-root) environment and would like to be able to import/use it. I initially tried to just load it into my root env, but it has so many conflicts, it's not possible. So trying to switch env mid script- is this not possible without going through hoops and going through Anaconda prompt? I've looked at about 30 articles and/or stackoverflow pages and it seems like my only option is to add my environment to the default sys.path within Jupyter OR potentially create another environment that contains both environment packages. This suggestion was unclear...not sure how to run this and I don't want to run through a shell/prompt either.

These were good resources: RealPython: On Virtual Envs StackOverflow: In which conda env is Jupyter executing?

Any clarity I can get on the topic for someone who doesn't understand this well, would be appreciated (Please explain as if I'm a Level 0/Beginner).

Answer 1

Now I'm looking back on this old post, thought I'd answer it since it never got an answer. While it's extremely frustrating to realize that needed packages don't mesh together mid-project, for each significant project, a new environment should be created. I still have a general environment that I use for minor projects, but for situations like this, I use anaconda's cheat sheet to make sure that in the new environment I'm creating, packages will all sync together by specifying versions directly in the creation in the environment. Also, getting comfortable with creating/copying .yml files and loading them directly into conda have also been helpful.

conda env create --file envname.yml

Once that's done, if I'm using jupyter notebooks, making sure it's been established as a kernel has been important.

Of course, if you're using an IDE like pycharm, you can bypass the conda prompt (pycharm is still talking to conda), which is sometimes just easier if you're on a deadline rather than going back and forth between different software.