1

I am using Jupyterhub and I hope to get students on to it.

My problem is I have setup a conda environment which has about 10GB of default packages to make use of GPU server. If a user wants to add a package not installed I (as the admin) do not want to install it in the main Conda enviroment. I thought clone would work but it seem to just copy the whole lot?

For example on a (smaller example) system env:

$ conda create --prefix /opt/conda/envs/python310 python=3.10 ipykernel matplotlib numpy
$ du -hs /opt/conda/envs/python310/
2.0G    /opt/conda/envs/python310/

So as a normal user I did 
$ conda create --name MyEnv --clone python310
$ du .conda/envs/MyEnv
1.99G   .conda/envs/MyEnv
$ conda install scipy
$ du -hs .conda/envs/MyEnv
2.12G   .conda/envs/MyEnv

It is the same size, so with a 10G system environment I would soon run out of storage if every user did it.

I thought I read clone just copied links to original? Is there a way I can get users to add a package to there own system so when they ran a jupypter notebook everything would work?

Thanks

Andrew Watkins
  • 11
  • 2
  • `PYTHONPATH` can be used to specify a directory of `site-packages` to side-load; also, [all user site packages](https://stackoverflow.com/a/70961159/570918) are loaded by default (and prioritized). Otherwise, one would need to use a common `pkgs_dirs` (package cache) for all users to source Conda packages. Note that [Conda's hardlinking](https://stackoverflow.com/a/55602940/570918) only works when the environment is colocated on the same volume. – merv Nov 18 '22 at 18:12

0 Answers0