0

I want to install a python3 package (in this specific case is torch and torchvision) on a cluster that is not connected to the external world. I have tried to do pip3 download <package> and then I move the file to the cluster and run pip3 install <downloaded-file> -t /custom/folder (because I am installing in one of the folders in my personal account, and not on the entire cluster).

When I run the install command, pip3 is trying to download numpy package -- for some reason (it is actually already installed and working on ipython3, I do not know why it is trying to download that).

Of course, the installation fails because there is no connection to the external world. Can I force python to download everything with any flags?

Moreover, I would like to be sure I am downloading the proper thing: the location from which I am downloading the file is different from the location where I want to install, the first it is a regular CPU and the second runs GPU and has CUDA installed.

martineau
  • 119,623
  • 25
  • 170
  • 301
simona
  • 2,009
  • 6
  • 29
  • 41
  • The numpy thing is most likely due to version clush i.e. the installed one is different from the requirements of the package. Can you elaborate a bit more on the gpu/cpu issue? – nickyfot Jan 15 '20 at 15:54
  • it is just that the cluster where I aim to install has GPU, and the cluster that has access to internet has not, so I was wondering if I am going to download the version that I effectively need – simona Jan 15 '20 at 15:54
  • You can check the version of numpy installed with `pip freeze` and then compare the version against the package requirements – nickyfot Jan 15 '20 at 15:55
  • Probably not an answer, but... Do you know about `pip install --user ...` (instead of `-t`)? – sinoroc Jan 15 '20 at 15:58
  • According to the [documentation](https://pip.pypa.io/en/stable/reference/pip_download/), the lack of GPU in the first environment should not be an issue for the installation on the second as long as the version requirement is correctly defined. I have not actually tried it for the package you need, but it should work – nickyfot Jan 15 '20 at 15:58
  • Especially this answer: https://stackoverflow.com/a/14447068/7976758. The most essential part of it is **--no-index**: `pip install --no-index --find-links /path/to/download/dir/ -r requirements.txt` – phd Jan 15 '20 at 16:24

1 Answers1

1

Formal answer here. Pip is attempting to install a different version of numpy specified in your package's requirements. You can easily compare the version installed with pip freeze and the version it is attempting to download (before it fails).

To make the download and installation fool-proof you probably should create a requirements.txt file and then run pip download -r requirements.txt in the environment that is connected to the internet and then transfer to the non-connected environment and carry on with the installation.

Regarding the GPU/CPU difference between the packages, it is hard to tell without actually knowing which package this is, but if it's similar to tensorflow where you have completely different package names when using pip the lack of GPU should not affect the download process. See pip download documentation for more information.

nickyfot
  • 1,932
  • 17
  • 25