0

I have a python project where I am using the maskrcnn-benchmark project from Facebook Research. The problem is that the setup file for the facebook project depends on pytorch i.e. the setup file has an import line like:

import torch

So, I need to have pytorch pre-installed and this is causing me some problems. For me, the cleanest solution would be if I could prebuild the maskrcnn-benchmark project as a wheel with all its dependencies like pytorch and then add this wheel as a requirement in my setup.py file.

However, I could not find an easy way to do so. Is there someway to adsd a wheel file as an install_requires step in the setup file of a python project.

Luca
  • 10,458
  • 24
  • 107
  • 234
  • As I am unfamiliar with how `torch` is used inside your `setup.py`, what you need to do is not import it directly but create a wrapper function that imports that package at the point where it will be used, and the dependency should be declared as a `setup_requires` rather than a `install_requires`. For packages that import and make use of Cython [this thread](https://stackoverflow.com/questions/37471313/setup-requires-with-cython) covers a similar issue. – metatoaster Mar 18 '19 at 08:34
  • This import is not in my setup file but in the maskrcnn-benchmark i.e. Facebook project – Luca Mar 18 '19 at 08:42

1 Answers1

2

The maskrcnn-benchmark project should have torch==1.0.1 (or whichever version) in install_requirements= (along with any other requirements).

Then, you can use 

pip wheel . --wheel-dir /tmp/deps

to have pip gather up the wheels (for your current architecture!) in /tmp/deps. Then, to install the dependencies from the wheel dir,

pip install --find-links=/tmp/deps -e .

This technique works for other target types too, like -r requirements.txt.

EDIT: If you also want to build a wheel for the project itself, that'd be python setup.py bdist_wheel, but that won't look for dependencies.

AKX
  • 152,115
  • 15
  • 115
  • 172
  • Thank you for the answer. Is it then possible to add this generated wheel in your first step for the maskrcnn-project as a requiremnt in the setup.py filee. The reason I ask is that this setup file gets picked up by another project and gets installed. So having the requirements in there would make me do these changees without disturbing the client. – Luca Mar 18 '19 at 08:17
  • These aren't generated wheels, they're wheels downloaded from PyPI. (I added an edit about how to build a wheel for the project itself.) In general, `pip` will happily download wheels for all dependencies as long as they're listed. My answer caters more for an offline/reproducible installation scenario. – AKX Mar 18 '19 at 08:20
  • yes, my issue is that for the maskrcnn-benchmark project the `setup.py` itself refers to torch i.e. there is an `import torch` line in setup.py file. So, what I was thinking was prebuilding maskrcnn as a wheel and then installing it from within my setup. So, I am guessing that cannot really be done. – Luca Mar 18 '19 at 08:25
  • Well, in that case you could have a "wrapper" project that depends on torch and maskrcnn-benchmark, so torch gets installed before maskrcnn-benchmark's setup.py is called, but it gets fiddly. Another option would be to edit that https://github.com/facebookresearch/maskrcnn-benchmark/blob/b3d1de0088ad84b7a1cdee62c08418c7b9095acc/setup.py in a way that makes the torch import optional. – AKX Mar 18 '19 at 08:44