32

I use a pip requirements file to maintain a list of dependencies for my projects.

I now find myself having to install a library using pip install --install-option='some-setup.py.option' but pip freeze doesn't record these options in its output which I save in my project's requirements.txt. This causes problems because a simple pip install -r requirements.txt on a new machine installs all the dependencies without supplying the required arguments for this one library, and I've lost the simple round-trip operation.

So, my 2 part question is:

  • Is there a way to maintain pip install options in the pip freeze output somehow?
  • If not, is there a way to manually hack the requirements file to add the install option? I'm ok with losing the round trip nature of pip freeze if I have to, and switching to manual maintenance of the requirements file. I've checked the documentation but couldn't see anything to help.

Unnecessary but possibly interesting details follow

I want to install pymongo but without building the C extension so I can use it asynchronously in an eventlet based app.

Install as desired and build requirements.txt:

(test)day@office:~/test$ pip install pymongo --install-option='--no_ext'
Downloading/unpacking pymongo
  Downloading pymongo-2.1.1.tar.gz (199Kb): 199Kb downloaded
  Running setup.py egg_info for package pymongo
Installing collected packages: pymongo
  Running setup.py install for pymongo
Successfully installed pymongo
Cleaning up...
(test)day@office:~/test$ pip freeze >  requirements.txt
(test)day@office:~/test$ cat requirements.txt
bottle==0.10.7
distribute==0.6.10
eventlet==0.9.16
greenlet==0.3.3
lxml==2.3.3
pymongo==2.1.1
simplejson==2.3.2
wsgiref==0.1.2

In new virtualenv, try to install same project from requirements.txt. pip builds the C extension for pymongo :(

(test2)day@office:~/test2$ pip install -r requirements.txt 
...
Downloading/unpacking pymongo==2.1.1 (from -r requirements.txt (line 6))
  Downloading pymongo-2.1.1.tar.gz (199Kb): 199Kb downloaded
  Running setup.py egg_info for package pymongo
Installing collected packages: pymongo
  Running setup.py install for pymongo
    building 'bson._cbson' extension
    gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -Ibson -I/usr/include/python2.6 -c bson/_cbsonmodule.c -o build/temp.linux-i686-2.6/bson/_cbsonmodule.o
...
Successfully installed pymongo
Cleaning up...

Update Issue 271 was opened against pip in April 2011 asking for the ability to specify per-line --install-option in requirements.txt. Please vote for the issue if you have the same problem.

Day
  • 9,465
  • 6
  • 57
  • 93
  • if you manually add --install-option='--no_ext' to the requirements file, does it work? - if not I suspect you'll need to request that the pip developers add this functionality (which is a great idea!) – lofidevops Feb 06 '12 at 12:10
  • 3
    @d3vid That doesn't work I'm afraid. There's already a feature request for this, opened in April 2011: https://github.com/pypa/pip/issues/271. Please add your +1 there too. – Day Feb 07 '12 at 00:01

2 Answers2

34

Since version 7.0 (released 2015-05-21), pip has the ability to parse --install-option and --global-option from requirement files. It should now be possible to have the following line in your requirements.txt:

pymongo==2.1.1 --install-option='--no_ext'

More information can be found here and here.

Day
  • 9,465
  • 6
  • 57
  • 93
gvalkov
  • 3,977
  • 30
  • 31
  • 1
    Plz fix first link to https://github.com/pypa/pip/blob/master/docs/reference/pip_install.rst#id40 – y0prst Jul 07 '16 at 09:01
  • What version of pip is this? – hlin117 Apr 16 '17 at 07:49
  • @hlin117 looks like version 7.0 onwards according to [the first link](https://github.com/pypa/pip/blob/master/docs/reference/pip_install.rst#id40) – Day Jul 14 '17 at 17:16
  • 2
    Using options in requirements.txt disables wheels for all modules. Thus means compiling evrything from scratch with dev dependencies. – MortenB Jan 20 '21 at 22:04
  • I tried this but it didn't work, might be outdated? – Pellet Aug 03 '23 at 09:23
7

This might probably be too naïve, but if you're fine with managing the requirements and corresponding options manually -- why not maintain them as a small shell script that includes the entire pip incantation? Just a work-around until the feature request is listened to :)

fgeller
  • 491
  • 3
  • 8
  • I'm having the same issue, and I feel a bit stupid for not having come up with this myself. Thank you for posting this suggestion! – kungphu Feb 01 '13 at 15:36
  • breaks the `requirements.txt` convention, which sucks because it breaks stuff downstream like, for example http://docs.saltstack.com/en/latest/ref/states/all/salt.states.pip_state.html#salt.states.pip_state.installed. But yeah, that's apparently what's gonna be necessary here. – Andrew Mar 04 '15 at 19:18