128

I have a python application that I wrote to be compatible with both, Linux and Windows platforms. However there is one problem... One of the python packages I need for Windows is not compatible with Linux. Fortunately there is another package that provides the same functionality on Linux. All other dependencies are compatible in both platforms.

I know I could have 2 separate requirement files to address both platform dependencies separately. Something like win_requirements.txt and linux_requirements.txt, however this approach doesn't feel like the best way to do it.

I wonder if there is a way I can have only one requirements.txt file so any user can use pip install -r requirements.txt to install all the dependencies regardless of what platform they are?

Maybe something like??:

SOAPpy>=0.12.22
pycrypto>=2.6.1
suds>=0.4
Python-ldap>=2.4.19
paramiko>=1.15.2
nose>=1.3.4
selenium>=2.44.0
bottle>=0.12.8
CherryPy>=3.6.0
pika>=0.9.14
if platform.system() == 'Linux':
    wmi-client-wrapper>=0.0.12
else if platform.system() == 'Windows':
    WMI>=1.4.9
Cas
  • 2,077
  • 3
  • 21
  • 24
  • 17
    Strongly recommend looking at Tony G's answer instead of the accepted answer: http://stackoverflow.com/a/35614580/872328 – PaulMest Aug 17 '16 at 20:55

5 Answers5

311

You can add certain conditional requirements after a semi-colon. Particularly useful for sys_platform and python_version.

Examples:

atomac==1.1.0; sys_platform == 'darwin'
futures>=3.0.5; python_version < '3.0'
futures>=3.0.5; python_version == '2.6' or python_version=='2.7'

Apparently you can also exclude particular versions of a library:

futures>=3.0,!=3.0.5

They are defined in PEP 508 and PEP 0345 (Environment Markers) but the syntax appears to follow the draft PEP 0496.

Tony G
  • 3,210
  • 2
  • 9
  • 5
  • 1
    is it possible to query for pypy too? – sebix Aug 02 '17 at 15:47
  • 3
    Actually, I believe its PEP 508. – Davoud Taghawi-Nejad Sep 04 '17 at 19:54
  • @ sebix yes have a look at the PEP. – Davoud Taghawi-Nejad Sep 04 '17 at 19:54
  • 1
    Does not seem to work with hash pinning: if I have the line `enum34==1.1.6 --hash=sha256:644837f692e5f550741432dd3f223bbb9852018674981b1664e5dc339387588a; python_version < '3.4'` I get `Expected sha256 644837f692e5f550741432dd3f223bbb9852018674981b1664e5dc339387588a;`, `Got 644837f692e5f550741432dd3f223bbb9852018674981b1664e5dc339387588a`, if I add a space before the `;`, the version condition is just ignored. – Simone Bronzini May 22 '18 at 09:10
  • Would there be a way to check if a user uses ssh or https to install packages from git? And depending on that provide a different link to install a package from a private git repo? Or do something like try except? – Zuabi Dec 14 '18 at 08:30
  • 1
    @SimoneBronzini try : enum34==1.1.6; python_version < '3.4' --hash=sha256:644837f692e5f550741432dd3f223bbb9852018674981b1664e5dc339387588a – anno Nov 26 '19 at 10:53
  • For PyPy: `platform_python_implementation == "PyPy"` – Hugo Dec 31 '20 at 11:53
21

You could create an install.py script and call pip by script.

import pip

_all_ = [
    "SOAPpy>=0.12.22",
    "pycrypto>=2.6.1",
    "suds>=0.4",
    "Python-ldap>=2.4.19",
    "paramiko>=1.15.2",
    "nose>=1.3.4",
    "selenium>=2.44.0",
    "bottle>=0.12.8",
    "CherryPy>=3.6.0",
    "pika>=0.9.14",
]

windows = ["wmi-client-wrapper>=0.0.12",]

linux = ["WMI>=1.4.9",]

darwin = []

def install(packages):
    for package in packages:
        pip.main(['install', package])

if __name__ == '__main__':

    from sys import platform

    install(_all_) 
    if platform == 'windows':
        install(windows)
    if platform.startswith('linux'):
        install(linux)
    if platform == 'darwin': # MacOS
        install(darwin)

Another way to resolve this issue using only requirements files should be using inheritance of requirements

requirements.txt

SOAPpy>=0.12.22
pycrypto>=2.6.1
suds>=0.4
Python-ldap>=2.4.19
paramiko>=1.15.2
nose>=1.3.4
selenium>=2.44.0
bottle>=0.12.8
CherryPy>=3.6.0

windows.txt

-r requirements.txt
WMI>=1.4.9

linux.txt

-r requirements.txt
WMI>=1.4.9

Then you can call just the requirement equivalent to platform.

pip install -r windows.txt
pip install -r linux.txt
Mauro Baraldi
  • 6,346
  • 2
  • 32
  • 43
  • 3
    In additon, one may want to keep the install.py cleaner by replacing the all block with a base_requirements.txt file and using -r base_requirements.txt at the top of the different requirements files. – Sven Oct 02 '17 at 15:24
  • pip.main() is deprecated now. See https://stackoverflow.com/a/50255019/7234896 – R. Yang Mar 10 '20 at 04:27
  • if you have '-r requirements.txt', and requirements has 'WMI>=1.4.9', while windows.txt has an earlier version 'WMI>=1.4.8', will it down grade the version? – mike01010 May 06 '20 at 16:27
  • 1
    @mike01010 take a look [here](https://stackoverflow.com/a/33812968/841339) – Mauro Baraldi May 06 '20 at 17:36
11

You can add additional requirements to any package after a semicolon. You may limit any package with multi-condition by and, or. more conditions: https://www.python.org/dev/peps/pep-0508/#environment-markers

examples:

futures>=3.0.5; python_version < '3.0'
futures>=3.0.5; python_version == '2.6' or python_version=='2.7'
futures>3 ; python_version >= "3.6" and sys_platform == "linux"
futures>3.3 ; python_version >= "3.6" and sys_platform == "darwin"
gshmu
  • 111
  • 1
  • 4
8

Use this in the requirements.txt file

uwsgi==2.0.18; platform_system != "Windows"

in this case pip will install uwsgi if not running on Windows

matirials
  • 145
  • 1
  • 2
  • 8
0

A Complimentary questions though, What if none of the default markers is suitable for our usecase. For example, if I want to install a package when it is run in CI vs when it runs on my machine , Can I set an environment variable and decideds based on its value ?

Muhammad Faizan-Ul-Haq
  • 330
  • 4
  • 13