20

Some background: The project I'm working on uses python-ldap library. Since we are a mixed-OS development team (some use Linux, some macOS and some Windows), I'm trying to make the project build on all environments. Unfortunately, python-ldap is not officially supported for Windows, but there are unofficial wheels maintained by Christoph Gohlke. I've tested the wheel file and it works fine.

The problem: how do I tell Poetry to use the wheel on Windows and the official python-ldap package on Linux and macOS?

I've tried multiple things, including using multiple constraint dependencies and markers:

python-ldap = [
    { markers = "sys_platform == 'linux'", version = "*" },
    { markers = "sys_platform == 'win32'", path="lib/python_ldap-3.2.0-cp36-cp36m-win_amd64.whl" }

... but, judging from the poetry.lock file, it seems markers are then merged and just determine whether the library should be installed at all:

[[package]]
category = "main"
description = "Python modules for implementing LDAP clients"
marker = "sys_platform == \"linux\" or sys_platform == \"win32\""
name = "python-ldap"

Is there another way of dealing with platform-specific dependencies in Poetry?

Igor Brejc
  • 18,714
  • 13
  • 76
  • 95
  • I have the same problem. Currently I'm using commented out lines of dependencies, but it's far from ideal. Please tell me if you found a solution. Thanks! – András Geiszl Apr 17 '20 at 12:32

3 Answers3

9

You can use the platform keyword

python-ldap = [
    { version = '*', platform = 'linux' },
]

Source

Jacob Pavlock
  • 673
  • 8
  • 20
  • 1
    Even better, such as for `faiss-cpu` vs `faiss-gpu` - let the CLI do it for you. `poetry add faiss-gpu --platform linux` `poetry add faiss-cpu --platform darwin` – rjurney Dec 21 '21 at 19:19
  • 1
    I don't understand how this answers the question. This only lists `linux`. As far as I can tell, there's no `windows` platform. – 404usernamenotfound Jul 14 '23 at 13:53
8

The best way to do this is the use the --platform option with the poetry add command. For installing faiss on both Mac (faiss-cpu with no CUDA GPU support) and Linux (faiss-gpu with GPU/CUDA support available) you run the following commands:

# Add each package to your project
poetry add faiss-gpu --platform linux
poetry add faiss-cpu --platform darwin

# Thereafter just install
poetry install

As mentioned above, you can do this in the pyproject.toml file as described in the other answer, but the CLI is best. Be sure to poetry update if you edit pyproject.toml directly:

[tool.poetry.dependencies]
faiss-cpu = {version = "^1.7.1", platform = "darwin"}
faiss-gpu = {version = "^1.7.1", platform = "linux"}

Holla!

rjurney
  • 4,824
  • 5
  • 41
  • 62
  • Why is using the CLI better than editing pyproject.toml directly? – burnpanck Aug 25 '22 at 05:58
  • @burnpanck The arguments are that it prevents you from making a mistake, and that it updates the lock file and virtualenv automatically, but it's perfectly common to edit it manually, as well. Whatever works best for you. – mikenerone Sep 06 '22 at 16:08
  • This does answer the questions, unfortunately. The package names are different in the answer, but the same in the original question. – bayer Feb 08 '23 at 12:22
  • @bayer I'm not clear on what I should do to improve it - can you edit this or explain? Oh, the `faiss` stuff. Yeah, that came from my code. It seemed a clearer answer? I can add his code. – rjurney Apr 03 '23 at 22:09
  • "The problem: how do I tell Poetry to use the wheel on Windows and the official python-ldap package on Linux and macOS?" No current answer shows this exact scenario. – 404usernamenotfound Jul 14 '23 at 13:55
  • This does not answer the question because the package names are different in your example. How would you express this platform dependency with packages having the same name? – Louis Lac Aug 17 '23 at 20:02
  • I believe the same thing works? – rjurney Aug 18 '23 at 20:44
1

It looks like Poetry does not support well platform-specific dependency alternatives when one of the constrain contains a version tag unfortunately. For your solution to work, you would have to specify a path or url tag to a local or distant wheel instead.

I had the same issue with PyTorch dependencies and I was able to solve it by changing this:

[tool.poetry.dependencies]
python = "^3.10"
torch = [
    {platform = "linux", url = "https://download.pytorch.org/whl/cu118/torch-2.0.1%2Bcu118-cp310-cp310-linux_x86_64.whl"},
    {platform = "darwin", version = "2.0.1"},
]

to this:

[tool.poetry.dependencies]
python = "^3.10"
torch = [
    {platform = "darwin", url = "https://download.pytorch.org/whl/cpu/torch-2.0.1-cp310-none-macosx_11_0_arm64.whl"},
    {platform = "linux", url = "https://download.pytorch.org/whl/cu118/torch-2.0.1%2Bcu118-cp310-cp310-linux_x86_64.whl"},
]

Notice the Linux specification which now links directly to the wheel file.

Louis Lac
  • 5,298
  • 1
  • 21
  • 36