Generate aligned requirements.txt and dev-requirements.txt with pip-compile

Question

I have a Python project that depends on two packages moduleA and moduleB. I have the following pyproject.toml:

[project]
name = "meta-motor"
version = "3.1.0.dev"
dependencies = [
    "moduleA==1.0.0"
]

[project.optional-dependencies]
dev = [
    "moduleB==1.0.0"
]

with moduleA depending on moduleC>=1.0.0 and moduleB depending on moduleC==1.1.0.

I compile my requirements.txt and dev-requirements.txt like this:

$ pip-compile -o requirements.txt pyproject.toml 
$ pip-compile --extra dev -o dev-requirements.txt pyproject.toml

With this, I get

requirements.txt

moduleA==1.0.0
    # via pyproject.toml
moduleC==1.2.0
    # via moduleA

dev-requirements.txt

moduleB==1.0.0
    # via pyproject.toml
moduleA==1.0.0
    # via pyproject.toml
moduleC==1.1.0
    # via 
    #   moduleB
    #   moduleA

As you can see, the moduleC version is different in both requirements.txt files. How can I solve this so I have moduleC==1.1.0 in both ?

I could specify moduleC==1.1.0 in my pyproject.toml, but this is not practicable for larger project with lots of dependencies like this.

Maybe generate `dev-requirements.txt` first and then use it as "constraints" when generating `requirements.txt`. It seems like this is more or less what is described in ["*Workflow for layered requirements*"](https://github.com/jazzband/pip-tools/tree/6.13.0#workflow-for-layered-requirements). — sinoroc, Apr 19 '23 at 15:47
Using your link, I can get something to work using 2 `requirements.in` and `dev-requirements.in` files, which is nice. Just wondering if I could somehow still use the `pyproject.toml` as input, but I'm doubting it. I will post an answer and close this question if I don't find anything in a few days. — Kins, Apr 24 '23 at 07:06
As far as I can tell from reading the doc `pip-compile` can take a `pyproject.toml` as input. -- I am a bit confused by your question because everything you seem to be asking for is quite clearly documented on the first page of the doc of *pip-tools*... — sinoroc, Apr 25 '23 at 08:05

score 0 · Answer 1 · answered Jul 25 '23 at 13:05

This PR #1936 introduces -c/--constraint option to pip-compile which allows you to pass requirements.txt as a constraint file when compiling dev-requirements.txt, for example:

$ pip-compile --extra dev -o dev-requirements.txt -c requirements.txt pyproject.toml

This should resolve the issue.

Generate aligned requirements.txt and dev-requirements.txt with pip-compile

1 Answers1