1

Context

I currently have 2 private repositories containing each a python package. For example: pkg_a, pkg_b

The package pkg_b need pkg_a inside his requirements.txt and setup.py file to be able to work.

Environment
  • Windows 10
  • Python 3.9.0
  • Pip 23.0.1

My problem

I am not able to package pkg_b with the private package pkg_a as a dependency because install_requires inside setup.py of pkg_b does not like the format of my requirements.txt :

requirements.txt (pkg_b)

matplotlib==3.5.2
numpy==1.21.6
--extra-index-url https://__token__:<token>@gitlab.com/api/v4/projects/<project>/packages/pypi/simple
pkg_a

setup.py (pkg_b)

def load_requirements(file_name):
    with open(file_name) as fp:
        reqs = filter(None, (parse_req_line(line) for line in fp))
        return list(reqs)

setup(
    name='pkg_b',
    version="1.0.0",
    packages=find_packages(),
    install_requires=install_requires,
    ...
)

Installing pkg_b

python .\setup.py install

Output

error in pkg_b setup command: 'install_requires' must be a string or list of strings containing valid project/version requirement specifiers; Parse error at "'--extra-'": Expected W:(abcd...)

(this is expected because load_requirements() does not deal with --extra-index-url)

What works

Installing the pkg_a from the requirements.txt inside a virtual environment :

powershell

python -m venv venv
.\venv\Scripts\activate
python -m pip install --upgrade pip
python -m pip install --no-input -r .\requirements.txt

pip list

...
matplotlib      3.5.2
numpy           1.21.6
pkg_a           1.0.0
...

Everything works when I run a python script of pkg_b that imports pkg_a

What I tried

  • Using pip's VCS support for git (does not work because I am using the package registry)
  • Using a .pypirc like suggested here
  • Using dependency_links of setup.py or setting up a different config file like suggested here

Usefull documentation

Some people said it's not impossible some people said it is, but I can't find a working solution. Let's unite together and find an answer :)

sinoroc
  • 18,409
  • 2
  • 39
  • 70
Hugo Barou
  • 31
  • 5

1 Answers1

3

It is not possible to get such requirements.txt content into the package metadata. Technically as well as conceptually package metadata can not contain --extra-index-url and other things like that.

I recommend you read this:

In the install_requires parameter of setuptools.setup() function call of the setup.py script, you should list simply pkg_a as an abstract dependency requirement. More concrete details about this dependency do not belong in the package metadata, they do not belong in install_requires. It's as simple as that.

I also recommend to read these:

What you can do is instruct users of your code to install directly from a requirements.txt file that contains the more concrete dependency requirements. For example in your documentation, you can give the recommendation of installing via a command such as this:

python -m pip install https://host.dev/path/to/requirements.txt
sinoroc
  • 18,409
  • 2
  • 39
  • 70