5

Python3 Pip error + Poetry Packaging

I am working in a python library that I am trying to publish to TestPypi. So far, there have been no issues with publishing my Poetry builds.

For context, as a beginner, I come from these websites :

  1. https://python-poetry.org/docs/
  2. https://packaging.python.org/en/latest/tutorials/packaging-projects/

The only issue that has arose is that dependencies listed in my pyproject.toml are not accounted for when installing the package with pip install.

I have attempted at updating setuptools and pip but I have done so to no avail.

My goal is to have clean dependency installation without the versioning errors.

This is the main solution I have tried.

pyproject.toml

I hid my real names.

[tool.poetry]
name = "package-name"
version = "0.1.0"
description = "<desc>"
authors = ["<myname> <myemail>"]
license = "MIT"

[tool.poetry.dependencies]
python = "^3.10"
beautifulsoup4 = {version = "4.11.1", allow-prereleases = true}
recurring-ical-events = {version = "1.0.2b0", allow-prereleases = true}
requests = {version = "2.28.0", allow-prereleases = true}
rich = {version = "12.4.4", allow-prereleases = true}



[tool.poetry.dev-dependencies]
black = {version = "22.3.0", allow-prereleases = true}

[build-system]
requires = ["poetry-core>=1.0.0"]
build-backend = "poetry.core.masonry.api"

As the installer iterates through a dependency, it will return this error depending on whichever one is ordered first. (Throughout my monkey-patch-like attempts at fixing this, I was able to change the order of installation by modifying the strictness of each dependency version)

the error pip returns

ERROR: Could not find a version that satisfies the requirement requests==2.28.1 (from homeworkpy) (from versions: 2.5.4.1)
ERROR: No matching distribution found for requests==2.28.1
  • I have tried changing the strictness of the versions. (I removed the ^)
  • Switching to Poetry as a manager was also an attempt. My previous attempts were manual.
  • I have verified that the builds are corresponding to the correct builds previously published.

For extra info: I am building on a Github Codespace in which I run on 18.04.1-Ubuntu

Would anyone have any knowledge to spare of an issue like this? I am quite new to packaging and building, and I have had some success in most parts except for dependencies.

coldenate
  • 79
  • 1
  • 9

1 Answers1

2

Main Error

TLDR; Pip tries to resolve dependencies with TestPypi, but they are in another index (Pypi). Workarounds at end of answer.

The fact that I am publishing to TestPypi is the reason this has happened. I will explain why what I did made this error appear, and then I will show how you, from the future, may solve this.

Difference between Pypi and TestPypi

Pypi is the Python Package Index. It's a giant index of Python packages one may install from with pip install. TestPypi is the Python Package Index designated for testing and publishing without touching the real Package Index. It can be useful in times when learning how to publish a package. The main difference is that it is a completely separate repository. Therefore, what's on TestPypi may not be exactly what's on Pypi. My research was limited, so if I confused anyone, the main difference is that they are two different Package Indexes. One was made for testing purposes.

I published my package to TestPypi and set my pip install to install from that repository. Not Pypi, but TestPypi.

Why dependency resolution failed

When I defined my project's dependencies, I defined them based off of their Pypi presences. Most dependencies are present in Pypi. Not TestPypi. This meant that when I asked for my package from TestPypi, pip only looked at TestPypi, and the pip installer workflow fell out to a pattern like this:

0.5. Set fetching repository to TestPypi and Not Pypi.

  1. Pull package from TestPypi
  2. Install and examine dependencies
  3. Find first dependency (e.g. Beautifulsoup4)
  4. Pull dependency from TestPypi
  5. Successfully install Beautifulsoup4 -. This is because beautifulsoup4 is actually present in the TestPypi.
  6. Move on to another dependency (e.g. rich)
  7. Fail to pull from TestPypi -. Rich is not present in TestPypi.
  8. Return dependency not found.

Why some dependencies oddly worked

As you see in workflow step 5., the beautifulsoup4 package was found on the TestPypi. (Someone had put it up there). image to TestPypi page with beautifulsoup4 However, as you see in step 7., Rich is not found on the TestPypi index. This issue occurs because I set my repoistiroy to install from TestPypi because my that is where my package was held. This caused pip to use TestPypi. for every single dependency as well.

How I got around it.

I got around it by using TestPypi to verify accurate build artifact publishing, and then I jumped to Normal Pypi to test installation and dependency installation.

Workarounds

Install from TestPypi

python3 -m pip install -i https://test.pypi.org/simple/ <package name>

Install from Pypi (by default)

python3 -m pip install <package name>

Install package from TestPypi but dependencies from Pypi

The Python Docs explains this very well.

If you want to allow pip to also download packages from PyPI, you can specify --extra-index-url to point to PyPI. This is useful when the package you’re testing has dependencies:

python3 -m pip install --index-url https://test.pypi.org/simple/ --extra-index-url https://pypi.org/simple/ your-package

coldenate
  • 79
  • 1
  • 9