4

Short version:

How can I poetry install a package where one of the dependencies is a local tarball/zip file? It doesn't seem to work, yet it is shown in the poetry docs?

I can poetry install the package when the dependency is pulled from gitlab, but the install fails when I manually download the dependency from gitlab as a tarball and try to poetry install with the dependency in the tarball.



Long version:

I am trying to use poetry to install two packages that I have developed:

  • a base package called my_package
  • an extension called extension_of_my_package.

Both packages are in private repos in gitlab, and both have a pyproject.toml containing their dependency list. I can successfully poetry install the extended package (extension_of_my_package) when the base package my_package is downloaded from gitlab. i.e. the pyproject.toml file in extension_of_my_package has a tool.poetry.source section that gives the location of the my_package private repo on gitlab.

However, external users cannot access my private repo, so I need to ensure the packages can be installed from tarballs (that I download from gitlab and give to the client).

To install extension_of_my_package I do this:

tar xzf extension_of_my_package.tgz
cd extension_of_my_package/python

and then edit the pyproject.toml, changing the dependency on my_package to point to the local tarball:

my_package = { path = "/path/to/my_package.tgz"}

and then run poetry install. This fails with the error message:

> poetry install
Updating dependencies
Resolving dependencies... (9.3s)

  TypeError

  expected string or bytes-like object

  at /home/user/.poetry/lib/poetry/_vendor/py3.8/poetry/core/utils/helpers.py:27 in canonicalize_name
       23│ _canonicalize_regex = re.compile(r"[-_]+")
       24│ 
       25│ 
       26│ def canonicalize_name(name):  # type: (str) -> str
    →  27│     return _canonicalize_regex.sub("-", name).lower()
       28│ 
       29│ 
       30│ def module_name(name):  # type: (str) -> str
       31│     return canonicalize_name(name).replace(".", "_").replace("-", "_")

According to the poetry docs it is possible to install from a local file:

[tool.poetry.dependencies]
# directory
my-package = { path = "../my-package/", develop = false }

# file
my-package = { path = "../my-package/dist/my-package-0.1.0.tar.gz" }

I also tried using my-package = { file = ... instead of my-package = { path = ..., but it didn't work either.

I tried adding a minimal setup.py file to my_package (see this post), but that didn't help.

I tried converting my_package (in tarball format) to a wheel. I can successfully poetry install when my package is in wheel format, but my_packages's dependencies are not installed. I can't see how to include the dependency info in the wheel. When I created the wheel I tried specifying the dependency info in two ways:

  1. in setup.cfg:
[metadata]
name = my_package
version = 0.1.0
description = My Package
license = Proprietary

[options]
packages = find:
install_requires =
   matplotlib >=3.2.0

and

  1. in setup.py"
from setuptools import setup

setup(
   name=`my_package`,
   version="0.1.0,
   packages=['.my_package'],
   install_requires=['matplotlib >= 3.2.0',]
)

To rule out any problem with my own package, I created a minimal test and tried to poetry install a publicly available package (tqdm) from its zip file (downloaded from github). It also fails. The pyproject.toml for this minimal test is:

[tool.poetry]
name = "tester"
version = "0.0.1"
description = "test package"
authors = [ "me" ]
packages = [
    { include = "tester" }
]

[tool.poetry.dependencies]
python = ">=3.7,<3.9"
tqdm = {file = "/home/user/tqdm-master.zip"}

and the error message is:

> poetry install
Updating dependencies
Resolving dependencies... (13.0s)

  RuntimeError

  Unable to determine package info from path: /home/user/tqdm-master.zip

  at /home/user/.poetry/lib/poetry/puzzle/provider.py:251 in get_package_from_file
      247│             package = PackageInfo.from_path(path=file_path).to_package(
      248│                 root_dir=file_path
      249│             )
      250│         except PackageInfoError:
    → 251│             raise RuntimeError(
      252│                 "Unable to determine package info from path: {}".format(file_path)
      253│             )
      254│ 
      255│         return package

I am using poetry version 1.1.13.

I am open to any alternative approaches, so long as all the dependencies are checked.

John
  • 451
  • 7
  • 17
  • How about keeping the package in gitlab and using https://docs.gitlab.com/ee/user/project/deploy_tokens/ to install – Origin Apr 06 '22 at 06:44

0 Answers0