Why can't I import my published library after installing it with pip?

Question

I published a library on Pypi (https://pypi.org/project/Extractable/0.0.1). I installed it using pip install Extractable. It is present in pip list, yet I cannot import it into a Python file.

I noticed the other packages which are importable all have a [package].dist-info folder as well as a Python package [package] folder, whereas my library only has theExtractable.dist-info folder.

In a separate file I tried to import Extractable but it gives me the following error:

Traceback (most recent call last):
  File "C:\deletemepls\main.py", line 6, in <module>
    import Extractable
ModuleNotFoundError: No module named 'Extractable'

For reference, my published pip library directory looks like this:

├───.idea
│   └───inspectionProfiles
├───build
│   ├───bdist.win-amd64
│   └───lib
│       ├───Extractable
│       └───TableGenerator
├───dist
├───src
│   ├───Extractable
│   │   ├───.idea
│   │   │   └───inspectionProfiles
│   │   ├───Datatypes
│   │   │   └───__pycache__
│   │   └───__pycache__
│   ├───Extractable.egg-info
│   ├───fpdf
│   │   └───__pycache__
│   ├───genalog
│   │   └───*
│   ├───TableGenerator
│   │   ├───fonts
│   │   └───__pycache__
│   ├───test
│   │   ├───files
│   │   │   └───tables
│   │   └───fonts
│   └───weasyprint
│       └───*
├───tests
├───setup
├───README
├───pyproject
└───MANIFEST

setup.py looks like this:

from setuptools import setup
with open("README.md", "r") as fh:
    long_description = fh.read()

setup(
    name='Extractable',
    version='0.0.1',
    description='Extract tables from PDFs',
    py_modules=['Extractable/Extractor', 'TableGenerator/CreatePDFTable'],
    package_dir={'': 'src'},
    include_package_data=True,
    package_data={
        'sample': ['package_data.dat'],
    },
    long_description=long_description,
    long_description_content_type="text/markdown",
    classifiers=[
        "Programming Language :: Python :: 3",
        "Programming Language :: Python :: 3.11",
        "License :: OSI Approved :: MIT License",
        "Natural Language :: English"
    ],
    extras_require={
        "dev": [
            "pytest>=3.7",
        ],
    },
    install_requires=[
        "toolz ~= 0.12.0",
        "torch ~= 2.0.1",
        "Pillow ~= 9.5.0",
        "transformers ~= 4.29.2",
        "matplotlib >=3",
        "pdf2image ~= 1.16.3",
        "pdf2jpg ~= 1.1",
        "scipy ~= 1.10.1",
        "timm",
        "defusedxml",
        "Faker ~= 18.9.0"

    ],
    url="https://github.com/SuleyNL/Extractable",
    author="Suleymen C. Kandrouch",
    author_email="suleyleeuw@gmail.com",
)

and pyproject.toml looks like this:

[build-system]
requires = ["setuptools>=61.0"]
build-backend = "setuptools.build_meta"

[project]
name = "Extractable"
version = "0.0.1"
authors = [
  { name="Suleymen C. Kandrouch ", email="suleyleeuw@gmail.com" },
]
description = "Extracting tables from PDFs"
readme = {file = "README.md", content-type = "text/markdown"}
requires-python = ">=3.7"
classifiers = [
    "Programming Language :: Python :: 3",
    "License :: OSI Approved :: MIT License",
    "Operating System :: OS Independent",
]
license = {file = "LICENSE.txt"}
dependencies = [
    "toolz ~= 0.12.0",
    "torch ~= 2.0.1",
    "Pillow ~= 9.5.0",
    "transformers ~= 4.29.2",
    "matplotlib >=3",
    "pdf2image ~= 1.16.3",
    "pdf2jpg ~= 1.1",
    "scipy ~= 1.10.1",
    "timm",
    "defusedxml",
    "Faker ~= 18.9.0"
  ]


[project.urls]
"Homepage" = "https://github.com/SuleyNL/Extractable"
"Bug Tracker" = "https://github.com/SuleyNL/Extractable/issues"

EDIT:

It is very likely that I missed a step in the building process, I am going to redo it and publish it as version 0.0.2. So the question still stands as version 0.0.1 will stay online. If it is resolved in 0.0.2 I will edit again with explanation of what worked for me.

Also when rerunning py -m build and twine upload publishing version 0.0.2 it is the same size .whl file. It doesn't seem like anything has changed. What have I done wrong?

Why is your `.tar.gz` 19.5 MB but your `.whl` is only 4.2 kb on PyPI? — MattDMo, May 30 '23 at 15:08
Is `Extractable` supposed to be capitalized? Some pip packages are imported by different names from what they're installed with. Also, take a look in `site-packages`. — TheTridentGuy supports Ukraine, May 30 '23 at 15:10
@TheTridentGuy yes, it is supposed to be capitalized. I dont know what you mean with site-packages but I am [looking into it](https://stackoverflow.com/questions/31384639/what-is-pythons-site-packages-directory) — suley, May 30 '23 at 15:15
@MattDMo is that wrong? have I likely missed a step in publishing it? — suley, May 30 '23 at 15:16
You do not need `py_modules` in `setup.py`, you need `packages`. -- Anyway you seem to be following outdated advice. You should get rid of `setup.py` and use a `pyproject.toml` (maybe with a `setup.cfg` file) instead — sinoroc, May 30 '23 at 17:34

score -1 · Answer 1 · answered May 30 '23 at 16:57

Thanks to @mugiseyebrows I managed to solve it:

STEP 1. Adding a __init__.py file in the src folder as well as in every folder inside the src resolved the issue.

STEP 2. import all files within each of these folders into every __init__.py file.

STEP 3. add a setup.cfg file

STEP 4. build using python setup.py sdist bdist_wheel

STEP 5. upload to pypi using twine upload + your dir

For more information I would refer anyone to this latest tutorial which helped me put the pieces together: https://towardsdatascience.com/the-complete-guide-for-creating-a-good-pypi-package-acb5420a03f8

There should not be any `__init__.py` at the root of the `src` directory. That is a red flag that the `src`-layout is not correctly understood. I recommend you look at this [packaging tutorial](https://packaging.python.org/en/latest/tutorials/packaging-projects/), pay close attention to the project directory structure. — sinoroc, May 30 '23 at 17:30

Why can't I import my published library after installing it with pip?

1 Answers1