6

I am trying to put the back-end of a simple web-scraping application that I'm working on into a package, but this application relies on loading from a pickled python object, which I'm unable to load into the file with importlib. Before, when I had all the code in a single file, and relied on open(), all worked fine, but now I get an error when I import the package. After this error, I tried loading the file with importlib, and could not make it work.

I am following the steps in this answer from a similar question: How to read a (static) file from inside a Python package?.

The file structure in my package is:

mypackage\
        __init__.py
        parse.py
        search.py
        categories\
                categories.pickle
                generate_categories_if_corrupted.py

The content of init.py:

from %mymodule% import search

The code where the error happens:

import importlib.resources as resources
from pickle import load
from . import categories

try:
    with resources.open_binary(categories, "categories.pickle") as cat:
        CATS = load(cat)
except FileNotFoundError:
    raise FileNotFoundError("")

The error:

Traceback (most recent call last):
  File "%mypackage%\parse.py", line 15, in <module>
    with resources.open_binary(categories, "categories.pickle") as cat:
  File "C:\Users\%me%\AppData\Local\Programs\Python\Python38\lib\importlib\resources.py", line 92, in open_binary
    _check_location(package)
  File "C:\Users\%me%\AppData\Local\Programs\Python\Python38\lib\importlib\resources.py", line 82, in _check_location
    raise FileNotFoundError(f'Package has no location {package!r}')
FileNotFoundError: Package has no location <module '%mypackage%.categories' (namespace)>

During handling of the above exception, another exception occurred:

## just a  FileNotFoundError with an error message, as expected.

How do I fix this? It's my first time trying to make my code into a package in Python.

Thanks in advance for your answers.

Luiz Cavalcanti
  • 63
  • 1
  • 4
  • out of curiosity, why are there % in the import line in init.py – ewokx Jul 16 '20 at 02:30
  • I just didn't find that the name of my package was relevant to the question, so it's just a placeholder. There is a regular windows folder path in place of it. – Luiz Cavalcanti Jul 16 '20 at 03:26
  • you mean something like ```from c:\foobar import search```? – ewokx Jul 16 '20 at 03:27
  • Not sure it's the issue, but I would add a `mypackage/categories/__init__.py` file (file can stay empty). -- Edit: I'd suggest reading [this answer](https://stackoverflow.com/a/58941536/11138259) completely, which indeed seems to suggest that the `__init__.py` file is needed for `importlib.resources` to do its job. – sinoroc Jul 16 '20 at 09:28

2 Answers2

10

As per this answer on a related question (you might want to go and upvote this answer), for importlib.resources to do its job there has to be a __init__.py file in the packages.

So in your case, I believe there should be a mypackage/categories/__init__.py file (as always, that file can be left empty but it has to exist).

sinoroc
  • 18,409
  • 2
  • 39
  • 70
0

Also oddly enough, using python3.10 solved the problem for me without adding the __init__.py as suggested by @sinoroc 's answer.

Actually I got to this question because I had no errors on python3.10 with

from importlib import resources
from blah.assets import my_data

stuff = resources.read_text(my_data, "something.json")

but I realized my ci tool (python3.8 !) was running into that FileNotFoundError as well. And haha glad I got here because okay after adding the __init__.py , the tests in my python3.8 ci environment passed .

HeyWatchThis
  • 21,241
  • 6
  • 33
  • 41
  • I am not sure what this answer adds if not more confusion... Why the import as `pkg_resources`? The function `read_text()` is now deprecated.(as well as the `open_binary` mentioned in the question). Since Python 3.9 it is the `importlib.resources.files()` function that should be used for all use cases. – sinoroc Apr 07 '23 at 06:03