1

I am in the file "Query.py" trying to import the class SqlQuery from the file "database.py" which is located up two levels & then down one in another directory (also called "database"). Then the import statement I expect to work looks like this: from ...database.database import SqlQuery

But that gives me:

Traceback (most recent call last):
  File "C:/Users/Roland/dir1/2020/Indeed-Scraper/scraper/classes/Query.py", line 16, in <module>
    from ....database.database import SqlQuery
ValueError: attempted relative import beyond top-level package

since I understand way you run the file can be important, I am trying to run Query.py using my PyCharm IDE

This is way harder than I expected it to be!

Here are the relevant parts of my file structure:

project_name/
    __init__.py
    database/
        __init__.py
        database.py
    scraper/
        __init__.py
        classes/
            __init__.py
            Query.py

And here are the various things I've tried: (this code sits at the top of Query.py, one line above the from ...database.database import SqlQuery statement)

import sys
sys.path.append(os.path.realpath('...'))  # attempt 1
sys.path.insert(0, '')  # attempt 2
sys.path.append("...")  # attempt 3 - I think it is strange that adding "..." to the path did not work?

I did Google & check out various StackOverflow threads about this. I either didn't find a satisfactory answer (I can't believe I need to write sys.path.append(anything) in the best solution) or I didn't understand a solution when I found it. I've been here, here, here, to this link, and here, among other places.

I'm also reading the top-level answer in the popular post, "Relative imports for the billionth time" and I understand that:

  • My Query.py file is exectued as a "top-level script" when I execute it directly in my PyCharm IDE.
  • "database.py" is loaded as a module since it comes from an import statement.
  • "Scripts can't import relative", the author says. "Relative imports are only for use within module files."

...and so, while the thread was informative, I just don't see any answers that seem to be solutions to my problem. I really don't want to post this since it must be a duplicate of some kind, but I haven't found an answer. What gives, Python? Why is importing so hard?

Footnote: Currently the __init__.py files are empty. Should I be filling them with something to make relative imports easier?

Roly Poly
  • 489
  • 1
  • 9
  • 19
  • 1
    One thing I do to avoid issues with imports is just make a `setup.py` for your project. Then run `pip install -e .`. – kimbo Mar 01 '20 at 22:13
  • Something like what's talked about in this thread? https://stackoverflow.com/questions/1471994/what-is-setup-py i.e. in the 2nd answer, "Using setup.py" and "Creating setup.py" – Roly Poly Mar 01 '20 at 22:28
  • 1
    Yup! That explains it nicely. – kimbo Mar 01 '20 at 23:05
  • 1
    Also, the `__init__.py` files don't need anything in them. They are there so Python knowns what's a module. You can put stuff in them if you want. A common thing to do is import everything into your root `__init__.py` so everything is importable from the root module. E.g. `from foo import bar` instead of `from foo.baz.bim.bop import bar`. – kimbo Mar 01 '20 at 23:09
  • so I made a setup.py file, uploaded it to PyPI, did pip install indeed-scraper (the name of my pkg) and then... ```import indeed-scraper``` did not work. I tried again setting the ```name``` value in ```setuptools.setup()``` to ```indeedScraper``` instead of ```indeed-scraper``` in case the dash was tripping things up in the import statement. Nada. After ```pip install indeedScraper```, importing "indeedScraper" gave me ```ModuleNotFoundError```. I've made progress but I'm still stuck! – Roly Poly Mar 02 '20 at 23:18
  • I should've been more specific. You don't actually have to upload it to PyPi for setup.py to work. Just go to the root of your project and run `pip install -e .` or `python setup.py install`. That will do the same thing as installing it from PyPi – kimbo Mar 02 '20 at 23:31
  • And for the name thing, Python modules can't have a dash in the name. – kimbo Mar 02 '20 at 23:34
  • You might be missing an __init__.py in your scraper directory – kimbo Mar 03 '20 at 00:43
  • Thank you Kimbo but I do have ```__init__.py``` there, just didn't list it – Roly Poly Mar 03 '20 at 00:44
  • Oh okay. If you're code is open source, you could post a link to your project and I could take a look a little later today – kimbo Mar 03 '20 at 00:48
  • Let us [continue this discussion in chat](https://chat.stackoverflow.com/rooms/208886/discussion-between-kimbo-and-roly-poly). – kimbo Mar 03 '20 at 02:04

0 Answers0