1

I know that there's likely a duplicate of this question floating around, so I apologize ahead of time as I've spent significant time digging and I haven't been able to find it.

I'm attempting to make my python package, svmksalesforce, installable so I can share it with my coworkers. I'm struggling to set-up my internal imports so that they work both when the package is installed and when I'm executing package modules directly while building/testing the package.

For example, in order to subclass my Insert to create a BulkInsert subclass, while building the package, in my bulk_insert.py file, I have the following import statement: from insert import Insert. This executes fine when executing bulk_insert.py (__name__ == '__main__').

However, after installing my package, I receive the following error when I execute import svmksalesforce:

    from insert import Insert
ModuleNotFoundError: No module named 'insert'

After doing some research, I determined that I needed to instead import from the top-level parent directory, so I updated my import in bulk_query.py to: from svmksalesforce.insert import Insert.

This import structure allows me to run import svmksalesforce and has allowed me to share the package with my coworkers, but now when I'm adding new functionality, I can no longer execute bulk_query.py directly. I again, receive an import error: 

    from svmksalesforce.insert import Insert
ModuleNotFoundError: No module named 'svmksalesforce'

I found this post that explains why the import fails, but I can't seem to find a solution that works both when __name__ == '__main__' (executed directly) and when __name__ == 'svmksalesforce' (executed via import)

Right now I'm updating all my import statements when I need to add new functionality and then updating them all back when I need to create a new sdist. Every time I create one more commit mentioning reverting import statements, I die a little inside.

Would be super appreciative to anyone who can save me that pain.

Directory Structure:

svmksalesforce directory structure

Daniel Long
  • 1,162
  • 14
  • 30
  • Please specify which code example belongs to which file. – Finomnis Jul 13 '19 at 01:34
  • Both import snippets are from the bulk_query.py file. I use one import syntax when executing directly in the package and one when importing the package – Daniel Long Jul 13 '19 at 01:42
  • There is no bulk_query.py file ... – Finomnis Jul 13 '19 at 01:53
  • Which IDE? Pycharm? – Finomnis Jul 13 '19 at 01:55
  • Also, you might find the concept of 'relative import paths' useful: https://realpython.com/absolute-vs-relative-python-imports/. E.g. if you want to access the `bulk_insert` package from `shared_func.py`, you could write `from .insert.bulk_insert import <...>`. – Finomnis Jul 13 '19 at 01:57
  • apologies bulk_insert.py. And I'm using Pycharm. Unfortunately, I've tried relative imports as well. Issue persists with them as well. Only works for executing directly or when executing via import but not both – Daniel Long Jul 13 '19 at 06:43
  • What exactly was the problem with relative imports? – Finomnis Jul 13 '19 at 08:22
  • Issue with relative imports is the same as issue with absolute imports. The issue is caused by the top-level directory changing. I either have to choose between the relative imports working when the module is executed directly or when imported. If I optimize for import, I receive `ValueError: attempted relative import beyond top-level package when executing module directly` when executing directly. – Daniel Long Jul 15 '19 at 16:04

1 Answers1

1

I ended up finding a solution that appears to be working.

  1. Create a virtual environment to use when executing modules in your package directly
  2. Activate your virtual environment
  3. Run pip install -e ~/path/to/package. In this example, your setup.py file should be in the package folder.

This process installs the package to your virtual environment in editable mode. This means any changes you make to the modules in the package are immediately reflected in the installed version of your package in your virtual env, just as they would be if you were building and executing an internal module directly.

This allows you to then import as if the package directory is always the top-level directory. In my example in the question description, I ended up using from svmksalesforce.insert import Insert

Daniel Long
  • 1,162
  • 14
  • 30