0

Time for another installment of "how do python imports work please god help me"

I am creating a REST API to consume a request, validate it, write some data to DynamoDb, and send a message to SQS, and another python application to serve as a worker to listen for SQS events, consume messages, and make a bunch of different API calls to a 3rd party service, and write updates to the same DDB instance.

My hope is to contain all of this logic in a single repository, dockerize both python apps, and use CDK to deploy all of the infrastructure to AWS.

My problem is that I have models and utility functions that I would like to share between both services to make it easy to handle the data transferred to SQS and do the database read/writes - but I cant seem to figure out a way to easily share the models between my applications.

Desired repository structure

.
├── README.md
├── app
│   ├── Dockerfile.api
│   ├── Dockerfile.worker
│   ├── Pipfile
│   ├── Pipfile.lock
│   ├── __init__.py
│   ├── api
│   │   ├── __init__.py
│   │   ├── main.py
│   │   ├── routers
│   │   │   ├── __init__.py
│   │   │   ├── accounts.py
│   │   │   └── health.py
│   │   └── utils
│   │       ├── __init__.py
│   │       ├── database.py
│   │       └── settings.py
│   ├── shared
│   │   ├── __init__.py
│   │   ├── models
│   │   │   ├── __init__.py
│   │   │   ├── api_response.py
│   │   │   └── provisioning_request.py
│   │   └── utils
│   │       ├── __init__.py
│   │       ├── database.py
│   │       └── settings.py
│   └── worker
│       ├── __init__.py
│       ├── main.py
│       └── routers
│           ├── __init__.py
│           └── health.py
└── operations
    └── all the CDK Stuff

What I want to be able to do:

# ./app/api/routers/account.py

from shared.models import ProvisioningRequest

# ./app/worker/main.py

from shared.models import ProvisioningRequest

I also need to be able to run this locally and bundle the shared + api/worker in my deploy to ECR (which i can manage in the respective dockerfiles once I get these imports sorted)

The problem:

❯ pipenv run python main.py
Traceback (most recent call last):
  File "/Users/user/Documents/GitHub/project/app/api/main.py", line 7, in <module>
    from routers import health, accounts
  File "/Users/user/Documents/GitHub/project/app/api/routers/accounts.py", line 9, in <module>
    from ..shared.models import ProvisioningRequest
ImportError: attempted relative import beyond top-level package

I keep getting import errors

I've Tried:

ajrice6713
  • 63
  • 1
  • 10
  • 1
    Show the full traceback of the error as properly formatted text (formatted as code) in the question. The error can't be caused directly by the shown imports, only by other code imported by the import. – Michael Butscher Apr 12 '23 at 15:38
  • Added the stack trace to the post – ajrice6713 Apr 12 '23 at 15:40
  • 1
    If you add the (absolute) path to the `app` directory to `sys.path`, the absolute `from shared.models import ...` should work. Making the relative import work would require the main file to be a sibling of the `app` directory or rather complicated trickery. – Michael Butscher Apr 12 '23 at 15:48
  • For my own curiosity - do you have an example of the `complicated trickery`? I can make the sys.path work since they will get deployed to ECR together - but I/m curious as to whether or not there is a prettier solution to this. – ajrice6713 Apr 12 '23 at 17:07
  • Basically by creating your own [`__import__`](https://docs.python.org/3/library/functions.html#import__) function which handles the import requests. Writing `__import__` right for all cases is the challenge. – Michael Butscher Apr 13 '23 at 05:38

0 Answers0