2

I have a custom Python package (call it MyProject) on my filesystem with a setup.py and a requirements.txt. This package needs to be used by a Flask server (which will be deployed on AWS/EC2/EB).

In my Flask project directory, I create a virtualenv and run pip install -e ../path/to/myProject.

But for some reason, MyProject's upstream git repo shows up in pip freeze:

...
llvmlite==0.19.0
-e git+https://github.com/USERNAME/MYPROJECT.git@{some-git-hash}
python-socketio==1.8.0
...

The reference to git is a problem, because the repository is private and the deployment server does not (and should not, and will never) have credentials to access it. The deployment server also doesn't even have git installed (and it seems extremely problematic that pip assumes without my permission that it does). There is nothing in MyProject's requirements.txt or setup.py that alludes to git, so I am not sure where the hell this is coming from.

I can dupe the project to a subdirectory of the Flask project, and then put the following in MyFlaskProject's requirements.txt:

...
llvmlite==0.19.0
./MyProject
python-socketio==1.8.0
...

But this doesn't work, because the path is taken as relative to the working directory of the pip process when it is run, not to requirements.txt. Indeed, it seems pip is broken in this respect. In my case, EC2 runs its install scripts from some other directory (with a full path to requirements.txt specified), and as expected, this fails.

What is the proper way to deploy a custom python package as a dependency of another project?

davidism
  • 121,510
  • 29
  • 395
  • 339
trbabb
  • 1,894
  • 18
  • 35
  • Does it work with an absolute path? – Simon Hobbs Aug 31 '17 at 00:25
  • @SimonHobbs: No, because the absolute path is determined by the deployment server. It's different than on my machine, not under my control, and might change from deployment to deployment. – trbabb Aug 31 '17 at 00:41

1 Answers1

1

To install your own python package from a git repo you might want to check this post

To sort out the credential issue, why not having git installed on the EC2? You could simply create an ssh key and share it with MyProject repository.

I am using this solution on ECS instances deployed by Jenkins (with Habitus to hide Jenkin's ssh keys while building the image) and it works fine for me!

SivolcC
  • 3,258
  • 2
  • 14
  • 32