9

I have two python projects that share some common libraries all organized into three git repositories: project1, project2, and common-lib. The two projects are each meant to be deployed to elastic beanstalk bundled with common-lib.

I'm trying to find the most idiomatic way to structure these projects to make it easy to develop for locally and to build a zip file for deployment using eb deploy.

Setting everything up for local development is easy. Just checkout each repo and do a python setup.py develop in common-lib to make the common libraries available in the virtualenv.

For EB deployment it would be nice to have a single setup.py command that produces an EB compatible zip file that contains the project and common-lib with a requirements.txt file that lists the pip dependencies for both. I have not yet found an easy way to do this which is a bit surprising because I imagine this is a fairly common scenario.

I can't specify the git repository for common-lib in either project1 or project2's requirements.txt file because the repository won't be reachable from AWS.

Eddie
  • 919
  • 2
  • 9
  • 21
  • 1
    So make it reachable: http://stackoverflow.com/questions/34727442/how-to-deploy-private-python-pip-dependency-with-amazon-aws-elastic-beanstalk/34727443#34727443 – smentek Jan 11 '16 at 17:57
  • @eddie What was your solution to this? I'm not sure, but I may be experiencing the same problem. My requirements.txt lists all my third-party dependencies, but excludes the project I'm developing myself. So I'm encountering a DistributionNotFound error where the distribution referenced is MyApp (aka the app I have written). Was your solution to make the private repository accessible via AWS using the solution provided by @smentek? – Drew Burnett Jul 17 '16 at 18:25
  • Do not do this. But, I junction (Windows) or hard-link my common packages into the multiple different ones. It is a pain to set up for developers after a pull, but works very nicely with git and EBS. – std''OrgnlDave Sep 06 '16 at 01:04
  • Not sure what kind of eb you're using but, why not use a docker container? Just have your DockerFile do what you do to set up your environment? – Luis F Hernandez Dec 10 '16 at 18:14

1 Answers1

1

For me the proper way would be to create a python package from the common lib, publish it to a private pypi server, like https://gemfury.com/l/pypi-server. Have it as a reference in requirements.txt as a python package.

Another solution can be to include the common-lib as a git submodule https://git-scm.com/docs/git-submodule. With that you will have the separation, because it will live in a separate repository and you will have a simple reference as a git submodule in your's project.

szaboat
  • 593
  • 3
  • 11