93

Im installing pipenv in my docker:

RUN pip install pipenv
RUN cd /my/app/path/ && pipenv install
RUN cd /my/app/path/ && pipenv shell

Im getting the error:

Traceback (most recent call last):
  File "/usr/local/bin/pipenv", line 11, in <module>
    sys.exit(cli())
  File "/usr/local/lib/python2.7/dist-packages/pipenv/vendor/click/core.py", line 722, in __call__
    return self.main(*args, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/pipenv/vendor/click/core.py", line 697, in main
    rv = self.invoke(ctx)
  File "/usr/local/lib/python2.7/dist-packages/pipenv/vendor/click/core.py", line 1066, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/usr/local/lib/python2.7/dist-packages/pipenv/vendor/click/core.py", line 895, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/local/lib/python2.7/dist-packages/pipenv/vendor/click/core.py", line 535, in invoke
    return callback(*args, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/pipenv/cli.py", line 2057, in shell
    do_shell(three=three, python=python, fancy=fancy, shell_args=shell_args)
  File "/usr/local/lib/python2.7/dist-packages/pipenv/cli.py", line 1952, in do_shell
    shell = os.path.abspath(PIPENV_SHELL)
  File "/usr/lib/python2.7/posixpath.py", line 360, in abspath
    if not isabs(path):
  File "/usr/lib/python2.7/posixpath.py", line 54, in isabs
    return s.startswith('/')
AttributeError: 'NoneType' object has no attribute 'startswith'

If I run

RUN cd /my/app/path/ && pipenv install --system

instead, im getting another error:

build   30-Sep-2017 16:50:45    Step 5/9 : RUN cd /my/app/path &&     pipenv install --system
build   30-Sep-2017 16:50:45     ---> Running in cffd31633074
build   30-Sep-2017 16:50:46    [91mPipfile.lock not found, creating…
build   30-Sep-2017 16:50:46    [0m[91mLocking [dev-packages] dependencies…
build   30-Sep-2017 16:50:46    [0m[91mLocking [packages] dependencies…
build   30-Sep-2017 16:50:49    [0m[91mCRITICAL:pip.utils:Error [Errno 2] No such file or directory while executing command python setup.py egg_info
build   30-Sep-2017 16:50:49    [0m[91mTraceback (most recent call last):
build   30-Sep-2017 16:50:49      File "/usr/local/bin/pipenv", line 11, in <module>
build   30-Sep-2017 16:50:49        sys.exit(cli())
build   30-Sep-2017 16:50:49      File "/usr/local/lib/python2.7/dist-packages/pipenv/vendor/click/core.py", line 722, in __call__
build   30-Sep-2017 16:50:49    [0m[91m    return self.main(*args, **kwargs)
build   30-Sep-2017 16:50:49      File "/usr/local/lib/python2.7/dist-packages/pipenv/vendor/click/core.py", line 697, in main
build   30-Sep-2017 16:50:49    [0m[91m    rv = self.invoke(ctx)
build   30-Sep-2017 16:50:49      File "/usr/local/lib/python2.7/dist-packages/pipenv/vendor/click/core.py", line 1066, in invoke
build   30-Sep-2017 16:50:49    [0m[91m    return _process_result(sub_ctx.command.invoke(sub_ctx))
build   30-Sep-2017 16:50:49      File "/usr/local/lib/python2.7/dist-packages/pipenv/vendor/click/core.py", line 895, in invoke
build   30-Sep-2017 16:50:49    [0m[91m    return ctx.invoke(self.callback, **ctx.params)
build   30-Sep-2017 16:50:49      File "/usr/local/lib/python2.7/dist-packages/pipenv/vendor/click/core.py", line 535, in invoke
build   30-Sep-2017 16:50:49        return callback(*args, **kwargs)
build   30-Sep-2017 16:50:49      File "/usr/local/lib/python2.7/dist-packages/pipenv/cli.py", line 1782, in install
build   30-Sep-2017 16:50:49    [0m[91m    do_init(dev=dev, allow_global=system, ignore_pipfile=ignore_pipfile, system=system, skip_lock=skip_lock, verbose=verbose, concurrent=concurrent, deploy=deploy)
build   30-Sep-2017 16:50:49      File "/usr/local/lib/python2.7/dist-packages/pipenv/cli.py", line 1290, in do_init
build   30-Sep-2017 16:50:49    [0m[91m    do_lock(system=system)
build   30-Sep-2017 16:50:49      File "/usr/local/lib/python2.7/dist-packages/pipenv/cli.py", line 1080, in do_lock
build   30-Sep-2017 16:50:49    [0m[91m    pre=pre
build   30-Sep-2017 16:50:49      File "/usr/local/lib/python2.7/dist-packages/pipenv/utils.py", line 421, in resolve_deps
build   30-Sep-2017 16:50:49    [0m[91m    resolved_tree.update(resolver.resolve())
build   30-Sep-2017 16:50:49      File "/usr/local/lib/python2.7/dist-packages/pipenv/patched/piptools/resolver.py", line 101, in resolve
build   30-Sep-2017 16:50:49    [0m[91m    has_changed, best_matches = self._resolve_one_round()
build   30-Sep-2017 16:50:49      File "/usr/local/lib/python2.7/dist-packages/pipenv/patched/piptools/resolver.py", line 199, in _resolve_one_round
build   30-Sep-2017 16:50:49    [0m[91m    for dep in self._iter_dependencies(best_match):
build   30-Sep-2017 16:50:49      File "/usr/local/lib/python2.7/dist-packages/pipenv/patched/piptools/resolver.py", line 293, in _iter_dependencies
build   30-Sep-2017 16:50:49        dependencies = self.repository.get_dependencies(ireq)
build   30-Sep-2017 16:50:49      File "/usr/local/lib/python2.7/dist-packages/pipenv/patched/piptools/repositories/pypi.py", line 171, in get_dependencies
build   30-Sep-2017 16:50:49        result = reqset._prepare_file(self.finder, ireq)
build   30-Sep-2017 16:50:49      File "/usr/local/lib/python2.7/dist-packages/pipenv/patched/pip/req/req_set.py", line 639, in _prepare_file
build   30-Sep-2017 16:50:49    [0m[91m    abstract_dist.prep_for_dist()
build   30-Sep-2017 16:50:49      File "/usr/local/lib/python2.7/dist-packages/pipenv/patched/pip/req/req_set.py", line 134, in prep_for_dist
build   30-Sep-2017 16:50:49    [0m[91m    self.req_to_install.run_egg_info()
build   30-Sep-2017 16:50:49      File "/usr/local/lib/python2.7/dist-packages/pipenv/patched/pip/req/req_install.py", line 438, in run_egg_info
build   30-Sep-2017 16:50:49    [0m[91m    command_desc='python setup.py egg_info')
build   30-Sep-2017 16:50:49      File "/usr/local/lib/python2.7/dist-packages/pipenv/patched/pip/utils/__init__.py", line 667, in call_subprocess
build   30-Sep-2017 16:50:49    [0m[91m    cwd=cwd, env=env)
build   30-Sep-2017 16:50:49      File "/usr/lib/python2.7/subprocess.py", line 711, in __init__
build   30-Sep-2017 16:50:49    [0m[91m    errread, errwrite)
build   30-Sep-2017 16:50:49      File "/usr/lib/python2.7/subprocess.py", line 1343, in _execute_child
build   30-Sep-2017 16:50:49    [0m[91m    raise child_exception
build   30-Sep-2017 16:50:49    OSError: [Errno 2] No such file or directory
error   30-Sep-2017 16:50:49    The command '/bin/sh -c cd /opt/supercrunch/function-service/lib &&     pipenv install --system' returned a non-zero code: 1
build   30-Sep-2017 16:50:49    [0mSending build context to Docker daemon 40.96 kB

But when I instead do the following:

RUN pip install pipenv
RUN cd /my/app/path && pipenv install
RUN cd /my/app/path && pipenv install --system

It is working...

So two question: First: Why is pipenv shell giving me this error and Second: Why do I have to do pipenv install before pipenv install --system to get it working?

I would like to use pipenv to create virtual environments with different python versions and differne dependency versions.

Mulgard
  • 9,877
  • 34
  • 129
  • 232
  • 1
    Seems like an issue, it doesn't even run in a interactive shell. So it won't run in your Dockerfile because you can't get a shell when running a RUN command, they are all run without a TTY. But even getting a shell by running the container doesn't work, so you should open a issue with the pipenv team – Tarun Lalwani Oct 01 '17 at 08:17

6 Answers6

168

Considering your problem, as I know it the easiest way is to update to the latest version of pipenv. It is still in development, so problems are fixed very soon.

We are using pipenv with docker in production. And we really like it. There are several things to keep in mind:

  1. You need to use --system flag, so it will install all packages into the system python, and not into the virtualenv. Since docker containers do not need to have virtualenvs
  2. You need to use --deploy flag, so your build will fail if your Pipfile.lock is out of date
  3. You need to use --ignore-pipfile, so it won't mess with our setup

Check the official docs to be sure that this information is up-to-date.

All in all:

pipenv install --system --deploy --ignore-pipfile

There's also one more thing. If you are using the same Dockerfile for both development and production it would be very nice to also use --dev flag for the development environment only.

Also, check out our django project template to see the full example: wemake-django-template

Martijn Pieters
  • 1,048,767
  • 296
  • 4,058
  • 3,343
sobolevn
  • 16,714
  • 6
  • 62
  • 60
  • 3
    I would love to see how you guys are able to get this working. I have tried to do pipenv install with all given parameters and it always fails with similar errors – Bagelstein Apr 18 '18 at 12:47
  • @Bagelstein so, checkout our template: https://github.com/wemake-services/wemake-django-template But, please not that there is a opened bug in pipenv right now: https://github.com/wemake-services/wemake-django-template/issues/287 – sobolevn Apr 18 '18 at 19:43
  • When I use --system I get the following error: __main__.py: error: the following arguments are required: envname – fccoelho Jun 01 '18 at 12:14
  • 8
    Please take this answer https://stackoverflow.com/a/55610857/2832282 into account ! The official recommendation is to no longer use the `--system` flag for good reasons ! – Cyril Duchon-Doris Sep 05 '19 at 14:25
  • 1
    not a solution `ENTRYPOINT ["pipenv", "shell"]` still not working – bm13kk Jun 11 '21 at 18:31
  • @bm13kk you're not supposed to need that command since the dependencies were installed into system python – cryanbhu Jun 14 '21 at 08:22
  • @cryanbhu but i still need it. At least it uploaded .env to environment variables. – bm13kk Jun 14 '21 at 11:20
  • I agree that this answer only circumnavigates the problem. Running `pipenv` means you want to make use of `pipenv run ...` commands (which still fails), not that you just want to install packages (because being in a docker container, you could install packages anyway). You aren't in fact "running pipenv" as the question title requires. – gented Oct 29 '21 at 10:16
  • 2
    Does `--ignore-pipfile` have any impact when used in conjuction with `--deploy`? – belkka Nov 24 '21 at 06:09
72

The pipenv documentation no longer officially recommends using the --system flag in docker instances. Instead, they suggest using virtual environments, as "it is for deployment onto a full-fledged OS". This is with the caveat:

...most containers are deployed without virtualenvs as I believe you both note, I believe the purpose is to stay slim and reduce attack surface area by installing as little as possible

As stated in https://github.com/pypa/pipenv/pull/2762.

Instead, the solution would be to run (as stated in a different answer here):

RUN pipenv install --deploy --ignore-pipfile

And then prefix all calls to python with pipenv run, e.g. CMD ["pipenv", "run", "python", "hello.py"]

Ps. I would have wanted to put this as a comment to the accepted answer, but I don't have the reputation.

Kalior
  • 821
  • 7
  • 5
  • What about if one wants to run gunicorn? how should it be running in CMD or compose command? – oyeyipo Jan 17 '23 at 11:40
  • @oyeyipo If anyone is still wondering you can run gunicorn with pipenv, like so `CMD ["pipenv", "run", "gunicorn", "main:app", "--reload", "-b", "0.0.0.0:8080"]` – Simeon Borisov Jul 13 '23 at 11:41
16

The direct answer to this question is to not use shell, but rather run:

CMD ["pipenv", "run", "python", "my/app.py"]

If you need more flexibility, you could also pipenv run sh init.sh, which would create a shell initialized with all the pipenv environment variables.

I actually prefer the approach C. Sweet mentions. If you can get away with prebuilding your virtual environment and simply copying it over (setting PIPENV_VENV_IN_PROJECT then using a nested FROM followed by COPY --from=builder-image), you don't need python nor pipenv nor pipenv dependencies in your final container. This greatly reduces the size of the final image.

Dockerfile:

WORKDIR /etc/service/
CMD ["sh", "/etc/service/init.sh"]

init.sh:

source /etc/service/my/.venv/bin/activate
python my/app.py
James Taylor
  • 6,158
  • 8
  • 48
  • 74
  • 3
    "you don't need `python`" - sorry, how would you avoid including python in the container, when deploying a python application? – scry Dec 11 '20 at 06:40
11

I just generate a requirements.txt file and run it with normal pip.

RUN  pip install pipenv  \
    && pipenv requirements > requirements.txt \
    && pip install -r requirements.txt

You can even skip installing pipenv in your Dockerfile by generating a requirements file before building the container.

Since this doesn't install dependencies in a pipenv-managed virtual environment, there's no need to worry about pipenv in the docker-compose command.

command: >
    - sh 
    - -c 
    - |
      python myscript.py
      python myapp.py

This Python⇒Speed article helped me, which also includes requirements file generation for poetry and pip-tools.

Previous version, before 2022-04

The requirements.txt generation used to be be part of pipenv lock. If you're stuck on an old version, use this. But otherwise, just upgrade.

RUN  pip install pipenv  \
    && pipenv lock --keep-outdated --requirements > requirements.txt \
    && pip install -r requirements.txt
Michael
  • 8,362
  • 6
  • 61
  • 88
muon
  • 12,821
  • 11
  • 69
  • 88
  • 2
    Thanks a lot for your solution! I tried it, but it failed with the error `Error: No such option: --requirements`. This is caused bcs pipenv removed the `-r || --requirements` flags & provided a dedicated command for it --> `pipenv requirements`. Be careful, this just prints the requirements to stdout, to create a requirements.txt file shell output redirection is required --> `pipenv requirements > requirements.txt`. For the exact version when it got removed I was too lazy (sorry) but here is the link to the related [doc](https://pipenv.pypa.io/en/latest/advanced/#generating-a-requirements-txt) – Michael Aicher Aug 29 '22 at 12:16
1

The best thing is not to use pipenv directly, as you usually don't need a virtualenv in your container.

There is a tool called micropipenv that you can use in your container instead of pipenv.

https://github.com/thoth-station/micropipenv

micropipenv use cases

Why should I use micropipenv instead of Pipenv or Poetry?

  • I would like to have a tool that "rules them all" - one lightweight tool to support all Python dependency lock file managers (pip-tools, Poetry, Pipenv) and lets users decide what they want to use when deploying Python applications in containerized environments (e.g. Kubernetes, OpenShift, ...).

  • I would like to have a fast and minimalistic tool to install software packages in CI.

  • I would like to have containerized Python applications as small as possible with minimum software shipped and required to build and run the Python application in production.

  • I would like to convert files produced by Pipenv/Poetry to a pip-tools compatible output.

  • I don't want to install Pipenv/Poetry, but I would like to run a project that uses Pipenv/Poetry for dependency management (e.g. restricted environments).

  • My Pipenv installation is broken and Pipenv upstream did not issue any new Pipenv release <https://github.com/pypa/pipenv/issues/4058>_.

  • I would like to deploy my application into a production environment and my application dependencies are managed by Pipenv/Poetry (dependencies are already resolved), but I don't want to run Pipenv/Poetry in production (e.g. OpenShift's s2i build process).

Vad1mo
  • 5,156
  • 6
  • 36
  • 65
1

Each RUN directive spawns a new shell so it is useless to try to alter it with a pipenv shell command: any following instruction will not benefit from that.

All RUN directive that are meant to use pipenv's environment must be prefixed with pipenv run.

This would also apply to the CMD directive, as well as any command run with a container once the image is built, but you can hack an ENTRYPOINT to save prefixes for commands!

FROM python
RUN pip install pipenv
WORKDIR /app/
COPY Pipfile* /app
RUN mkdir /app/.venv
RUN pipenv install --deploy
RUN pipenv run something
ENTRYPOINT ["python", "-m", "pipenv", "run"]
CMD ["something"]
N1ngu
  • 2,862
  • 17
  • 35