Container command fails in Django on Elastic Beanstalk Python 3.7

Question

I am using Django Python 3.7 on Elastic Beanstalk (Amazon Linux 2) and the following command fails:

container_commands:
  01_migrate:
    command: "pipenv run python ./manage.py migrate"
    leader_only: true

2020-07-17 09:31:57,017 [ERROR] Command 01_migrate (pipenv run python ./manage.py migrate) failed
2020-07-17 09:31:57,017 [ERROR] Error encountered during build of postbuild_0_sarahandanatol: Command 01_migrate failed
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/cfnbootstrap/construction.py", line 542, in run_config
    CloudFormationCarpenter(config, self._auth_config).build(worklog)
  File "/usr/lib/python2.7/site-packages/cfnbootstrap/construction.py", line 260, in build
    changes['commands'] = CommandTool().apply(self._config.commands)
  File "/usr/lib/python2.7/site-packages/cfnbootstrap/command_tool.py", line 117, in apply
    raise ToolError(u"Command %s failed" % name)
ToolError: Command 01_migrate failed

Hi. On AL2, if you want to execute your code under python 3, I think it should be `python3`. Have you tried with `python3`? — Marcin, Jul 17 '20 at 10:24
@Marcin I get the same error with this container command: `pipenv run python3 ./manage.py migrate` and for some reason, the exception still shows it is using python 2.7 — Anatol, Jul 17 '20 at 10:39
So this was working on AL1, but now when you migrate to AL2 it stopped working? — Marcin, Jul 17 '20 at 10:47
@Marcin the command worked on AL1: `python ./manage.py migrate` but `pipenv run python3 ./manage.py migrate` doesn't work on AL2 — Anatol, Jul 17 '20 at 10:53
can you check `/var/log/cfn-init-cmd.log`? Maybe it has more useful error message? — Marcin, Jul 17 '20 at 11:15
@Marcin yep you were right! `Couldn't import Django. Are you sure it's installed and available on your PYTHONPATH environment variable? Did you forget to activate a virtual environment?` I have django installed though and I run `pipenv run` before the commands? — Anatol, Jul 17 '20 at 11:19
Well that's the root of the issue. That's good it got established. Have you tried without `pipenv run`? — Marcin, Jul 17 '20 at 11:21
@Marcin Tried just now like this `python3 ./manage.py migrate`. still the same issue — Anatol, Jul 17 '20 at 11:27
Are you sure django is installed? when you login to the instance does your application work? — Marcin, Jul 17 '20 at 11:53
I have Django listed in my Pipfile. The instance has a sample app because my app couldn't be deployed because it will fail on the migrate command — Anatol, Jul 17 '20 at 11:58
@Marcin no. I am using pipenv. that's why I was running `pipenv run` before the migrate command — Anatol, Jul 17 '20 at 12:04
Let us [continue this discussion in chat](https://chat.stackoverflow.com/rooms/218030/discussion-between-marcin-and-anatol). — Marcin, Jul 17 '20 at 12:07

score 8 · Accepted Answer · answered Jul 18 '20 at 04:54

I tried to replicate the issue on my sandbox account. It was minimal version of the django just with the welcome screen. No database nor the use of environmental variables.

My Pipfile was minimal as well:

[[source]]
name = "pypi"
url = "https://pypi.org/simple"
verify_ssl = true

[dev-packages]

[packages]
django = "*"
djangorestframework = "*"

[requires]
python_version = "3.7"

I can confirm that using plain pipenv fails. Specifically, I had the following config file in my .ebextantions

container_commands:
  10_migrate:
    command: |
      pipenv run python ./manage.py migrate

The error message was about missing Django.

However, for me the solution was the following:

container_commands:
  10_migrate:
    command: |      
      source $PYTHONPATH/activate
      pipenv run python ./manage.py migrate

This activates python environment that EB is using to install Pipfile dependencies before pipenv is executed.

Below is a version which will also load EB environmental variables that maybe required to run the migrate job if you have database connection details passed as such.

container_commands:
  10_migrate:
    command: |
      export $(cat /opt/elasticbeanstalk/deployment/env | xargs)
      source $PYTHONPATH/activate
      pipenv run python ./manage.py migrate

Here is an example output from /var/log/cfn-init-cmd.log showing successful migrate run:

20-07-18 04:50:41,615 P3836 [INFO] Command 10_migrate
2020-07-18 04:50:42,969 P3836 [INFO] -----------------------Command Output-----------------------
2020-07-18 04:50:42,969 P3836 [INFO]    cat: /opt/elasticbeanstalk/deployment/env: No such file or directory
2020-07-18 04:50:42,969 P3836 [INFO]    export EB_IS_COMMAND_LEADER="true"
2020-07-18 04:50:42,969 P3836 [INFO]    export HOME="/root"
2020-07-18 04:50:42,969 P3836 [INFO]    export MYVAR="my-eb-env-value"
2020-07-18 04:50:42,969 P3836 [INFO]    export OLDPWD
2020-07-18 04:50:42,970 P3836 [INFO]    export PATH="/bin:/sbin:/usr/bin:/usr/sbin:/usr/local/bin:/usr/local/sbin"
2020-07-18 04:50:42,970 P3836 [INFO]    export PWD="/var/app/staging"
2020-07-18 04:50:42,970 P3836 [INFO]    export PYTHONPATH="/var/app/venv/staging-LQM1lest/bin"
2020-07-18 04:50:42,970 P3836 [INFO]    export SHLVL="4"
2020-07-18 04:50:42,970 P3836 [INFO]    export _="/bin/jq"
2020-07-18 04:50:42,970 P3836 [INFO]    Courtesy Notice: Pipenv found itself running within a virtual environment, so it will automatically use that environment, instead of creating its own for any project. You can set PIPENV_IGNORE_VIRTUALENVS=1 to force pipenv to ignore that environment and create its own instead. You can set PIPENV_VERBOSITY=-1 to suppress this warning.
2020-07-18 04:50:42,970 P3836 [INFO]    Operations to perform:
2020-07-18 04:50:42,970 P3836 [INFO]      Apply all migrations: admin, auth, contenttypes, sessions
2020-07-18 04:50:42,970 P3836 [INFO]    Running migrations:
2020-07-18 04:50:42,970 P3836 [INFO]      Applying contenttypes.0001_initial... OK
2020-07-18 04:50:42,970 P3836 [INFO]      Applying auth.0001_initial... OK
2020-07-18 04:50:42,970 P3836 [INFO]      Applying admin.0001_initial... OK
2020-07-18 04:50:42,970 P3836 [INFO]      Applying admin.0002_logentry_remove_auto_add... OK
2020-07-18 04:50:42,970 P3836 [INFO]      Applying admin.0003_logentry_add_action_flag_choices... OK
2020-07-18 04:50:42,971 P3836 [INFO]      Applying contenttypes.0002_remove_content_type_name... OK
2020-07-18 04:50:42,971 P3836 [INFO]      Applying auth.0002_alter_permission_name_max_length... OK
2020-07-18 04:50:42,971 P3836 [INFO]      Applying auth.0003_alter_user_email_max_length... OK
2020-07-18 04:50:42,971 P3836 [INFO]      Applying auth.0004_alter_user_username_opts... OK
2020-07-18 04:50:42,971 P3836 [INFO]      Applying auth.0005_alter_user_last_login_null... OK
2020-07-18 04:50:42,971 P3836 [INFO]      Applying auth.0006_require_contenttypes_0002... OK
2020-07-18 04:50:42,971 P3836 [INFO]      Applying auth.0007_alter_validators_add_error_messages... OK
2020-07-18 04:50:42,971 P3836 [INFO]      Applying auth.0008_alter_user_username_max_length... OK
2020-07-18 04:50:42,971 P3836 [INFO]      Applying auth.0009_alter_user_last_name_max_length... OK
2020-07-18 04:50:42,971 P3836 [INFO]      Applying auth.0010_alter_group_name_max_length... OK
2020-07-18 04:50:42,971 P3836 [INFO]      Applying auth.0011_update_proxy_permissions... OK
2020-07-18 04:50:42,971 P3836 [INFO]      Applying sessions.0001_initial... OK

`source $PYTHONPATH/activate` was key for me after several hours of frustration. — amucunguzi, Jan 05 '21 at 13:07
This answer [was plagiarised](https://stackoverflow.com/questions/73720169/container-commands-fail-to-migrate-django-database/73721485#73721485). — Peter Mortensen, Oct 08 '22 at 14:14
[Among others](https://meta.stackexchange.com/questions/382697/how-can-i-get-unblocked-from-answering-questions#comment1276489_382697). — Peter Mortensen, Oct 08 '22 at 15:16
@PeterMortensen Not sure what do you mean? My answer is from 2020. The answer you linked is from 2022. — Marcin, Oct 08 '22 at 16:54
I thought you would misunderstand it. That is why I wrote "This answer was plagiarised.". But that was apparently not enough. "This answer" = your answer. *Your* answer was the victim of plagiarism. Or in order words: Somebody plagiarised *your* answer. Somebody copy and pasted your answer. That somebody else is the one that plagiarised. You didn't plagiarise anything. You are not a plagiariser. Somebody else is a plagiariser. I now realise the verb tense and/or passive construction makes a difference, *is* ambiguous, and *can* be misunderstood. My fault. Sorry for the confusion. — Peter Mortensen, Oct 09 '22 at 14:29
I should have written *"This answer was plagiarised by somebody else."*, *"This answer was plagiarised in 2022."*, *"Somebody else plagiarised this answer."*, or similar. — Peter Mortensen, Oct 09 '22 at 14:34

Container command fails in Django on Elastic Beanstalk Python 3.7

1 Answers1

Linked