Django application fails to deploy on AWS ElasticBeanstalk with 'firebase-admin' library

Question

I am deploying my python/django application to aws elasticbeanstalk. It gets deployed fine if i don't specify 'firebase-admin' library in requirements.txt. But fails to deploy if i add 'firebase-admin' no matter what version in requirements.txt as shown below:

Here's my requirements.txt:

Django==3.0.1
djangorestframework==3.11.0
psycopg2==2.7.3.1
django-cors-headers==3.2.0
pytz==2017.2
tinys3==0.1.12
apiclient==1.0.3
drf_yasg==1.17.1
google-api-python-client==1.7.3
google-auth==1.5.0
google-auth-httplib2==0.0.3
httplib2==0.11.3
django-rest-auth==0.9.5
oauth2client==4.1.2
geographiclib==1.50
geopy==1.20.0
numpy==1.15.2
pyyaml==5.3.1

And AWS elasticbeanstalk python instance details: Python 3.6 running on 64bit Amazon Linux/2.9.7

Please Help. Thanks

UPDATE: The following requirements.txt works fine with the localhost virtual environment:

Django==3.0.1
djangorestframework==3.11.0
psycopg2==2.7.3.1
django-cors-headers==3.2.0
pytz==2017.2
tinys3==0.1.12
apiclient==1.0.3
drf_yasg==1.17.1
google-api-python-client==1.7.8
google-auth==1.14.0
google-auth-httplib2==0.0.3
httplib2==0.11.3
django-rest-auth==0.9.5
oauth2client==4.1.2
geopy==1.20.0
pyyaml==5.3.1
firebase-admin==2.13.0

But it gives following error while deploying to AWS:

ERROR   Your requirements.txt is invalid. Snapshot your logs for details.
2020-04-22 00:14:47    ERROR   [Instance: i-0864d69c58c343870] Command failed on instance. Return code: 1 Output: (TRUNCATED)...)
  File "/usr/lib64/python2.7/subprocess.py", line 190, in check_call
    raise CalledProcessError(retcode, cmd)
CalledProcessError: Command '/opt/python/run/venv/bin/pip install -r /opt/python/ondeck/app/requirements.txt' returned non-zero exit status 1.

COMPLETE AWS LOG:

-------------------------------------
/var/log/eb-activity.log
-------------------------------------
          pp_opts)
        File "/usr/lib64/python3.6/distutils/unixccompiler.py", line 129, in _compile
          raise CompileError(msg)
      distutils.errors.CompileError: command 'gcc' failed with exit status 1

      During handling of the above exception, another exception occurred:

      Traceback (most recent call last):
        File "<string>", line 1, in <module>
        File "/tmp/pip-build-h4kd5_1k/grpcio/setup.py", line 400, in <module>
          cmdclass=COMMAND_CLASS,
        File "/opt/python/run/venv/local/lib/python3.6/site-packages/setuptools/__init__.py", line 144, in setup
          return distutils.core.setup(**attrs)
        File "/usr/lib64/python3.6/distutils/core.py", line 148, in setup
          dist.run_commands()
        File "/usr/lib64/python3.6/distutils/dist.py", line 955, in run_commands
          self.run_command(cmd)
        File "/usr/lib64/python3.6/distutils/dist.py", line 974, in run_command
          cmd_obj.run()
        File "/opt/python/run/venv/local/lib/python3.6/site-packages/setuptools/command/install.py", line 61, in run
          return orig.install.run(self)
        File "/usr/lib64/python3.6/distutils/command/install.py", line 593, in run
          self.run_command('build')
        File "/usr/lib64/python3.6/distutils/cmd.py", line 313, in run_command
          self.distribution.run_command(command)
        File "/usr/lib64/python3.6/distutils/dist.py", line 974, in run_command
          cmd_obj.run()
        File "/usr/lib64/python3.6/distutils/command/build.py", line 135, in run
          self.run_command(cmd_name)
        File "/usr/lib64/python3.6/distutils/cmd.py", line 313, in run_command
          self.distribution.run_command(command)
        File "/usr/lib64/python3.6/distutils/dist.py", line 974, in run_command
          cmd_obj.run()
        File "/opt/python/run/venv/local/lib/python3.6/site-packages/setuptools/command/build_ext.py", line 87, in run
          _build_ext.run(self)
        File "/usr/lib64/python3.6/distutils/command/build_ext.py", line 339, in run
          self.build_extensions()
        File "/tmp/pip-build-h4kd5_1k/grpcio/src/python/grpcio/commands.py", line 267, in build_extensions
          "Failed `build_ext` step:\n{}".format(formatted_exception))
      commands.CommandError: Failed `build_ext` step:
      Traceback (most recent call last):
        File "/usr/lib64/python3.6/distutils/unixccompiler.py", line 127, in _compile
          extra_postargs)
        File "/tmp/pip-build-h4kd5_1k/grpcio/src/python/grpcio/_spawn_patch.py", line 54, in _commandfile_spawn
          _classic_spawn(self, command)
        File "/usr/lib64/python3.6/distutils/ccompiler.py", line 909, in spawn
          spawn(cmd, dry_run=self.dry_run)
        File "/usr/lib64/python3.6/distutils/spawn.py", line 36, in spawn
          _spawn_posix(cmd, search_path, dry_run=dry_run)
        File "/usr/lib64/python3.6/distutils/spawn.py", line 159, in _spawn_posix
          % (cmd, exit_status))
      distutils.errors.DistutilsExecError: command 'gcc' failed with exit status 1

      During handling of the above exception, another exception occurred:

      Traceback (most recent call last):
        File "/tmp/pip-build-h4kd5_1k/grpcio/src/python/grpcio/commands.py", line 262, in build_extensions
          build_ext.build_ext.build_extensions(self)
        File "/usr/lib64/python3.6/distutils/command/build_ext.py", line 448, in build_extensions
          self._build_extensions_serial()
        File "/usr/lib64/python3.6/distutils/command/build_ext.py", line 473, in _build_extensions_serial
          self.build_extension(ext)
        File "/opt/python/run/venv/local/lib/python3.6/site-packages/setuptools/command/build_ext.py", line 208, in build_extension
          _build_ext.build_extension(self, ext)
        File "/usr/lib64/python3.6/distutils/command/build_ext.py", line 533, in build_extension
          depends=ext.depends)
        File "/usr/lib64/python3.6/distutils/ccompiler.py", line 574, in compile
          self._compile(obj, src, ext, cc_args, extra_postargs, pp_opts)
        File "/tmp/pip-build-h4kd5_1k/grpcio/src/python/grpcio/commands.py", line 246, in new_compile
          pp_opts)
        File "/usr/lib64/python3.6/distutils/unixccompiler.py", line 129, in _compile
          raise CompileError(msg)
      distutils.errors.CompileError: command 'gcc' failed with exit status 1


      ----------------------------------------
  Command "/opt/python/run/venv/bin/python3.6 -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-h4kd5_1k/grpcio/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-okr6a_v3-record/install-record.txt --single-version-externally-managed --compile --install-headers /opt/python/run/venv/include/site/python3.6/grpcio" failed with error code 1 in /tmp/pip-build-h4kd5_1k/grpcio/
  You are using pip version 9.0.1, however version 20.0.2 is available.
  You should consider upgrading via the 'pip install --upgrade pip' command.
  2020-04-22 23:07:23,479 ERROR    Error installing dependencies: Command '/opt/python/run/venv/bin/pip install -r /opt/python/ondeck/app/requirements.txt' returned non-zero exit status 1
  Traceback (most recent call last):
    File "/opt/elasticbeanstalk/hooks/appdeploy/pre/03deploy.py", line 22, in main
      install_dependencies()
    File "/opt/elasticbeanstalk/hooks/appdeploy/pre/03deploy.py", line 18, in install_dependencies
      check_call('%s install -r %s' % (os.path.join(APP_VIRTUAL_ENV, 'bin', 'pip'), requirements_file), shell=True)
    File "/usr/lib64/python2.7/subprocess.py", line 190, in check_call
      raise CalledProcessError(retcode, cmd)
  CalledProcessError: Command '/opt/python/run/venv/bin/pip install -r /opt/python/onde

ck/app/requirements.txt' returned non-zero exit status 1 (Executor::NonZeroExitStatus)

When posting to Stack Overflow, please do not provide pictures with text. It's better to copy the text into the question itself, and format it so it's easy to ready, copy, and search. — Doug Stevenson, Apr 18 '20 at 23:37
Please help guys it's been around 36 hours that i am stuck here. — Muhammad Sadiq Alvi, Apr 20 '20 at 13:56
I don't see `firebase-admin` in the given `requirements.txt` file. How did you specify it? — Hiranya Jayathilaka, Apr 20 '20 at 18:54
Yes that's the requirements,txt that works: I added firebase-admin to it like: firebase-admin==4.0.0 — Muhammad Sadiq Alvi, Apr 20 '20 at 23:39

score 1 · Accepted Answer · answered Apr 21 '20 at 20:56

1

I tried your requirements.txt file locally, which resulted in the following errors.

ERROR: google-api-core 1.17.0 has requirement google-auth<2.0dev,>=1.14.0, but you'll have google-auth 1.5.0 which is incompatible.
ERROR: google-cloud-storage 1.27.0 has requirement google-auth<2.0dev,>=1.11.0, but you'll have google-auth 1.5.0 which is incompatible.
ERROR: firebase-admin 4.0.0 has requirement google-api-python-client>=1.7.8, but you'll have google-api-python-client 1.7.3 which is incompatible.

That's most likely what's happening in your deployment environment as well, except it's not logging the errors. Try resolving the above dependency conflicts, and try again.

answered Apr 21 '20 at 20:56

Hiranya Jayathilaka

7,180
1
23
34

First of all thanks:) I have tried changing versions and that works well on localhost virtual env but give error on AWS. I have added details below the Update heading on the post. Please have a look. – Muhammad Sadiq Alvi Apr 22 '20 at 00:41
Can't tell for sure. It is still most likely a dependency error, but since its hiding the `pip` error log, we can't know for sure what's going on. Also I see you've downgraded `firebase-admin`. Instead I'd recommend upgrading it and the dependencies to the latest. – Hiranya Jayathilaka Apr 22 '20 at 22:56
I downgraded firebase-admin since higher version of it were giving errors in virtual environment. I have updated the post with Complete AWS logs below the COMPLETE AWS LOGS heading. Please check :) – Muhammad Sadiq Alvi Apr 22 '20 at 23:13
I really wouldn't recommend that. Please try with the latest dependencies. The error messages I posted above states clearly that you need to also upgrade `google-auth` and `google-api-python-client`. Your log indicates a grpcio compilation error, which most likely happens during Firestore installation. This may be something you can avoid by using the latest dependencies. But if the problem continues to occur, try reaching out to the AWS community to figure out how to install packages like `google-cloud-firestore` on EBS. – Hiranya Jayathilaka Apr 22 '20 at 23:22
A few searches revealed many similar issues with Python+gcc+EBS. You might be experiencing a similar situation. If upgrading dependencies doesn't help try the workarounds in https://stackoverflow.com/questions/48334158/aws-elastic-beanstalk-failed-to-install-python-package-using-requirements-txt-gi – Hiranya Jayathilaka Apr 22 '20 at 23:32
Somehow after upgrading versions now it's successfully deployed after some days. Thanks a lot for all the help :) – Muhammad Sadiq Alvi Apr 26 '20 at 16:28

Django application fails to deploy on AWS ElasticBeanstalk with 'firebase-admin' library

1 Answers1