I'm trying to deploy a little python script using Selenium on my GCP Virtual Machine following this tutorial. Unfortunately, I can't pass the requirements.txt when building the container image. Indeed as one can read:
mikempc3@instance-1:~$ sudo docker build --tag my-python-app .
Sending build context to Docker daemon 387.1MB
Step 1/6 : FROM python:alpine3.7
alpine3.7: Pulling from library/python
48ecbb6b270e: Pull complete
692f29ee68fa: Pull complete
6439819450d1: Pull complete
3c7be240f7bf: Pull complete
ca4b349df8ed: Pull complete
Digest: sha256:35f6f83ab08f98c727dbefd53738e3b3174a48b4571ccb1910bae480dcdba847
Status: Downloaded newer image for python:alpine3.7
---> 00be2573e9f7
Step 2/6 : COPY . /app
---> d8ae78db92f8
Step 3/6 : WORKDIR /app
---> Running in 3f6269c90e90
Removing intermediate container 3f6269c90e90
---> 717897752d1d
Step 4/6 : RUN pip install -r requirements.txt
---> Running in 061da28ee5cb
Collecting selenium (from -r requirements.txt (line 1))
Downloading https://files.pythonhosted.org/packages/80/d6/4294f0b4bce4de0abf13e17190289f9d0613b0a44e5dd6a7f5ca98459853/selenium-3.141.0-py2.py3-none-any.whl (904kB)
Collecting pandas (from -r requirements.txt (line 2))
Downloading https://files.pythonhosted.org/packages/6f/29/32ff85413724ffa7cc8d52373f93c2ef1cb197ffd0c7b1b10d36452dd0ca/pandas-1.1.0.tar.gz (5.2MB)
Installing build dependencies: started
Installing build dependencies: finished with status 'error'
Complete output from command /usr/local/bin/python /usr/local/lib/python3.7/site-packages/pip install --ignore-installed --no-user --prefix /tmp/pip-build-env-ou2zb3ns/overlay --no-warn-script-l
ocation --no-binary :none: --only-binary :none: -i https://pypi.org/simple -- setuptools wheel Cython>=0.29.16,<3 "numpy==1.15.4; python_version=='3.6' and platform_system!='AIX'" "numpy==1.15.4;
python_version=='3.7' and platform_system!='AIX'" "numpy==1.17.3; python_version>='3.8' and platform_system!='AIX'" "numpy==1.16.0; python_version=='3.6' and platform_system=='AIX'" "numpy==1.16.0
; python_version=='3.7' and platform_system=='AIX'" "numpy==1.17.3; python_version>='3.8' and platform_system=='AIX'":
Ignoring numpy: markers 'python_version == "3.6" and platform_system != "AIX"' don't match your environment
Ignoring numpy: markers 'python_version >= "3.8" and platform_system != "AIX"' don't match your environment
Ignoring numpy: markers 'python_version == "3.6" and platform_system == "AIX"' don't match your environment
Ignoring numpy: markers 'python_version == "3.7" and platform_system == "AIX"' don't match your environment
Ignoring numpy: markers 'python_version >= "3.8" and platform_system == "AIX"' don't match your environment
Collecting setuptools
Downloading https://files.pythonhosted.org/packages/8e/11/9e10f1cad4518cb307b484c255cae61e97f05b82f6d536932b1714e01b47/setuptools-49.2.0-py3-none-any.whl (789kB)
Collecting wheel
Downloading https://files.pythonhosted.org/packages/8c/23/848298cccf8e40f5bbb59009b32848a4c38f4e7f3364297ab3c3e2e2cd14/wheel-0.34.2-py2.py3-none-any.whl
Collecting Cython<3,>=0.29.16
Downloading https://files.pythonhosted.org/packages/ad/4b/9e53bcce3c959fd0db143626e573210bba07be810fe8d7296373948c4183/Cython-0.29.21-py2.py3-none-any.whl (974kB)
Collecting numpy==1.15.4
Downloading https://files.pythonhosted.org/packages/2d/80/1809de155bad674b494248bcfca0e49eb4c5d8bee58f26fe7a0dd45029e2/numpy-1.15.4.zip (4.5MB)
Building wheels for collected packages: numpy
Building wheel for numpy (setup.py): started
Building wheel for numpy (setup.py): finished with status 'error'
Complete output from command /usr/local/bin/python -u -c "import setuptools, tokenize;__file__='/tmp/pip-install-800mjafl/numpy/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.rea
d().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" bdist_wheel -d /tmp/pip-wheel-q_b64mdo --python-tag cp37:
Running from numpy source directory.
blas_opt_info:
blas_mkl_info:
customize UnixCCompiler
libraries mkl_rt not found in ['/usr/local/lib', '/usr/lib']
NOT AVAILABLE
blis_info:
customize UnixCCompiler
libraries blis not found in ['/usr/local/lib', '/usr/lib']
NOT AVAILABLE
openblas_info:
customize UnixCCompiler
customize UnixCCompiler
libraries openblas not found in ['/usr/local/lib', '/usr/lib']
NOT AVAILABLE
atlas_3_10_blas_threads_info:
Setting PTATLAS=ATLAS
customize UnixCCompiler
libraries tatlas not found in ['/usr/local/lib', '/usr/lib']
NOT AVAILABLE
atlas_3_10_blas_info:
customize UnixCCompiler
libraries satlas not found in ['/usr/local/lib', '/usr/lib']
NOT AVAILABLE
atlas_blas_threads_info:
Setting PTATLAS=ATLAS
customize UnixCCompiler
libraries ptf77blas,ptcblas,atlas not found in ['/usr/local/lib', '/usr/lib']
NOT AVAILABLE
atlas_blas_info:
customize UnixCCompiler
libraries f77blas,cblas,atlas not found in ['/usr/local/lib', '/usr/lib']
NOT AVAILABLE
accelerate_info:
NOT AVAILABLE
/tmp/pip-install-800mjafl/numpy/numpy/distutils/system_info.py:625: UserWarning:
Atlas (http://math-atlas.sourceforge.net/) libraries not found.
Directories to search for the libraries can be specified in the
numpy/distutils/site.cfg file (section [atlas]) or by setting
the ATLAS environment variable.
self.calc_info()
blas_info:
customize UnixCCompiler
libraries blas not found in ['/usr/local/lib', '/usr/lib']
NOT AVAILABLE
...
File "/usr/local/lib/python3.7/distutils/core.py", line 148, in setup
dist.run_commands()
File "/usr/local/lib/python3.7/distutils/dist.py", line 966, in run_commands
self.run_command(cmd)
File "/usr/local/lib/python3.7/distutils/dist.py", line 985, in run_command
cmd_obj.run()
File "/tmp/pip-install-800mjafl/numpy/numpy/distutils/command/install.py", line 62, in run
r = self.setuptools_run()
File "/tmp/pip-install-800mjafl/numpy/numpy/distutils/command/install.py", line 36, in setuptools_run
return distutils_install.run(self)
File "/usr/local/lib/python3.7/distutils/command/install.py", line 545, in run
self.run_command('build')
File "/usr/local/lib/python3.7/distutils/cmd.py", line 313, in run_command
self.distribution.run_command(command)
File "/usr/local/lib/python3.7/distutils/dist.py", line 985, in run_command
cmd_obj.run()
File "/tmp/pip-install-800mjafl/numpy/numpy/distutils/command/build.py", line 47, in run
old_build.run(self)
File "/usr/local/lib/python3.7/distutils/command/build.py", line 135, in run
self.run_command(cmd_name)
File "/usr/local/lib/python3.7/distutils/cmd.py", line 313, in run_command
self.distribution.run_command(command)
File "/usr/local/lib/python3.7/distutils/dist.py", line 985, in run_command
cmd_obj.run()
File "/tmp/pip-install-800mjafl/numpy/numpy/distutils/command/build_src.py", line 148, in run
self.build_sources()
File "/tmp/pip-install-800mjafl/numpy/numpy/distutils/command/build_src.py", line 159, in build_sources
self.build_library_sources(*libname_info)
File "/tmp/pip-install-800mjafl/numpy/numpy/distutils/command/build_src.py", line 294, in build_library_sources
sources = self.generate_sources(sources, (lib_name, build_info))
File "/tmp/pip-install-800mjafl/numpy/numpy/distutils/command/build_src.py", line 377, in generate_sources
source = func(extension, build_dir)
File "numpy/core/setup.py", line 666, in get_mathlib_info
raise RuntimeError("Broken toolchain: cannot link a simple C program")
RuntimeError: Broken toolchain: cannot link a simple C program
----------------------------------------
Command "/usr/local/bin/python -u -c "import setuptools, tokenize;__file__='/tmp/pip-install-800mjafl/numpy/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\
n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-record-sgbls4z_/install-record.txt --single-version-externally-managed --prefix /tmp/pip-build-env-ou2zb3ns/overlay -
-compile" failed with error code 1 in /tmp/pip-install-800mjafl/numpy/
You are using pip version 19.0.1, however version 20.2 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
----------------------------------------
Command "/usr/local/bin/python /usr/local/lib/python3.7/site-packages/pip install --ignore-installed --no-user --prefix /tmp/pip-build-env-ou2zb3ns/overlay --no-warn-script-location --no-binary :n
one: --only-binary :none: -i https://pypi.org/simple -- setuptools wheel Cython>=0.29.16,<3 "numpy==1.15.4; python_version=='3.6' and platform_system!='AIX'" "numpy==1.15.4; python_version=='3.7'
and platform_system!='AIX'" "numpy==1.17.3; python_version>='3.8' and platform_system!='AIX'" "numpy==1.16.0; python_version=='3.6' and platform_system=='AIX'" "numpy==1.16.0; python_version=='3.7
' and platform_system=='AIX'" "numpy==1.17.3; python_version>='3.8' and platform_system=='AIX'"" failed with error code 1 in None
You are using pip version 19.0.1, however version 20.2 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
The command '/bin/sh -c pip install -r requirements.txt' returned a non-zero code: 1
Here is my requirements.txt file:
selenium
pandas
numpy
collections
json
time
requests
And here is the file I'm trying to contenerize:
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.common.exceptions import ElementClickInterceptedException
from selenium.common.exceptions import NoSuchElementException
from selenium.webdriver.chrome.options import Options
import pandas as pd
import numpy as np
from collections import defaultdict
import json
import time
import requests
from requests.exceptions import ConnectionError
# Define Browser Options
chrome_options = Options()
chrome_options.add_argument("--headless") # Hides the browser window
# Reference the local Chromedriver instance
chrome_path = r"C:\Programs\chromedriver.exe"
driver = webdriver.Chrome(executable_path=chrome_path, options=chrome_options)
df = pd.read_csv('path/to/file')
tradable = []
print(len(df['Ticker']))
for ticker in df['Ticker']:
print("ticker: ", ticker)
location = "https://www.etoro.com/markets/" + ticker.lower()
try:
request = requests.get(location)
driver.get(location)
time.sleep(2)
current_url = driver.current_url
if current_url == location:
tradable.append(ticker)
else:
print("no page but request= ", request)
except ConnectionError:
print('Ticker isn\'t tradable')
else:
tradable.append(ticker)
Here is my Dockerfile:
FROM python:alpine3.7
COPY . /app
WORKDIR /app
RUN pip install -r requirements.txt
EXPOSE 5000
CMD python ./find_tradable.py
Here are my os name and version:
mikempc3@instance-1:~$ cat /etc/os-release
PRETTY_NAME="Debian GNU/Linux 9 (stretch)"
NAME="Debian GNU/Linux"
VERSION_ID="9"
VERSION="9 (stretch)"
VERSION_CODENAME=stretch
ID=debian
HOME_URL="https://www.debian.org/"
SUPPORT_URL="https://www.debian.org/support"
BUG_REPORT_URL="https://bugs.debian.org/"
Here is my Linux kernel version:
mikempc3@instance-1:~$ uname -r
4.9.0-12-amd64
Update
I'm trying Serhii Rohoza's answer.
I have updated my requirements.txt file, I don't have the same issue but now I have an error when trying the command to build the image from the Dockerfile:
mikempc3@instance-1:~$ sudo docker build --tag my-python-app:1 .
Sending build context to Docker daemon 387.1MB
Step 1/6 : FROM python:python3.7-slim
manifest for python:python3.7-slim not found: manifest unknown: manifest unknown
So I tried:
mikempc3@instance-1:~$ sudo docker pull python:3.7-slim
3.7-slim: Pulling from library/python
6ec8c9369e08: Already exists
401b5acb42e6: Already exists
2e487de6656a: Pull complete
519de614852e: Pull complete
a3d1a61e090c: Pull complete
Digest: sha256:47081c7bca01b314e26c64d777970d46b2ad7049601a6f702d424881af9f2738
Status: Downloaded newer image for python:3.7-slim
docker.io/library/python:3.7-slim
mikempc3@instance-1:~$ sudo docker build --tag my-python-app:1 .
Sending build context to Docker daemon 387.1MB
Step 1/6 : FROM python:python3.7-slim
manifest for python:python3.7-slim not found: manifest unknown: manifest unknown
