0

I have been attempting to deploy a web scraper written in python for the past few weeks on azure. I initially tried to do this in an azure app service by building pushing a docker image to the service. I have had success previously with this method when deploying a flask rest api. Unfortunately the time out limit of the azure app service meant that the web scraper container is terminated as it is does not give a proper response when azure attempts to get a response - this option won't work.

I have since tried setting up a windows based app service in order to create an azure WebJob however, this has the problem of being capped at python 3.6 which i believe is causing import problems. I cannot import the "requests" module which is essential for the scraper to work correctly. I have a requirements.txt inside the zip i upload for the webjob but this doesnt seem to allow the importing of this module either. Is there a way to import modules from inside the webjob?

Below is the error i am receiving in the webjob logs :

[12/19/2020 16:36:48 > 658fd3: SYS INFO] Run script 'run.py' with script host - 'PythonScriptHost'
[12/19/2020 16:36:48 > 658fd3: SYS INFO] Status changed to Running
[12/19/2020 16:36:48 > 658fd3: ERR ] Traceback (most recent call last):
[12/19/2020 16:36:48 > 658fd3: ERR ]   File "run.py", line 12, in <module>
[12/19/2020 16:36:48 > 658fd3: ERR ]     import requests
[12/19/2020 16:36:48 > 658fd3: ERR ] ModuleNotFoundError: No module named 'requests'
Jono
  • 1
  • Did you change you site packages path? – Nir Elbaz Dec 29 '20 at 11:23
  • I have not altered the site packages path – Jono Dec 29 '20 at 12:22
  • You should, The path in azure is sitepackage = "d:\home\python364x86\lib\site-packages" and you also need to install the packages there first (When the python version in the path is depends on your installed version) – Nir Elbaz Dec 29 '20 at 12:26

1 Answers1

0

Search Advanced Tools in Filter. Click on the GO Link. Then choose Debug Console and click on CMD.

Find your Source Code directory path. Then type in the path:

python -m pip install - requirements.txt

d:\ |-- home
      |-- site
          |-- wwwroot
              |-- App_Data
                  |-- jobs
                      |-- triggered
                      | -- continuous
Jeremy Caney
  • 7,102
  • 69
  • 48
  • 77
Dinesh
  • 1