Running Scrapy command in Docker Container, View my Dockerfile below

Question

FROM python:3.8

MAINTAINER Darix SAMANI
COPY . /app
WORKDIR /app


RUN pip install --upgrade pip
RUN pip install -r requirements.txt

RUN pip install virtualenv 
RUN virtualenv env
 
SHELL ["env/bin/activate"]

RUN cd ./pdfdrive

ENTRYPOINT ["scrapy", "crawl", "pdfdrive"]

when a run docker run pdfdrive:latest

i have this error

Scrapy 2.10.0 - no active project

Unknown command: crawl

Use "scrapy" to see available commands

You're not in the scrapy project directory ([unknown command: crawl error](https://stackoverflow.com/questions/10123104/unknown-command-crawl-error)), partly because `RUN cd` doesn't actually do anything ([difference between RUN cd and WORKDIR in Dockerfile](https://stackoverflow.com/questions/58847410/difference-between-run-cd-and-workdir-in-dockerfile)). — David Maze, Aug 12 '23 at 10:03

score 0 · Answer 1 · answered Aug 12 '23 at 10:03

0

It looks like you have not started you project yet. From the documentation and some testing you should be using this command before a crawl:

scrapy startproject pdfdrive /pdfdrive

Make sure your /pdfdrive is your working directory and the scraper is located in that directory and you should be ready to go.

answered Aug 12 '23 at 10:03

My Digital life

516
5
7

The project work well local I want to dockerize my project – Darix SAMANI SIEWE Aug 12 '23 at 14:04
Change your workdir to `WORKDIR /app/pdfdrive`. Most likely thats the only thing you need to do to fix it. Also you don't need a venv in a docker container, its only needed for local development. – My Digital life Aug 12 '23 at 14:19

Running Scrapy command in Docker Container, View my Dockerfile below

1 Answers1