0

I use scrapy-splash with docker. In Dockerfile I have this line to export the result in a .jl .

CMD ["scrapy", "crawl", "quotesjs", "-o", "quote.jl"]

When I run docker-compose build and docker-compose up, the log informs me that:

scrapy1      | 2017-12-18 00:00:00 [scrapy.extensions.feedexport] INFO: Stored jl feed (10 items) in: quote.jl

I don't see any quote.jl in my local folder (where the Dockerfile and the scrapy project is), so I guesse it should be in my container. I trie to cp the content of the container with this command but without success.

docker cp containerID:. ./copy_of_container

How can I retrieve the quote.jl file.

I am on Windows10 and I use Docker for Windows

My dockerfile

FROM python:alpine
RUN apk --update add libxml2-dev libxslt-dev libffi-dev gcc musl-dev libgcc openssl-dev curl bash
RUN pip install scrapy scrapy-splash scrapy-fake-useragent
ADD . /scraper
WORKDIR /scraper

CMD ["scrapy", "crawl", "apkmirror", "-o", "apkmirror.jl"]
J. Does
  • 785
  • 3
  • 10
  • 23
  • you can "assign" local folder as folder in docker and then it write in your folder instead folder inside docker but probably you can do only when you create new image (which later runs as container). – furas Dec 18 '17 at 16:53
  • [How to mount host directory in docker container?](https://stackoverflow.com/questions/23439126/how-to-mount-host-directory-in-docker-container) – furas Dec 18 '17 at 16:54
  • Thanks furas! On the link provided, it seems that something like `docker run -v /host/directory:/container/directory -other -options image_name command_to_run` could do what I want but I am not sure how to implement it especially the image_name and command_to_run. I simply create the docker image by doing a `docker-compose build` and then `docker-compose up`. The parameters are in the Dockerfile and docker-compose.yml as explained here: https://stackoverflow.com/a/45105145/7227370. – J. Does Dec 18 '17 at 17:01
  • as shown in previous link you could use `ADD` in `Dockerfile` to asssign/link your local folder as folder in docker and then it can write directly to your folder. – furas Dec 18 '17 at 17:07
  • as for `command_to_run` - if you run without `command_to_run` then it should run internal `bash` and you may use `ls` to display files in folder. See examples in doc: [docker run](https://docs.docker.com/engine/reference/commandline/run/#parent-command) – furas Dec 18 '17 at 17:12
  • Thanks for all this @furas! I have added my dockerfile. I have replace the `COPY . /scraper` by `ADD . /scraper`. but still I dont' see any `apkmirror.jl` in my local folder – J. Does Dec 18 '17 at 17:13

0 Answers0