Python file writing is not working inside docker

Question

I'm a trying to dockerize a python script in which file writing of a text file and CSV file is there. but it is not working, the file is not getting generated.

For reference my code is

FROM python:3



ADD ./HMS.py /
ADD ./constants.py /
ADD ./Docker.py /
ADD ./SNT.py /
ADD ./Utility.py /


RUN pip install --trusted-host pypi.org --trusted-host files.pythonhosted.org docker
RUN pip install --trusted-host pypi.org --trusted-host files.pythonhosted.org grepfunc
RUN pip install --trusted-host pypi.org --trusted-host files.pythonhosted.org requests


CMD [ "python", "./HMS.py" ]

However, when I run the python script outside am able to generate the files. May I know why it is not getting generated inside the work directory.

UPDATE

    def write_log(self, filename, message, type='info'):
        with open(filename, 'a') as log:
            log.write('\n' + message)

    def write_csv(self, filepath, data):
        with open(filepath, 'a', newline='') as file:
            writer = csv.writer(file)
            writer.writerow(data)

constant file contain the path

LOG_FILE='./snt_alarm_log.txt'
COMPONENT_NAME='SNT Alarm'
DB_FILE='./snt.csv'

What flags are you using to start the docker container? Can you paste your entire docker run command? — prithajnath, Jun 15 '20 at 19:00
How are you running the container? What is the output? If `HMS.py` is simple, let's see it. Otherwise, try to reproduce the issue using a simple script that just writes a file. Also, I think you have some unnecessary `ADD` commands there. — chash, Jun 15 '20 at 19:00
We will write some msg and status into txt file and also in a CSV file — Aaditya R Krishnan, Jun 15 '20 at 19:03
The output of HMS.py to the console is just UP or DOWN. But we are writing some message and status into the text file and CSV file — Aaditya R Krishnan, Jun 15 '20 at 19:06
If the main goal of your process is to write files to the host filesystem, Docker's filesystem isolation and other setup might get in your way more than simplify things. A Python virtual environment might be a better match, with tools like `pipenv` that can help set it up. — David Maze, Jun 15 '20 at 19:49

prithajnath · Answer 1 · 2020-06-15T21:04:47.993

4

Containers are ephemeral by nature. If you want to retain files generated inside a container you need to mount your host file system to the file system of the container. There are many ways to mount a volume, but a common way is a bind mount

docker run -v /path/to/host:/project sntalarmdocker_snt

Whatever gets saved to /project will be visible in /path/to/host even after the container is killed

edited Jun 15 '20 at 21:04

answered Jun 15 '20 at 19:07

prithajnath

2,000
14
17

How to find the path to container ? – Aaditya R Krishnan Jun 15 '20 at 19:13
It is typically your `WORKDIR`, so in your case it will be `/project`. I updated the original post – prithajnath Jun 15 '20 at 19:15
1

But still am not able to get the files which need to generated by python – Aaditya R Krishnan Jun 15 '20 at 19:36
Actually sntalarmdocker_snt is a docker image not container – Aaditya R Krishnan Jun 15 '20 at 19:36
1

Even am not able the see log file too – Aaditya R Krishnan Jun 15 '20 at 19:41
It is working fine am able to see the output in the console – Aaditya R Krishnan Jun 15 '20 at 19:43
1

When run the image /path/to/host is still empty – Aaditya R Krishnan Jun 15 '20 at 19:44
I recommend shelling into the container and snooping around. Try `docker run -it --entrypoint /bin/bash -v /path/to/host:/project `. This should give you shell access to the running container. Then run your command `python HMS.py` and see if the desired file is getting generated in the container. If so then the file should exist in your host file system if the volume mount was successful – prithajnath Jun 15 '20 at 21:03
Is it possible to generate a file outside the container location – Aaditya R Krishnan Jun 16 '20 at 03:06
`Checking last update timestamp for ONT Log Traceback (most recent call last): File "./AMS.py", line 3, in ONT() File "/ONT.py", line 12, in __init__ self.build_test_case() File "/ONT.py", line 16, in build_test_case self.generate_log('Checking last update timestamp for ONT Log') File "/ONT.py", line 80, in generate_log self.util.write_log(constants.LOG_FILE,msg) File "/Utility.py", line 76, in write_log with open(filename, 'a') as log: FileNotFoundError: [Errno 2] No such file or directory: '/home/edward/checker_docker/ont_alarm_log.txt' ` – Aaditya R Krishnan Jun 16 '20 at 03:07
Actually I need to read a file from outside the location of docker & write two files into outer location – Aaditya R Krishnan Jun 16 '20 at 03:09
Any file you need to read inside a container needs to be present inside the container. Is that error from inside the container? That path looks like it belongs in the host – prithajnath Jun 16 '20 at 03:30
1

I tried by keeping the file inside the docker image location and then it didn't raise any error. Is this file write not happened because of the base image as python. Do I need an another DockerFile for Ubuntu – Aaditya R Krishnan Jun 16 '20 at 14:18

Nate Anderson · Answer 2 · 2023-07-17T21:40:46.630

I am on a Windows machine with a Git bash trying to follow this tutorial. When I run this command to execute a Python script to write a file, I don't see the file in my local directory!

$ docker run --volume="/c/Users/nanders2/.python-xml-docker":/app pythonxml:latest

The issue is with Git Bash's Posix path conversion as described here.

This is due to Mingw/msys2 doing automatic path conversion; see http://www.mingw.org/wiki/Posix_path_conversion

You can disable this by prefixing paths with // or setting the MSYS_NO_PATHCONV=1 environment variable (see https://stackoverflow.com/a/34386471)

Because this path conversion happens before docker receives the value, its not something we can fix in docker.

The solution that works for me is to use double-forward-slash in my path

$ docker run --volume="//c//Users//nanders2//.python-xml-docker":/app pythonxml:latest

Or I could use Powershell (i.e. stop using Git Bash) instead, but I don't like that solution.

Slightly better solution is to use Docker Desktop itself to run the container, and manually specify the volume/mount arguments.

I started realizing the docker + Git Bash was struggling when it logged errors like this:

docker: Error response from daemon: the working directory 'C:/Program Files/Git/app' is invalid, it needs to be an absolute path.

Python file writing is not working inside docker

UPDATE

2 Answers2

Linked