How to force python to write a file before exiting the program?

Question

I already checked this answer but it didn't work. I am looking for a way to force python to write a file (let's call it file_A.py) before exiting the program. I tried the suggested approach in the above answer, but it only worked when the destination file existed and we add more content. In my situation, the file does not exist and it is supposed to be generated before the program exits normally. The reason is that the generated file has to be used in the following steps of the program.

What I've coded is as follows:

with open('file_A.py', 'w') as f:
    f.write(content)
    f.flush()
    os.fsync(f.fileno())

But file_A.py was not generated before the program finishes execution normally. How do I fix it?

What do you mean by "finishes execution"? There's a big difference between "exited normally after the code shown was executed" and "OS used SIGKILL before the code shown could execute". — chepner, Mar 16 '23 at 17:28
Better use an absolute path because a relative one may not point to the expected location. — Michael Butscher, Mar 16 '23 at 17:28
Files are flushed when closed, so `flush` typically is not needed unless you want to force data to be written before it's closed. There's virtually no difference between `with open(...) as f: f.write()` and `with open(...) as f: f.write(); f.flush()`, because the latter is just going to be executing `f.flush` while the former would be closing (and flushing) the file. — chepner, Mar 16 '23 at 17:30
You are probably going to have to figure out why the file *isn't* being created before you can fix the problem, because under normal circumstances none of this extra code is necessary. — chepner, Mar 16 '23 at 17:32
Based on your description, what happens if you open and immediately close the file, just to make sure it exists, *then* open it again and use the code that works for existing files? — chepner, Mar 16 '23 at 17:33
You should leave that in, though, because if you continue to use `fsync` with a buffered file, `fsync`'s documentation says you *do* need to call `flush` first. My comment didn't make that clear. — chepner, Mar 16 '23 at 17:35
Have you considered the *atexit* module? Also, the os.fsync() call shouldn't be necessary — DarkKnight, Mar 16 '23 at 17:36
I'm not saying you don't have a problem, but without knowing exactly *why* the file isn't being created, it's going to be hard to say what code you need to ensure it. — chepner, Mar 16 '23 at 17:36

score 1 · Answer 1 · answered Mar 16 '23 at 17:43

If you want to ensure that something (whatever that something is) happens when the Python interpreter exits you can utilise the atexit module.

Here's an example:

import atexit
import sys

some_data = 'Hello world!\n'

def cleanup():
    try:
        with open('/Volumes/G-Drive/atexit.txt', 'w') as f:
            f.write(some_data)
    except Exception as e:
        print(e, file=sys.stderr)
    
atexit.register(cleanup)

# Python interpreter terminates here and the previously registered function will be called

Note that cleanup() is not explicitly called in the main program - it is merely registered

score 0 · Answer 2 · answered Mar 16 '23 at 18:00

0

fsync will only apply to data that has been written to disk. Python output is buffered, so the data isn't necessarily written to disk when write is finished. Put f.flush() after the write.

answered Mar 16 '23 at 18:00

Mark Ransom

299,747
42
398
622

It was mentioned in the comments as well. Still with `f.flush()` it doesn't work. – Coder Mar 16 '23 at 18:07
@Coder I can see that you've updated the question. I cannot conceive a situation where that wouldn't work, the file should be created immediately with `open` even if you never write to it. – Mark Ransom Mar 16 '23 at 18:20

David · Answer 3 · 2023-03-16T18:41:05.923

If you want to create and write into a file while the code is still running I suggest using an outside function in order to solve it.

The "outside function" would be a function written in another file.

I have solved it like this:

outsidefuntion.py

import sys
def create_file(file_name, ending, text):
    try:
        with open('{}.{}'.format(file_name, ending), 'w') as f:
            f.write("{}".format(text))
            f.flush()
    except Exception as e:
        print(e, file=sys.stderr)
    else:
        print('code is written')

main.py

from datetime import datetime
import outsidefunctions as of # assuming the main and the 'outsidefunctions ' are in the same directory. 
import time

logger = str(datetime.now())[:10]

# an example of how to use it.
for i in range(1000):
    print(i) 
    time.sleep(0.1)

    if i == 10:
        of.create_file(logger, 'txt', '{} was done ...'.format(logger))
        continue

As you said, it will be written only then the code will end, so you can create another file that will run for a short time only to write into a file, your code will be interrupted for a little time (which would have happened anyways). But it will write it into a file while the code is still running.

How to force python to write a file before exiting the program?

3 Answers3