Not Writing in File Python

Question

I have a Threaded class in which I am trying to append data but it is not doing anything, no error no success. The following is the basic structure of code

from threading import Thread

class ABC:
   def func1(self, user):
       # Do processing
       self.func2(lst) # lst Generated in processing

   def func2(self, lst):
        thrd_list = []
        for ele in lst:
            x = Thread(target=self.func3, args(ele, ))
            x.start()
            thrd_list.append(x)
        for thrd in thrd_list:
            thrd.join()

   def func3(self, ele):
       # Do some stuff and if successful write to file
       OUTPUT.write(f"{ele}\n")

with open("users.txt", "r") as f:
     users = f.readlines()
OUTPUT = open("result.txt", "a")
thrd_list = []
for user in users: 
    new_obj = ABC()
    x = Thread(target=new_obj.func1, args(user, ))
    x.start()
    thrd_list.append(x)
for thrd in thrd_list:
    thrd.join()
OUTPUT.close()

The Data is being written on encounter of OUTPUT.close(). I want it to append as it goes so there is no data loss due to crash or bugs

Please read [mre] and make sure the example can run. As it stands, `x = Thread(target=func3, args(ele, ))` will give a `NameError`; classes in Python do not create a scope, so the names of methods are not automatically available to each other - they need to be looked up either explicitly in the class, or upon an instance. — Karl Knechtel, Aug 23 '23 at 02:41
"The Data is being written on encounter of OUTPUT.close(). I want it to append as it goes so there is no data loss due to crash or bugs" To be clear: the problem is that you expect the file contents to be updated immediately when `OUTPUT.write` happens, but instead you only see a change when `OUTPUT.close` happens? That's nothing to do with threading, then; please see the linked duplicate. — Karl Knechtel, Aug 23 '23 at 02:42
@KarlKnechtel I apologize for that. I am using self in my original code forgot here — farhan jatt, Aug 23 '23 at 02:46

score 1 · Accepted Answer · answered Aug 23 '23 at 02:42

1

The data being written to the file is being buffered, and only when the buffer is full or the file is closed is the buffered data written to the file.

Try changing the buffering to be line-buffered:

OUTPUT = open("result.txt", "a", buffering=1)

See https://docs.python.org/3/library/functions.html#open for more details.

answered Aug 23 '23 at 02:42

Logitude

61
4

Your solution is better than mine, thanks. Upvote :) – farhan jatt Aug 23 '23 at 02:44
Could I ask you to contribute this information [at the canonical](https://stackoverflow.com/questions/9824806)? – Karl Knechtel Aug 23 '23 at 02:47
[Done!](https://stackoverflow.com/a/76958028/9970366) – Logitude Aug 23 '23 at 03:18

score 0 · Answer 2 · answered Aug 23 '23 at 02:43

Silly question from me, on research I found that the append mode writes data on encounter of File.close(). This is how the append mode work in python.

from threading import Thread, Lock

class ABC:

    def __init__():
        self.lock = Lock()

    def func1(user):
        # Do processing
        func2(lst) # lst Generated in processing

    def func2(lst):
        thrd_list = []
        for ele in lst:
            x = Thread(target=func3, args(ele, ))
            x.start()
            thrd_list.append(x)
        for thrd in thrd_list:
            thrd.join()

    def func3(ele):
        # Do some stuff and if successful write to file
        self.lock.aquire()
        with open("result.txt", "a") as f:
            f.write(f"{ele}\n")
        self.lock.release()

with open("users.txt", "r") as f:
    users = f.readlines()
thrd_list = []
for user in users: 
    new_obj = ABC()
    x = Thread(target=new_obj.func1, args(user, ))
    x.start()
    thrd_list.append(x)
for thrd in thrd_list:
    thrd.join()

The solution is to open/close file immediately. Used the Thread lock so two or more threads do not try to write in the same file.

score -1 · Answer 3 · answered Aug 23 '23 at 02:28

from threading import Thread

class ABC:
    @staticmethod
    def func1(user):
        ABC.func2(user)

    @staticmethod
    def func2(lst):
        thrd_list = []
        for ele in lst:
            x = Thread(target=ABC.func3, args=(ele, ))
            x.start()
            thrd_list.append(x)
        for thrd in thrd_list:
            thrd.join()

    @staticmethod
    def func3(ele):
        # Do some stuff and if successful write to file
        with open("result.txt", "a") as OUTPUT:
            OUTPUT.write(f"{ele}\n")

if __name__ == "__main__":
    with open("users.txt", "r") as f:
        users = f.readlines()

    thrd_list = []
    for user in users:
        x = Thread(target=ABC.func1, args=(user.strip(), ))character
        x.start()
        thrd_list.append(x)
    for thrd in thrd_list:
        thrd.join()

Please explain your answer, what it does, why this solution, how it solves the problem — farhan jatt, Aug 23 '23 at 02:31

Not Writing in File Python

3 Answers3