python - Writing to csv file out of coroutine sink ... how to avoid closed file error?

Question

My code (simplified):

import csv


def generate_record(downstream):
    try:
        while True:
            incoming = (yield)
            record = incoming.strip()
            for worker in downstream:
                worker.send(record)
    except GeneratorExit:
        for worker in downstream:
            worker.close()
        print('generate_record shutdown')


def file_writer(filename):
    l = list()    
    try:
        while True:
            record = (yield)
            l.append(record)
    except GeneratorExit:
        with open(filename, 'w', newline=''):
            writer = csv.writer(f)
            writer.writerows(l)
        print('file_writer shutdown')


if __name__ == '__main__':
    sink = file_writer('C:/Users/Some User/Downloads/data.csv')
    next(sink)    
    worker = generate_record([sink])
    next(worker)
    with open('C:/Users/Some User/Downloads/Energy.txt') as f:
        for line in f:
            worker.send(line)
    worker.close()

Generates the following error:

Traceback (most recent call last):

  File "<ipython-input-43-ff97472f6399>", line 1, in <module>
    runfile('C:/Users/Some User/Documents/Python Scripts/isii.py', wdir='C:/Users/Some User/Documents/Python Scripts')

  File "C:\Users\Some User\Anaconda3\lib\site-packages\spyderlib\widgets\externalshell\sitecustomize.py", line 699, in runfile
    execfile(filename, namespace)

  File "C:\Users\Some User\Anaconda3\lib\site-packages\spyderlib\widgets\externalshell\sitecustomize.py", line 88, in execfile
    exec(compile(open(filename, 'rb').read(), filename, 'exec'), namespace)

  File "C:/Users/Some User/Documents/Python Scripts/isii.py", line 75, in <module>
    worker.close()

  File "C:/Users/Some User/Documents/Python Scripts/isii.py", line 49, in generate_record
    worker.close()

  File "C:/Users/Some User/Documents/Python Scripts/isii.py", line 63, in file_writer
    writer.writerows(l)

ValueError: I/O operation on closed file.

What have I tried?

I've tried incrementally writing with writerow within file_writer within the try block, but that generates the same error.

`worker.send(output)` seems to be typo of `worker.send(record)`. — falsetru, Apr 18 '16 at 01:58

score 3 · Accepted Answer · answered Apr 18 '16 at 01:57

The with statement in the file_writer is missing as f part; by missing that, f references the global variable f instead which is closed at the time of writing; cases the IOError.

with open(filename, 'w', newline='') as f:
                                     ^^^^

score 0 · Answer 2 · edited May 23 '17 at 11:59

0

When you use with open(filename) as f:, it will do the operation you have added and then closes the file. So you don't need to use worker.close() since you are trying to close a file that already is closed.

See: What is the python keyword "with" used for?

This should be a comment, but I do not seem to have enough reputation for that.

edited May 23 '17 at 11:59

Community

1
1

answered Apr 18 '16 at 01:54

Ediz12

1
4

You still have to close the coroutine, though. Without that final worker.close, the coroutine stays open. – MikeRand Apr 18 '16 at 04:57

python - Writing to csv file out of coroutine sink ... how to avoid closed file error?

2 Answers2