Python: Problem closing files in a while loop

Question

I've ran into an error while dealing with a while loop. I am able to input the number I want to run, and the results are written correctly to the corresponding .CSV. Once the section for that number is done running, it will ask if I want to try again with a new number. It runs the new numbers code and creates the new .CSV but the filesize stays at 0kb. I thought this has to do with closing the file once complete but I have written the f#.close() in there.

Ex: Choose number 1, code for #1 runs and saves .CSV correctly, choose Yes for another run and new number (#2), code #2 runs but does not correctly close thus not saving any info to the second CSV.

This happens regardless of which number I choose first or second. (ex: choose 3 first, 3 runs fine and saves fine. Then choosing 2 and runs but does not save correctly.)

Here is my current code:

f1 = file('1.csv', 'rb')
f2 = file('2.csv', 'rb')
f3 = file('3.csv', 'rb')

c1 = csv.reader(f1)
c2 = csv.reader(f2)
c3 = csv.reader(f3)

number = raw_input("Enter number: ")

while True:
    if number == "1":
        f4 = file('No_1_Results.csv', 'wb')
        c4 = csv.writer(f4)
       <snip> #do stuff here
        print "Took ", time.time() - start, "seconds."
        f4.close()
        reset_answer = raw_input("Again? Type Y or N : ")
        if reset_answer == ("n" or "N"):
            print "Bye! from  #1"
            break
        if reset_answer == ("y" or "Y"):
            number = raw_input("Enter new number #: ")
            continue
        if reset_answer != ("n" or "N" or "y" or "Y"):
            print "Y or N only allowed. Try again."
            continue

    if number == "2":
        f5 = file('No_2_Results.csv', 'wb')
        c5 = csv.writer(f5)
        <snip> #do stuff here
         print "Took ", time.time() - start, "seconds."
        f5.close()
        reset_answer = raw_input("Again? Type Y or N : ")
        if reset_answer == ("n" or "N"):
            print "Bye! from  #2"
            break
        if reset_answer == ("y" or "Y"):
            number = raw_input("Enter new number #: ")
            continue
        if reset_answer != ("n" or "N" or "y" or "Y"):
            print "Y or N only allowed. Try again."
            continue

    if number =="3":
        f6 = file('No_3_Results.csv', 'wb')
        c6 = csv.writer(f6)
        <snip> #do stuff here
        print "Took ", time.time() - start, "seconds."
        f6.close()
        reset_answer = raw_input("Again? Type Y or N : ")
        if reset_answer == ("n" or "N"):
            print "Bye! from  #3"
            break
        if reset_answer == ("y" or "Y"):
            number = raw_input("Enter new number #: ")
            continue
        if reset_answer != ("n" or "N" or "y" or "Y"):
            print "Y or N only allowed. Try again."
            continue

    if number is not "1" or "2" or "3":
        print "Invalid number selected."
        number = raw_input("Please choose a number: ")
        continue

f1.close()
f2.close()
f3.close()

Note: Using Python 2.6 on Windows / still learning python -

Your file logic looks fine. What's happening in the `` between opening and closing? — Nathan Ostgard, Mar 13 '11 at 17:24
Why don't you just put your whole `if number == n:` code as a function so you don't have to type it out three (or more if you change your code)? — John Howard, Mar 13 '11 at 17:28
@Matti Yes, but it works. I am still learning. @Nathan I'm trying to do the code from the following: [link](http://stackoverflow.com/questions/5268929/python-comparing-two-csv-files-and-searching-for-similar-items) — serk, Mar 13 '11 at 17:34

score 4 · Accepted Answer · answered Mar 13 '11 at 17:34

It would be nice to know what you are exactly doing with the file descriptor you opened at the beginning of the loop (f1, f2, f3). Nevertheless I believe that the issue you have is related with the fact that you are trying to read twice from one of those file descriptors but you are not resetting their position when doing it. Take look at the following:

$ echo "Test reading from a file" >> test.out
$ python
>>> f1 = file('test.out')
>>> f1.readlines()
['Test reading from a file\n']
>>> f1.readlines()
[]

After reading from a file the file descriptor remembers your position form the last read. To solve your issue you would have to perform a seek and going to the beginnings of the file:

$ echo "Test reading from a file" >> test.out
$ python
>>> f1 = file('test.out')
>>> f1.readlines()
['Test reading from a file\n']
>>> f1.seek(0)
>>> f1.readlines()
['Test reading from a file\n']

I hope this is the issue you have, otherwise you should show the logic that you have when reading the file.

@serk, Also look if the other fd have the same problem. Maybe rearranging the loop will make the solution cleaner. — mandel, Mar 13 '11 at 18:13

score 0 · Answer 2 · edited May 23 '17 at 11:55

import csv

try:
    inp = raw_input    # Python 2.x
except NameError:
    inp = input        # Python 3.x

def processFile(infname, outfname, perRow):
    with open(infname,'rb') as inf, open(outfname,'w') as outf:
        incsv  = csv.reader(inf)
        outcsv = csv.writer(outf)
        outcsv.writerows(perRow(row) for row in incsv)
    print("{0} -> {1} successful".format(infname, outfname))

def doStuff(row):
    # do stuff here
    return row

def main():
    while True:
        name = inp('Enter next file name (or nothing to exit)')
        if name.strip():
            try:
                processFile(name+'.csv', name+'_result.csv', doStuff)
            except IOError, e:
                print(e)
        else:
            print("Goodbye.")
            break

if __name__=="__main__":
    main()

Edit: after looking at your link to Python: Comparing two CSV files and searching for similar items I would continue with

def readMasterList():
    res = {}
    with open('masterlist.csv','rb') as inf:
        incsv = csv.reader(inf)
        head = incsv.next()  # skip header row
        for rownum,dat in enumerate(incsv):
            res[tuple(dat)] = rownum
    return res
masterList = readMasterList()

def doStuff(row, _ml=masterList):
    key = (row[1], row[3])
    try:
        rownum = _ml[key]
        row.append("FOUND in master list (row {0})".format(rownum))
    except KeyError:
        row.append("NOT FOUND in master list")
    return row

Python: Problem closing files in a while loop

2 Answers2