1

I was refactoring some code for my program and I have a mistake somewhere in the process. I am reading and writing .csv files.

In the beginning of my program I iterate through a .csv file in order to find which data from the file I need.

with open(csvPath, mode='r') as inputFile:
    csvReader = csv.reader(inputFile)
    potentialVals = []
    paramVals = {}
    for row in csvReader:
        if row[3] == "Parameter":
            continue
        # Increment vales in dict
        if row[3] not in paramVals:
            paramVals[row[3]] = 1
        else:
            paramVals[row[3]] += 1

This iterates and works fine, the for loop gets me every row in the .csv file. I them perform some calculations and go to iterate through the same .csv file again later, and then select data to write to a new .csv file. My problem is here, when I go to iterate through a second time, it only gives me the first row of the .csv file, and nothing else.

# Write all of the information to our new csv file
with open(outputPath, mode='w') as outputFile:
    csvWriter = csv.writer(outputFile, delimiter=',', quotechar='"', quoting=csv.QUOTE_ALL)
    inputFile.seek(0)
    rowNum = 0
    for row in csvReader:
        print(row)

Where the print statement is, it only prints the first line of the .csv file, and then exits the for loop. I'm not really sure what is causing this. I thought it might have been the

inputFile.seek(0)

But even if I opened a 2nd reader, the problem persisted. This for loop was working before I refactored it, all the other code is the same except the for loop I'm having trouble with, here is what it used to look like:

what it used to look like

Edit: So I thought maybe it was a variable instance error, so I tried renaming my variables instead of reusing them and the issue persisted. Going to try a new file instance now,

Edit 2: Okay so this is interesting, when I look at the line_num value for my reader object (when I open a new one instead of using .seek) it does output 1, so I am at the beginning of my file. And when I look at the len(list(csvReader)) it is 229703, which shows that the .csv is fully there, so still not sure why it won't do anything besides the first row of the .csv

Edit 3: Just as a hail mary attempt, I tried creating a deep copy of the .csv file and iterating through that, but same results. I also tried just doing an entire separate .csv file and I also got the same issue of only getting 1 row. I guess that eliminates that it's a file issue, the information is there but there is something preventing it from reading it.

Edit 4: Here is where I'm currently at with the same issue. I might just have to rewrite this method completely haha but I'm going to lunch so I won't be able to actively respond now. Thank you for the help so far though!

# TODO: BUG HERE
with open(csvPath, mode='r') as inputFile2:
    csvReader2 = csv.reader(inputFile2)
    ...

    for row2 in csvReader2:
        print("CSV Line Num: " + str(csvReader2.line_num))
        print("CSV Index: " + str(rowNum))
        print("CSV Length: " + str(len(list(csvReader2))))
        print("CSV Row: " + str(row2))

Also incase it helps, here is csvPath:

nameOfInput = input("Please enter the file you'd like to convert: ")
csvPath = os.path.dirname(os.path.realpath(nameOfInput))
csvPath = os.path.join(csvPath, nameOfInput)
Toastuy
  • 11
  • 2
  • 1
    Did you mean `for row in csvWriter:`? You have `cvsReader` in your code. – PM 77-1 May 24 '21 at 16:11
  • Not here, I'm trying to iterate through the same .csv file a 2nd time to append information to lists in this 2nd for loop, and then I use csvWriter after. – Toastuy May 24 '21 at 16:13
  • 1
    Why did you not just paste the code (what it used to look like) as text instead of trying to include a screenshot of it? – mkrieger1 May 24 '21 at 16:14
  • Apologies I just don't have that specific snippet of code anymore, that screenshot was from a powerpoint presentation I had showcasing the code before I refactored it. – Toastuy May 24 '21 at 16:16
  • Okay. Consider using a version control system so that you can always refer to previous versions of the code. This will make it much easier to find bugs like these as a bonus. – mkrieger1 May 24 '21 at 16:18
  • I did look at that thread before posting this, which is why I tried out `inputFile.seek(0)` instead of closing and opening. It's weird because the 2nd for loop starts at the beginning of the .csv file, so it is resetting, but then it's not going through any other row. – Toastuy May 24 '21 at 16:22
  • Is the `with ... outputFile` block inside the `with ... inputFile` block? If not, `inputFile` has been closed because the program left its `with` block and you can't read from it anymore – Pranav Hosangadi May 24 '21 at 16:54
  • To demonstrate this: if the file is still open, you can `.seek(0)` and read another time without any problem: [Proof](https://i.stack.imgur.com/OAti0.png) – Pranav Hosangadi May 24 '21 at 16:55
  • Yeah the outputFile chunk was nested within the inputFiles, but even then if I do something like `with open(csvPath, mode='r') as inputFile2:` `csvReader2 = csv.reader(inputFile2)` `for row in csvReader2:` `print(row)` I still only get 1 row – Toastuy May 24 '21 at 16:58

1 Answers1

0

If you read the documentation carefully, it says csv reader is just a parser and all the heavy lifting is done by the underlying file object.
In your case, you are trying to read from a closed file in the second iteration and that is why it isn't working.
For csv reader to work you'll need an underlying object which supports the iterator protocol and returns a string each time its next() method is called — file objects and list objects are both suitable.

Link to the documentation: https://docs.python.org/3/library/csv.html

Shashank Ks
  • 450
  • 5
  • 9