Python iteration through CSV

Question

for i, line in enumerate(lines):
    testIDLines.append(line)
        if line.find("TEXT") > 0:
            for row in csvFile:
                dosomething
                if dosomething = True:
                    break
                else:
                    continue

I have a small problem within this code - The first iteration works perfectly, but when I come into the second for loop another time the for row in csvFile starts not at the start of the csvFile, but where it left the loop the last time.

csvFile Variable is a dictionary created with csvFile = csv.DictReader(open("Filename.csv"))

Is there a way to reset the csvFile variable to start from the first Row? Or how can this even happen, that the csvFile variable in this case gets touched when I make a for row in csvFile

and what is `lines` - are they from the same file `"Filename.csv"` ? — RomanPerekhrest, Oct 03 '19 at 12:55
Possible duplicate of https://stackoverflow.com/questions/2868354/reading-from-csvs-in-python-repeatedly or https://stackoverflow.com/questions/431752/python-csv-reader-how-do-i-return-to-the-top-of-the-file. — mkrieger1, Oct 10 '19 at 11:47

score 1 · Accepted Answer · answered Oct 09 '19 at 08:43

1

csv.DictReader use internal iterator that is created from file descriptor passed in. Which in turn use file iterator to iterate over data. So as @Maciej stated - you need to reset file position using seek.

To do so you need to save value of file discriptor prior to passing it to DictReader. By the way you have leak there as close will not be called.

with open("Filename.csv") as csvFile:
  csvDict = csv.DictReader(csvFile)
  ...
    for i, line in enumerate(lines):
        testIDLines.append(line)
            if line.find("TEXT") > 0:
                    for row in csvDict:
                        dosomething
                        if dosomething = True:
                            break
                        else:
                            continue
                    csvFile.seek(0) # reset iteration

But if your files are not big it's better to just cache data before iteration in dict and then use it in calculation so there wouldn't be any redundant IO calls.

answered Oct 09 '19 at 08:43

j2ko

2,479
1
16
29

Thanks - the File close was done outside of this loop i showcased, but i used your approach now :) Works fine this way – Dominik Lemberger Oct 09 '19 at 11:41
In both the question and this answer is a typo, a double equal sign is needed: if dosomething == True: – MA53QXR Oct 11 '19 at 07:15
@MA53QXR that is true but it isbpseudo code anyway. – j2ko Oct 11 '19 at 07:24
yeah ur right - but as j2ko said right it was just pseudo code I wrote here and not the copied code in my actual code I had the right == sign - was an error that happened when deleting the code that are not for the public :) thanks though – Dominik Lemberger Oct 11 '19 at 10:38

score 0 · Answer 2 · answered Oct 03 '19 at 13:05

0

In order to reset to start of a file try this:

>>> csvFile.seek(0)

Python file method seek() sets the file's current position at the offset

answered Oct 03 '19 at 13:05

Maciej M

786
6
17

csvFile should be a Dictionary as i created it with csvFile = csv.DictReader(open("Filename.csv")) – Dominik Lemberger Oct 03 '19 at 13:18

score 0 · Answer 3 · answered Oct 09 '19 at 08:27

A look at the (cpython) implementation of DictReader, especially the __next__ implementation, shows that it internally iterates over the file's lines (using a csv.reader) and parses each line as a dict. Iterating a second time on the same object without seeking the file to 0 (i.e., the beginning) won't work, because the reader is still at the end of the file.

Is there a way to reset the csvFile variable to start from the first Row?

It doesn't seem to be straightforward. You need to seek the underlying file to 0, but you'd also need to reset the reader inside your DictReader (and possibly other state). The simplest solution (and IMO correct, since you don't tamper with internal object state) is to seek the file back to the beginning and create a new DictReader instance for the second loop.

Python iteration through CSV

3 Answers3