why is the second loop never executed ?

Question

Hi, i am actually working on a python program and i need to read a csv file and use data.append(line) to fill a data Array.

I wrote this following part of the program :

print "Lecture du fichier", table1
lecfi = csv.reader(open(table1,'r'),skipinitialspace = 'true',delimiter='\t')
# delimiter = caractere utilisé pour séparer les différentes valeurs

tempSize = 0
tempLast = ""
oldSize = 0
#on initialise la taille du fichier et la derniere ligne du fichier 
if os.path.exists(newFilePath):
    tempSize = os.path.getsize(newFilePath) 
else:
    tempSize = 0
if os.path.exists(newFilePath) and tempSize != 0:
    #Si le fichier tampon n'existe pas, on le créer
    #Lecture du fichier tampon      
    lecofi = csv.reader(open(newFilePath,'r'),skipinitialspace = 'true',delimiter='\t')             
    csvFileArray = []
    for lo in lecofi:
        csvFileArray.append(lo)
    tempLast = str(csvFileArray[0])
    tempLast = tempLast[2:-2]
    oldSize = csvFileArray[1]               
    print "Tempon de Last : ", tempLast
    print "Taille du fichier : ", str(oldSize)   
    #on récupere la ligne représentant la derniere ligne de l'ancien fichier
else:
    #si le fichier n'existe pas, on lui laisse cette valeur par défaut pour le traitement suivant
        tempLast = None

    # remplissage des données du fichier pulse dans la variable data
cpt = 0
indLast = 0
fileSize = os.path.getsize(table1)
if oldSize != fileSize:
    for lecline in lecfi:
        cpt = cpt + 1
        last = str(lecline)
        if tempLast != None and last == tempLast:
            print "TEMPLAST != NONE", cpt
            indLast = cpt
            print "Indice de la derniere ligne : ", indLast
    print last, tempLast
    print "Variable indLast : ", indLast
    i = 0
    for co in lecfi:
        print "\nCOOOOOOO : ", co
        if i == indLast:
            data.append(co[0])
        i=i+1
    for da in data:
        print "\n Variable data : ", da

now look at the prints :

Lecture du fichier Data_Q1/2018-05-23/2018-5-23_13-1-35_P_HOURS_Q1
Tempon de Last :  ['(2104.72652']
Taille du fichier :  ['20840448']
TEMPLAST != NONE 317127
Indice de la derniere ligne :  317127
['(2104.72652'] ['(2104.72652']
Variable indLast :  317127

It seems like the program doesn't care about what's following my for loop. I assume that it can be a really basic mistake but i can't get it.

Any help ?

[Python csv.reader: How do I return to the top of the file?](//stackoverflow.com/q/431752) — 001, May 30 '18 at 13:13
Yup it would work if i had a file reader, but it's a csv reader so i can't apply the seek attribute... — lL. Bob, May 30 '18 at 13:21
Look again, the state of the csv reader based on the file reader is changed after the underlying file reader is reset. In your code you'd just have to store the result of `open(table1,'r')` in a variable before feeding it to `csv.reader` so that you can reset it between your two iterations. — Aaron, May 30 '18 at 13:23
Yup but it won't be that easy, the initial complete code isn't mine and it is a little rough. Thanks for your answer! — lL. Bob, May 30 '18 at 13:36

Elad · Answer 1 · 2018-05-30T13:23:24.510

1

You are trying to iterate over the CSV twice without reseting it. this is the reason your data array is empty.

The first time you actually iterates over the file:

for lecline in lecfi:

The second time, the original iterator already reached it's end and is empty:

for co in lecfi:

As mentioned in the comments by Johnny Mopp one possible solution is using the following method:

Python csv.reader: How do I return to the top of the file?

Hope this explains your issue.

edited May 30 '18 at 13:23

answered May 30 '18 at 13:18

Elad

221
1
6

Updated the answer to include the possible solution mentioned in the comments. – Elad May 30 '18 at 13:22
As i said previously on another comment, i don't have any file reader, i only have a csv reader so i can't use the .seek() attribute. – lL. Bob May 30 '18 at 13:28
I noticed that the answer by bruno has a good example on how to use it. – Elad May 30 '18 at 13:33

bruno desthuilliers · Answer 2 · 2018-05-30T13:30:31.927

Here:

for lecline in lecfi:
    cpt = cpt + 1
    # ...

you are reading the whole file. After this loop, the file pointer is at the end of the file and there's nothing more to be read. Hence here:

i = 0
for co in lecfi:
    # ...

this second loop is never executed, indeed. You'd need to either reset the file pointer, or close and reopen the file, or read it in a list right from the start and iterate over this list instead.

FWIW, note that opening files and not closing them is bad practice and can lead to file corruption (not that much in your case since you're only reading but...). A proper implementation would look like:

with open(table1) as tablefile:
    lecfi = csv.reader(tablefile, ....)

    for lecline in lecfi:
       # ....

    tablefile.seek(0)
    for lecline in lecfi:
       # ....

Also, this:

lecofi = csv.reader(open(newFilePath,'r'),skipinitialspace = 'true',delimiter='\t')             
csvFileArray = []
for lo in lecofi:
    csvFileArray.append(lo)

would be better rewritten as:

with open(newFilePath) as newFile:
   lecofi = csv.reader(newFile, ...)
   csvFileArray = list(lecofi)

why is the second loop never executed ?

2 Answers2