csv.writer.writerows missing rows from list

Question

I'm new to python.

I have a list with 19188 rows that I want to save as a csv.

When I write the list's rows to the csv, it does not have the last rows (it stops at 19112).

Do you have any idea what might cause this?

Here is how I write to the csv:

mycsvfile = open('file.csv', 'w')
thedatawriter = csv.writer(mycsvfile, lineterminator = '\n')
list = []
#list creation code
thedatawriter.writerows(list)

Each row of list has 4 string elements.

Another piece of information:

If I create a list that contains only the last elements that are missing and add them to the csv file, it kind of works (it is added, but twice...).

mycsvfile = open('file.csv', 'w')
thedatawriter = csv.writer(mycsvfile, lineterminator = '\n')
list = []
#list creation code
thedatawriter.writerows(list)
list_end = []
#list_end creation code
thedatawriter.writerows(list_end)

If I try to add the list_end alone, it doesn't seem to be working. I'm thinking there might be a csv writing parameter that I got wrong.

Another piece of information:

If I open the file adding ", newline=''", then it write more rows to it (though not all)

mycsvfile = open('file.csv', 'w', newline='')

There must be a simple mistake in the way I open or write to the csv (or in the dialect?)

Thanks for your help!

Don’t shadow built in functions like `list`: change your variable name. — Laurent LAPORTE, Dec 01 '16 at 22:54
Your list is empty, how do you get 19112 rows? Post your real code… — Laurent LAPORTE, Dec 01 '16 at 22:55
Possible duplicate of [Read and Write CSV files including unicode with Python 2.7](http://stackoverflow.com/questions/17245415/read-and-write-csv-files-including-unicode-with-python-2-7) — Laurent LAPORTE, Dec 01 '16 at 22:57
Hi Laurent. Do you mean that I should rename my list to something else? I did not include the part of the code that builds the list (it's opening other csv and merging them together). I'm not sure that the link you gave applies to my issue, but thanks for your help! — yoann, Dec 01 '16 at 23:30

score 4 · Accepted Answer · edited May 23 '17 at 12:30

I found my answer! I was not closing the filehandle before script end which left unwritten rows.

Here is the fix:

with open('file.csv', 'w', newline='') as mycsvfile:
    thedatawriter = csv.writer(mycsvfile, lineterminator = '\n')
    thedatawriter.writerows(list)

See: Writing to CSV from list, write.row seems to stop in a strange place

Close the filehandle before the script ends. Closing the filehandle will also flush any strings waiting to be written. If you don't flush and the script ends, some output may never get written.

Using the with open(...) as f syntax is useful because it will close the file for you when Python leaves the with-suite. With with, you'll never omit closing a file again.

csv.writer.writerows missing rows from list

1 Answers1