Remove leading blank space from CSV results in insertion of blank rows and deletion of rows

Question

With the following (note the first row has no leading space):

Test1@bigfoot.com
 Test11@bigfoot.com
 Test1111@bigfoot.com
 Test111ew@bigfoot.com
 Test12312@bigfoot.com
 Test1231321@bigfoot.com
 Test1342321@bigfoot.com
 ....
 481 total rows

The following correctly removes the leading space, but inserts a blank row after each string row, AND, truncates the total list by a random number of rows each time it is executed.

csvfile= open('list.csv','r')
csvfile1= open('complete_list.csv','w')
stripped = (row.strip() for row in csvfile)
reader = csv.reader(stripped,delimiter=' ')
writer= csv.writer(csvfile1)
for row in reader:
    writer.writerow([e.strip() for e in row])

And:

with open('list.csv') as infile:
    reader = csv.DictReader(infile)
    fieldnames = reader.fieldnames
    for row in reader:
        row.update({fieldname: value.strip() for (fieldname, value) in row.items()})

Does nothing, as the first row is assumed to be the fieldname, when in fact it is simply...a row.

That solved the blank line issue, but is still truncating the list: Output only has 459 rows,should have the 481 from the original CSV, which is why the question is not an exact duplicate. — tpcolson, Dec 26 '17 at 17:58
you should probably do `csvfile1.close()` before reading it back again (flush buffers) — Jean-François Fabre, Dec 26 '17 at 18:00
Regardless of where I put csvfile1.close(), it returns I/O operation on closed file — tpcolson, Dec 26 '17 at 18:11
it's not the close operation that returns this, it's a read/write performed _after_ having closed the file. — Jean-François Fabre, Dec 26 '17 at 18:12
I'm confused. If the only call to csvfile1 is to open it in write mode, how does closing it before writing to it accomplish that? — tpcolson, Dec 26 '17 at 18:17
check my answer, I'm proposing a solution without "close". The small fixes I'm proposing should make your code work (of course I didn't test with your data) — Jean-François Fabre, Dec 26 '17 at 18:19

score 1 · Accepted Answer · answered Dec 26 '17 at 18:17

several issues here:

csv files must be opened in write mode with newline="" in python 3, else it insert blanks on windows
don't use strip but lstrip on the lines, else it removes newline at the end of the line. Can confuse csv reader
use with context block so it ensures that the files are closed when exiting the block (should handle your random missing lines in the end)

my proposal:

with open('list.csv','r') as csvfile, open('complete_list.csv','w',newline="") as csvfile1:  # newline="" to avoid blanks
    stripped = (row.lstrip() for row in csvfile)  # lstrip not strip
    reader = csv.reader(stripped,delimiter=' ')
    writer= csv.writer(csvfile1)
    writer.writerows(reader)   # don't overstrip: just write rows as-is

Ahh that did it! I was drawing a complete blank on the correct order of closing/opening preparatory to writing thanks! — tpcolson, Dec 26 '17 at 18:21

Remove leading blank space from CSV results in insertion of blank rows and deletion of rows

1 Answers1