8

I must be missing something, but I don't get it. I have a csv, it has 1200 fields. I'm only interested in 30. How do you get that to work? I can read/write the whole shebang, which is ok, but i'd really like to just write out the 30. I have a list of the fieldnames and I'm kinda hacking the header.

How would I translate below to use DictWriter/Reader?

for file in glob.glob( os.path.join(raw_path, 'P12*.csv') ):
    fileReader = csv.reader(open(file, 'rb'))
    fileLength = len(file)
    fileGeom = file[fileLength-7:fileLength-4]
    table = TableValues[fileGeom]
    filename = file.split(os.sep)[-1]
    with open(out_path + filename, "w") as fileout:
        for line in fileReader:
            writer = csv.writer(fileout, delimiter=',', quotechar='"', quoting=csv.QUOTE_MINIMAL)
            if 'ID' in line:
                outline = line.insert(0,"geometryTable")
            else:
                outline = line.insert(0,table) #"%s,%s\n" % (line, table)
            writer.writerow(line)
Martijn Pieters
  • 1,048,767
  • 296
  • 4,058
  • 3,343
greenwar
  • 203
  • 1
  • 2
  • 6

1 Answers1

23

Here's an example of using DictWriter to write out only fields you care about. I'll leave the porting work to you:

import csv

headers = ['a','b','d','g']

with open('in.csv','rb') as _in, open('out.csv','wb') as out:
    reader = csv.DictReader(_in)
    writer = csv.DictWriter(out,headers,extrasaction='ignore')
    writer.writeheader()
    for line in reader:
        writer.writerow(line)

in.csv

a,b,c,d,e,f,g,h
1,2,3,4,5,6,7,8
2,3,4,5,6,7,8,9

Result (out.csv)

a,b,d,g
1,2,4,7
2,3,5,8
Mark Tolonen
  • 166,664
  • 26
  • 169
  • 251
  • 1
    Perfect. Before, I was missing the extrasaction. Does that remove the...extras? I was also attempting to limit the fields on the read, but that didn't get me anywhere. Is that possible, tho, to limit read? for example `fileReader = csv.DictReader(open(file, 'rb'), fieldnames=EstimateColumns)` – greenwar Mar 29 '12 at 05:56
  • DictReader reads all the columns. The default for dict writer is to throw an exception if the dictionary (line) has more keys than specified in the headers list. `extrasaction='ignore'` suppresses that. DictReader always reads the whole csv line. It does take a field names parameter, but if used it specifies the column names instead of using the first row of the csv for column names. See http://docs.python.org/library/csv.html#csv.DictReader. – Mark Tolonen Mar 29 '12 at 06:08
  • Where can we add a filter for row in the above code? ```for example, remove rows where column-g==7``` – Prem Dec 01 '21 at 14:00
  • 1
    @Prem after the for, qualify the write with `if line['g'] != 7:`. – Mark Tolonen Dec 01 '21 at 15:55