Copy specific rows from csv to csv in Python 2.7

Question

So far I have been trying to copy specific rows including headers from original csv file to a new one. However, once I run my code it was copying a total mess creating a huge document.

This is one of the options I have tried so far, which seems to be the closest to the solution:

import csv
with open('D:/test.csv', 'r') as f,open('D:/out.csv', 'w') as f_out:
     reader = csv.DictReader(f)
     writer = csv.writer(f_out)
     for row in reader:
         if row["ICLEVEL"] == "1":
            writer.writerow(row)

The thing is that I have to copy only those rows where value of "ICLEVEL"(Header name) is equal to "1".

Note: test.csv is very huge file and I cannot hardcode all header names in the writer.

Any demostration of pythonic way of doing this is greatly appreciated. Thanks.

Are you getting any error? What's happening with your solution? — Hussain, Dec 08 '16 at 06:14
When I run it says "sequence expected" and points at the last row. However, if I go with writer.writerow([row]) it makes a total mess. — Coke, Dec 08 '16 at 06:15

Hussain · Answer 1 · 2016-12-08T06:31:58.350

4

writer.writerow expects a sequence (a tuple or list). You can use DictWriter which expects a dict.

import csv
with open('D:/test.csv', 'r') as f, open('D:/out.csv', 'w') as f_out:
    reader = csv.DictReader(f)
    writer = csv.DictWriter(f_out, fieldnames=reader.fieldnames)
    writer.writeheader()  # For writing header
    for row in reader:
        if row['ICLEVEL'] == '1':
            writer.writerow(row)

edited Dec 08 '16 at 06:31

answered Dec 08 '16 at 06:20

Hussain

5,057
6
45
71

Thanks for an effort, but it gave me a key error ['ICLEVEL']. – Coke Dec 08 '16 at 06:25
Yes, I just double checked it again :( – Coke Dec 08 '16 at 06:30
It should have worked. Could you put few sample rows of your CSV? – Hussain Dec 08 '16 at 06:33
I just replaced my old csv with new one. It worked, however, it copies all the data, not specific one that I need. – Coke Dec 08 '16 at 06:36
There must be some whitespace around `row['ICLEVEL']`. It will be helpful if you add some snippet of your CSV. – Hussain Dec 08 '16 at 06:45
Possibly, that's very interesting, I will try to make your solution work as well. Appreciate your assistance. Thank you! – Coke Dec 08 '16 at 06:47
1

Passing `skipinitialspace=True` to the reader might help in that case http://stackoverflow.com/questions/14885908/strip-white-spaces-from-file-csv – Hussain Dec 08 '16 at 06:49

score 1 · Accepted Answer · answered Dec 08 '16 at 06:18

1

Your row is a dictionary. CSV writer cannot write dictionaries. Select the values from the dictionary and write just them:

writer.writerow(reader.fieldnames)
for row in reader:
  if row["ICLEVEL"] == "1":
    values = [row[field] for field in reader.fieldnames]
    writer.writerow(values)

answered Dec 08 '16 at 06:18

DYZ

55,249
10
64
93

You did a great job. Can you suggest how to copy headers as well? – Coke Dec 08 '16 at 06:24
The first line of my snippet copies the headers. – DYZ Dec 08 '16 at 06:25
Works for me in Python 2.7. How does it not work for you? – DYZ Dec 08 '16 at 06:30
Thank you for the solution! – Coke Dec 08 '16 at 06:45

score 0 · Answer 3 · answered Dec 08 '16 at 06:40

0

I would actually use Pandas, not a CSV reader:

import pandas as pd

df=pd.read_csv("D:/test.csv")
newdf = df[df["ICLEVEL"]==1]
newdf.to_csv("D:/out.csv",index=False)

The code is much more compact.

answered Dec 08 '16 at 06:40

DYZ

55,249
10
64
93

I am not allowed to use Pandas. Your previous answer was correct, I would appreciate if you could restore it. Thanks. – Coke Dec 08 '16 at 06:43
1

Ok, I see. I believe @Hussain's answer is actually better. – DYZ Dec 08 '16 at 06:44
I appreciate his help, however, it didn't filter the data. – Coke Dec 08 '16 at 06:45

Copy specific rows from csv to csv in Python 2.7

3 Answers3