Iterating over csv.reader() in a pythonic way

Question

I'm developing a program that reads among other things a csv with csv.reader(). I want to save variables that are matching one of two conditions in two different classvariables. Because i like writing list comprehensions and i'm trying to write my code in a more pythonic way, i was wondering if this is the right approach. csv.reader() seems to be a generator object, because i can iterate over it just once. So for the second condition i'm creating the reader-object a second time. That seems like a ressource waste to me. Of course i could write it with a normal for loop and different if cases. But is there a more pythonic way to write this piece of code?

with open(file, "r") as f:
    reader = csv.reader(f, delimiter=",")
    self.xitireq = [row[0] for row in reader if "xiti" in row[0]]
with open(file, "r") as f:
    reader = csv.reader(f, delimiter=",")
    self.apireq = [row[0] for row in reader if "https://www.blabal.de/api" in row[0]]

If i write the code like that, self.apireq is empty:

with open(file, "r") as f:
    reader = csv.reader(f, delimiter=",")
    self.xitireq = [row[0] for row in reader if "xiti" in row[0]]
    self.apireq = [row[0] for row in reader if "https://www.blabal.de/api" in row[0]]

"So for the second condition i'm creating the reader-object a second time. That seems like a ressource waste to me" Not really, how is it a resource waste? — juanpa.arrivillaga, Mar 27 '19 at 17:30
There is nothing inherently wrong with for loops,but list comprehensions can look cool. As you point out, it is worth testing the performance differences, which are usually negligible. This question does address the idea converting generators to lists (https://stackoverflow.com/questions/24130745/convert-generator-object-to-list-for-debugging), and it does seem to me that creating the generator from a csv file twice could be noticeably less efficient if you have a big csv to read twice, but I will try to test this if I have time soon. — bart cubrich, Mar 27 '19 at 17:47

bart cubrich · Accepted Answer · 2019-03-27T17:34:36.087

2

The simplest solution I can think of to do it the way you want is like this

with open(file, "r") as f:
    reader = list(csv.reader(f, delimiter=","))
    self.xitireq = [x for x in reader if "xiti" in x]
    self.apireq = [x for x in reader if "https://www.blabal.de/api" in x]

The only difference here is that I converted your reader to a list, with list(), based on this posting Python import csv to list and this documentation https://docs.python.org/3/library/csv.html#csv.Dialect.

May be more accurate at this point to rename reader to something, but that's just semantics.

edited Mar 27 '19 at 17:34

answered Mar 27 '19 at 17:16

bart cubrich

1,184
1
14
41

Genius!! Thats it!! haha – jo87casi Mar 27 '19 at 17:28
Do you want self.xitireq and self.apireq separately? It would be possible to shorten your code by combing those two statements into one. – bart cubrich Mar 27 '19 at 17:32
I changed the row[0] to x, by the way. I think that should work. – bart cubrich Mar 27 '19 at 17:33
Yeah two different class variables. But i'm curious how you would combine them – jo87casi Mar 27 '19 at 17:33
Well you can use extend to add the two together post hoc with list1.extend(list2), but you could do them together in one statement like your_var=[x for x in reader if if any(s in x for s in ["xiti","htt ps://w ww.bla bal.de/api"]). On extending:https://www.programiz.com/python-programming/methods/list/extend. I had to put some spaces in the second string cuz it's turning it into a hyperlink in the comments. – bart cubrich Mar 27 '19 at 17:41

Iterating over csv.reader() in a pythonic way

1 Answers1