Python 1 'for' loop for 2 lists (csv)

Question

I have 2 .csv files which both contain 3001 lines, and I have to do the same thing to both of them, so I was wondering if I can put it into 1 for loop.

So I thought to be smart and tried this:

self.csv_readingon = csv.reader(open('heisig6_reading_ja_on.csv'))
self.csv_readingkun = csv.reader(open('heisig6_reading_ja_kun.csv'))

for [d1, d2] in [self.csv_readingon, self.csv_readingon]:
    for i, d in enumerate([d1, d2]):
        # Do same stuff to both files

However this results in:

for [d1, d2] in [self.csv_readingon, self.csv_readingon]:
ValueError: too many values to unpack (expected 2)

Is it possible to put it this nicely in 1 for loop or should I make 1 for loop for 1 file (d1) and then manually assign a line from the 2nd csv file to d2?

Edit: The .csv files are not related to each other. Both are going to be put in a separate table in a SQLite database. So probably tuples are not the way to go. They just have the same structure, so only the table name will differ.

@KarolyHorvath I don't think `zip` will do what's the OP asks for... `zip([1, 2, 3], [4, 5, 6])` => `[(1, 4), (2, 5), (3, 6)]` — bruno desthuilliers, Dec 16 '15 at 13:27
@brunodesthuilliers It does exactly what the OP is asking for. You just use tuple unpacking and it will give you the zeroth item of each iterable, followed by the first item of each iterable, and so on. — pzp, Dec 16 '15 at 13:30
@OP What are you going to being doing to the stuff? Do you need to compare the items from each csv file to each other or do you just want to process the first csv file and then the second csv file. Please update your question to make it more specific. — pzp, Dec 16 '15 at 13:34
@pzp I understand the OP question as "just want to process the first csv file and then the second csv file" - at least from "I have 2 .csv files and I have to do the same thing to both of them". — bruno desthuilliers, Dec 16 '15 at 13:36
@OP: If you want to put the data of each file into a different table it won't work to have one for loop iterating first over all lines from one file and then over all lines from the other file. Also the zip solution would not be nice here, because it's simple coincidence (?) that both files have the same length. I'd put the for loop into a function taking file handle and table name as parameters and call this function first for the first and and the second file. — elzell, Dec 16 '15 at 13:46
@elzell, that's what I also realized after some time. The files looked so similar that I was so focused on how to get this to work in 1 for-loop, that I didn't take a step back and realize I can just give the .csv as parameter. If you add this to your answer, I'll accept it. — NumesSanguis, Dec 16 '15 at 16:03

elzell · Accepted Answer · 2015-12-16T16:32:44.923

3

Use a combination of enumerate and zip:

for i, (d1, d2) in enumerate(zip(self.csv_readingon, self.csv_readingkun)):
    # pass

Edit: If you want to put the data of each file into a different table it won't work to have one for loop iterating first over all lines from one file and then over all lines from the other file. Also the zip solution would not be nice here, because it's simple coincidence (?) that both files have the same length. I'd put the for loop into a function taking file handle and table name as parameters and call this function first for the first and and the second file.

edited Dec 16 '15 at 16:32

answered Dec 16 '15 at 13:25

elzell

2,228
1
16
26

Beat me to it! I'll delete my answer :) – pzp Dec 16 '15 at 13:26
Either I missed the point or the OP's example using `enumerate` fooled someone, but what I read was : "I'd like to apply the same treatment to rows from 2 csv files without havin to write two `for` loops... in which case your solution is possibly one of the most complicated way to do a very simple thing ;-) – bruno desthuilliers Dec 16 '15 at 13:32
I understood OP in such a way that he wants to process each two lines of the two files at the same time and that he needs the line numbers as well. That's why I disagreed with your solution at first glance, but thinking about it it's also possible that processing first file first and second file after that is sufficient for OP ... :) – elzell Dec 16 '15 at 13:36
@brunodesthuilliers In my interpretation of PS's question, I agree with _elzell_ and _pzp_ but I recognize that the question is not a good one... – gboffi Dec 16 '15 at 13:47
@brunodesthuilliers Now the question is more specific, so my solution doesn't quite fit. But simply chaining the inputs doesn't work either (see my comment on the original post above). – elzell Dec 16 '15 at 13:47

bruno desthuilliers · Answer 2 · 2015-12-16T14:01:54.953

csv.reader are iterables. The simplest way to chain two iterables is to use itertools.chain():

chain(*iterables) --> chain object

Return a chain object whose .next() method returns elements from the first iterable until it is exhausted, then elements from the next iterable, until all of the iterables are exhausted.

So in your case:

from itertools import chain

for row in chain(reader1, reader2):
   proceed_with(row)*

Now given your added explanations (insert rows into two distinct tables with similar structures but different names), this won't work. You could try to be smart and "generic" by passing the table name with the reader, ie:

for tablename, reader in (("table1", reader1), ("table2", reader2)):
    for row in reader:
        proceed_with(row, tablename)

but this will break as soon as one or the other csv and associated table structure changes.

score 0 · Answer 3 · answered Dec 16 '15 at 13:30

0

You could also try this:

for list in [self.csv_readingon, self.csv_readingon]:
    for i, d in enumerate(list):
        # Do same stuff to both files

answered Dec 16 '15 at 13:30

Corby

21
3

Python 1 'for' loop for 2 lists (csv)

3 Answers3