0
df_list1=[A,B,C]
df_list2=[A,B,C]

I want to use pandas concat to concatenate A(df_list1) to A(df_list2), B to B like wise without loop. Here A, B, C are dataframes. I have tried iterating one of the list and just concatenating one dataframe from each list at a time.

i=0
for df in df_list1:
   l=[df,df_list2[i]]
   df=pd.concat(l)
   i=i+1

Could someone help me doing this without loop as I have certain time execution requirement(basically I need to reduce time)?

jkat
  • 49
  • 1
  • 4
  • [Never call DataFrame.append or pd.concat inside a for-loop. It leads to quadratic copying.](https://stackoverflow.com/a/36489724/1422451) – Parfait Nov 06 '19 at 23:38
  • Try iterating through letters not dfs: `df_list3 = [pd.concat(df_list1[x], df_list2[x]) for x in list('ABC')]` – Parfait Nov 06 '19 at 23:40
  • @Parfait hey but A,B,C are not characters. so I meant writing [A,B,C]. I edited my post – jkat Nov 06 '19 at 23:57
  • Then try elementwise loop with `zip`: `df_list3 = [pd.concat(d1, d2) for d1, d2 in zip(df_list1, df_list2)]` – Parfait Nov 07 '19 at 15:31

1 Answers1

1

I don't think you can get past using a loop, but you may be interested in the zip function, which matches the elements of respective lists as you iterate through them.

df_list1=[A,B,C]
df_list2=[A,B,C]
concat_list = []

for x, y in zip(df_list1, df_list2):
    concat_list.append(pd.concat([x, y]))

EDIT:

Actually, you could use map:

concat_list = list(map(lambda x: pd.concat([x[0], x[1]), zip(df_list1, df_list2)))
PeptideWitch
  • 2,239
  • 14
  • 30
  • hey thanks @PeptideWitch. If both list are not in order like d1=[B,A,C] and d2=[A,C,B] and still I want to concat A to A and B to B likewise, could you help how can I do it? – jkat Nov 08 '19 at 00:05
  • No worries! So your follow-on question can be taken a few different ways. It comes down to how you know that the list is out of order. You could go through each list and check the index of the dataframe or name of a column, or if you already know the order of the lists then you could make an index list: `[1,0,2]` and `[0,2,1]` then do an iteration: `for x in range(len(l))` and access the dataframe lists using the `x` as your indexer -- `df_list1[index_list.index(x)]`. – PeptideWitch Nov 08 '19 at 02:41