Multiple pandas.dataframe to one csv file

Question

I have multiple pandas dataframes, and hope to write them as one CSV file. What is the most straightforward way?

For example, from following four dataframes,

enter image description here

how can I create below CSV?

enter image description here

Note The dataframes all have the same dimensions.

Do the dataframes all have the same shape? – unutbu Jun 14 '15 at 13:10 — unutbu, Jun 14 '15 at 13:10
@unutbu Yes. All the dataframes have the same shape. – Light Yagmi Jun 14 '15 at 13:12 — Light Yagmi, Jun 14 '15 at 13:12

Ami Tavory · Accepted Answer · 2018-11-21T20:50:53.390

15

A very straightforward way would be to concat pairs horizontally, concat the results vertically, and write it all out using to_csv:

 import pandas as pd

 pd.concat([
    pd.concat([df1, df2], axis=1),
    pd.concat([df3, df4], axis=1)]).to_csv('foo.csv')

A possibly more memory-conserving way would be to write it piecemeal:

with open('foo.csv', 'w') as f:
     pd.concat([df1, df2], axis=1).to_csv(f)
with open('foo.csv', 'a') as f:
     pd.concat([df3, df4], axis=1).to_csv(f, header=False)

Omitting headers=False would repeat the headers.

edited Nov 21 '18 at 20:50

answered Jun 14 '15 at 13:14

Ami Tavory

74,578
11
141
185

Thank you. Is it possible to keep headers visible for df3 and df4? In my question, the destination CSV has four rows and third one is the headers of df3 and df4. – Light Yagmi Jun 14 '15 at 14:41
I love this solution. Thanks! – Light Yagmi Jun 15 '15 at 06:28
@MichaelTamillow Thanks! – Ami Tavory Nov 21 '18 at 20:51
@AmiTavory how to write each dataframe in diferent sheets of the same csv file – S.EB May 28 '20 at 16:00
@S.EB That's mainly a question of how to read multiple sheets. It's fairly doable, but a separate question. Could you open one on it? Plenty will help you, and if you link here, I'll be happy to look at it too. – Ami Tavory May 28 '20 at 16:18

score 10 · Answer 2 · edited Jul 02 '21 at 20:57

10

For anyone that needs to append multiple dataframes to same csv vertically, you can use this:

with open('all_dfs.csv','a') as f:
    for df in list_of_dfs:
        df.to_csv(f)
        f.write("\n")

edited Jul 02 '21 at 20:57

Community

1
1

answered Dec 14 '20 at 18:01

Nicolae Stroncea

587
2
6
17

Multiple pandas.dataframe to one csv file

2 Answers2

Linked