0

Here I have three csv files with same headers. I want to merge that three csv files into one csv file with same headers. here is my three csv file. I want to write it using panda python with column names. I saw so many solutions but it didn't work for me. Can anyone helps me to solve this problem?

csv 1

enter image description here

csv2

enter image description here

csv 3

enter image description here

output be like:

enter image description here

After trying the codes that you all suggest me and it gives me this without no values in some columns, no header name.

enter image description here

After trying your code (@Benji) it gave me an output with NaN values: enter image description here

error: enter image description here

dataframe after change the code

enter image description here

  • 1
    Could you specify how your output is going to look like? Merging in order by time? or just merging? – benji Feb 18 '19 at 03:41
  • @benji I upload the image of the output. You can see that csv 1,csv2,csv3 files data come in together –  Feb 18 '19 at 03:53
  • can you us how you did read a `.csv` file? – benji Feb 18 '19 at 04:08
  • @benji I tried your code, but it gave output with NaN values. –  Feb 18 '19 at 04:10
  • I read my csv file using panda. (data1 = pd.read_csv('temp1.csv') data2 = pd.read_csv('temp2.csv') data3 = pd.read_csv('temp3.csv') –  Feb 18 '19 at 04:11
  • Possible duplicate of [append multiple pandas data frames at once](https://stackoverflow.com/questions/36526282/append-multiple-pandas-data-frames-at-once) – Mohamed Thasin ah Feb 18 '19 at 05:26

4 Answers4

0

I assume that by “merge” you mean union. Here is how to do that

import pandas as pd
pd.concat(
    [pd.read_csv(filename, dtype=str) for filename in ["f1.csv", "f2.csv", "f3.csv"]],
    axis=0,
).to_csv("union.csv")
BallpointBen
  • 9,406
  • 1
  • 32
  • 62
  • when I tried your code it gives me this no header name , no values in some column . I pasted the image of that. Can you look at that? –  Feb 18 '19 at 04:00
  • Try adding `header = 0` before `dtype=str` in the `pd.read_csv` part – m13op22 Feb 18 '19 at 04:13
0
df1 = pd.read_csv('f1.csv')
df2 = pd.read_csv('f2.csv')
df3 = pd.read_csv('f3.csv')

frames = [df1, df2, df3]
result = pd.concat(frames)
envy_intelligence
  • 453
  • 1
  • 6
  • 21
  • when I tried your code it gives me this but without error (FutureWarning: Sorting because non-concatenation axis is not aligned. A future version of pandas will change to not sort by default. To accept the future behavior, pass 'sort=True'. To retain the current behavior and silence the warning, pass sort=False) Can you explain this please? –  Feb 18 '19 at 03:56
0

There's might be a problem when you read .csv file Try to read like this:

df1 = pd.read_csv('1.csv', header = 0, names=['date', 'time', 'x1', 'x2', 'x3'])
df2 = pd.read_csv('2.csv', header = 0, names=['date', 'time', 'x1', 'x2', 'x3'])
df3 = pd.read_csv('3.csv', header = 0, names=['date', 'time', 'x1', 'x2', 'x3'])    
df = pd.concat([df1, df2, df3], axis = 0)
df = df.sort_values('time').reset_index(drop = True)
benji
  • 186
  • 1
  • 11
  • yes that is true. I read my csv file wrongly. After change it and using your code it gave me this error ( '<' not supported between instances of 'float' and 'str') I pasted the image of the error.Can you tell me why is it coming like that? –  Feb 18 '19 at 04:36
  • @awa Can you print out a datafram after reading? I guess that time column might contain both str and float which it's supposed to be only float. – benji Feb 18 '19 at 05:09
  • Sure I paste the image of that. You may can look at it. –  Feb 18 '19 at 05:18
  • @awa Please upload a sample of your .csv file (not the whole data, just sample of it). It would be a lot easier for me to help you. – benji Feb 18 '19 at 05:22
  • Sure I did it and I pasted it here. –  Feb 18 '19 at 05:47
0

I had this exact problem and solved it like this:

df = pd.concat([pd.read_csv(i) for i in csv_files])
df = df.reset_index(drop=True)

The important line is the df.reset_index(drop=True. That's what joins the columns.

Recessive
  • 1,780
  • 2
  • 14
  • 37
  • I had a bunch of csv files all following a particular format, so I read them with something like this: `csv_files = ['data_america_2018_03_0' + str(i) for i in range(1,7)]` – Recessive Feb 18 '19 at 05:27
  • Okay I will try . Thank you –  Feb 18 '19 at 08:13