3

I don't know if this is something possible. I am trying to append 12 files into a single file. One of the files is tab delimited and the rest comma delimitted. I loaded all the 12 files into dataframe and append it into an empty dataframe one by one in a loop. 

list_of_files = glob.glob('./*.txt')
df = pd.DataFrame()
for filename in list_of_files:
    file = pd.read_csv(filename)
    dfFilename = pd.DataFrame(file)
    df = df.append(dfFilename, ignore_index=True)

But the big file is not in the format I wanted it to be. And I think the problem is with the tab delimited file. And I tried to run the code without the tab delimited file and the format of the appended file is fine. So I was thinking if it is possible to change the tab delimited format into comma delimited using pandas.

Thank you for your help and suggestion

Same
  • 759
  • 2
  • 9
  • 15

2 Answers2

13

You need to tell Pandas that the file is tab delimited when you import it. You can pass a delimiter to the read_csv method but in your case, since the delimiter changes by file, you want to pass None - this will make Pandas auto-detect the correct delimiter.

Change your read_csv line to:

pd.read_csv(filename,sep=None)
AustinC
  • 826
  • 1
  • 8
  • 23
3

For the file that is tab-separated, you should use:

file = pd.read_csv(filename, sep="\t")

Pandas read_csv has quite a lot of parameters, check it out in the docs

asiviero
  • 1,225
  • 10
  • 16