Converting multiple .dat files with binary data to .csv with same file name

Question

I have a folder with .dat files with binary data which I want to parse and write to .csv files with same name as that of .dat files. I could take single .dat file and convert it to its respective .csv to the folder I need.

     import numpy as np
     import pandas as pd
     raw_file= '/home/targetfolder/channel1.dat'
     with open(raw_file, "rb") as f:

              raw_data = np.fromstring(f.read(), dtype=np.float32)
              comb_np_array = np.vstack(raw_data)
              big_frame = pd.DataFrame(comb_np_array)
              big_frame.to_csv("/home/destinationfolder/channel1.csv")

     f.close()

Output in ´filename.csv 0,-47.418867 1,-47.443828 2,-47.4445311 3,-47.4738281 4,-47.4193871 5,-47.4222221 6,-47.4193878

Here is the link I followed to do the same : Python: How to save *.dat-files as *.csv-files to new folder

    import csv
    from os import listdir
    from os.path import isfile, join, splitext

    dat_folder = "/home/nri/"
    csv_folder = "/home/nri/tmp/"

     onlyfilenames = [f for f in listdir(dat_folder) if 
                      isfile(join(dat_folder,f))]
     for fullfilename in onlyfilenames:
          file_name, file_extension = splitext(fullfilename)
          if file_extension == ".dat":
              inputfile=dat_folder + fullfilename
              with open(inputfile, "rb") as f:

                   inputfile = np.fromstring(f.read(), dtype=np.float32)
                   comb_np_array = np.vstack(raw_data)
                   n = pd.DataFrame(comb_np_array)

             with open(join(csv_folder, file_name + ".csv"), "w", 
                            newline='') as f:
                    writer = csv.writer(f,lineterminator='\n')
                    for row in range(len(n)):
                          writer.writerows(n)

But the above gives an error : "sequence expected" . Please let me know how to desired result.

post the complete traceback, where is the exception coming from (line number?)? Do you have a sample .dat file? — sathyz, Jun 18 '18 at 16:03
[link ]https://www.dropbox.com/s/du00e29th219cp1/channel1.dat?dl=0 sample dat file and the error is at last line of code : writer.writerrows(n) — Nrithya M, Jun 18 '18 at 16:34

sathyz · Answer 1 · 2018-06-19T04:07:26.707

You are iterating over len(n) but writing n every time.

for row in range(len(n)):
    writer.writerows(n)

writerows accepts sequence of sequences, but you are passing a data frame and expecting the writer to iterate on it, but a data frame is not a sequence in your case (python 2?). Here is how you can check this.

>>> import collections
>>> isinstance(big_frame, collections.Sequence)
False

You will need to iterate over the data frame and write the cell,

for i, row in big_frame.iterrows():
    writer.writerow([i, row[0]]) # 0 is the col name here

Converting multiple .dat files with binary data to .csv with same file name

1 Answers1