0

I have a dat file that looks like this:

#Occultation start (UTC): 2020-02-23 00:04:22
#Occultation stop  (UTC): 2020-02-23 00:06:40
#Occultation point latitude (degN):    28.0
#Occultation point longitude (degE):  -17.1
#Center of curvature (m):    11574.705   -3554.708  -13613.902
#Radius of curvature (m):  6369130.294
#Azimuth (degN):    129.789
#Undulation (m):     44.336
#Elevation (m):       0.000
#Processing method:  Wave optics (CT2)
#Background data:    ECMWF forecast 
#Receiver data type: L1caL2p
#Navbit correction:  extern
#Occultation type:   Rising 
#OL/CL mode:         OL->CL
#Alt OL/CL (m):   11884
#alt w.r.t. EGM96
 geoid|lat|lon|azimuth|ba_opt|impact_opt|refrac|dry_press|dry_temp|geopotential height|ba_raw|ba_L1|ba_L2|ba_model|snr_L1|snr_L2
              0.000  -99999000.000  -99999000.000  -99999000.000   -0.99999E+08  -99999000.000   -0.99999E+08  -99999000.000  -99999000.000  -99999000.000   -0.99999E+08   -0.99999E+08   -0.99999E+08   -0.99999E+08  -99999000.000  -99999000.000

A text at the beginning describes general information about the observation date, time, location, etc., After that, there is a line that lists the data names separated by "|" which are:

geoid|lat|lon|azimuth|ba_opt|impact_opt|refrac|dry_press|dry_temp|geopotential height|ba_raw|ba_L1|ba_L2|ba_model|snr_L1|snr_L2

The next lines (601 rows) include data for each variable named above. I want to skip the first 17 information rows and extract for example lat and long values as two columns.

I tried converting the data file to a CSV format so I can plot them, however, the converted CSV file was not what I really expected

with open('file.dat', 'r') as input_file:
    lines = input_file.readlines()
    newLines = []
    for line in lines:
        newLine = line.strip('\t').split()
        newLines.append(newLine)
with open('file.csv', 'w') as output_file:
    file_writer = csv.writer(output_file)
    file_writer.writerows(newLines)
bah
  • 21
  • 6
  • You forgot to include the Python code that was your attempt to solve this problem. – Scott Hunter Oct 27 '22 at 12:46
  • Not exactly the same, but there are ways to filter out comment lines: https://stackoverflow.com/questions/14158868/python-skip-comment-lines-marked-with-in-csv-dictreader Is the data itself tab separated? (Hard to tell from what you have posted) – doctorlove Oct 27 '22 at 12:50
  • 1
    Thank you @doctorlove , the link somehow leads me in the right direction. – bah Oct 27 '22 at 13:41
  • @bah it's ok to answer your own question if you find something helpful – doctorlove Oct 27 '22 at 13:58
  • @doctorlove well as I said, it just gave me some ideas but not the exact answer, I simply used: `newfile=[] for line in file[16:]: newfile.append(line.strip()) with open('newfile.csv','w') as line: for element in newfile: line.write(element+ "\n")` and then : `import pandas as pd import pandas import pandas as pd df = pd.read_csv('newfile.csv',sep=',') #print(df.head()) print(df)` the output CSV file seems to be fine, but when I open it in python, python reads all data into one column! – bah Oct 28 '22 at 10:25

0 Answers0