22

I have a file with some metadata, and then some actual data consisting of 2 columns with headings. Do I need to separate the two types of data before using genfromtxt in numpy? Or can I somehow split the data maybe? What about placing the file pointer to the end of the line just above the headers, and then trying genfromtxt from there? Thanks The format of the file is shown below:

 &SRS
<MetaDataAtStart>
multiple=True
Wavelength (Angstrom)=0.97587
mode=assessment
background=True
issid=py11n2g
noisy=True
</MetaDataAtStart>
&END
Two Theta(deg)  Counts(sec^-1)
10.0    41.0
10.1    39.0
10.2    38.0
10.3    38.0
10.4    41.0
10.5    42.0
10.6    38.0
10.7    44.0
10.8    42.0
10.9    39.0
11.0    37.0
11.1    37.0
11.2    45.0
11.3    36.0
11.4    37.0
11.5    37.0
11.6    40.0
11.7    44.0
11.8    45.0
11.9    46.0
12.0    44.0
12.1    40.0
12.2    41.0
12.3    39.0
12.4    41.0
Nirvan
  • 337
  • 1
  • 3
  • 7

1 Answers1

45

If you don't want the first n rows, try (if there is no missing data):

data = numpy.loadtxt(yourFileName,skiprows=n)

or (if there are missing data):

data = numpy.genfromtxt(yourFileName,skiprows=n)

If you then want to parse the header information, you can go back and open the file parse the header, for example:

fh = open(yourFileName,'r')
for i,line in enumerate(fh):
    if i is n: break
    do_other_stuff_to_header(line)
fh.close()
cm2
  • 1,815
  • 15
  • 20
  • I think I got the idea, will I need to use the csv.dictreader to read in the header? – Nirvan Nov 25 '13 at 22:34
  • What I have above will loop over the lines until you hit line `n` and then it will stop. When it loops over them, you can do whatever you want to parse them. – cm2 Nov 25 '13 at 22:38
  • how do I not import the last n lines? – Swift Feb 17 '17 at 09:32
  • loadtxt or genfromtxt doesn't work for me. I am getting the following error: File "/Users/vivekchowdary/opt/anaconda3/lib/python3.7/site-packages/numpy/lib/npyio.py", line 965, in loadtxt NameError: name 'os_PathLike' is not defined File "/Users/vivekchowdary/opt/anaconda3/lib/python3.7/site-packages/numpy/lib/npyio.py", line 1756, in genfromtxt replace_space=replace_space) NameError: name 'os_PathLike' is not defined – Vivek Apr 14 '20 at 10:38