12

I have a .fits file that contains data.

I would like to construct a pandas dataframe from this particular file but I don't know how to do it. 

data = fits.open('datafile')
data.info

gives: 

No.    Name         Type      Cards   Dimensions   Format
0    PRIMARY     PrimaryHDU       6   (12, 250000)   float64

and: 

data[0].data.shape

gives:

(250000, 12)
  • impossible to answer unless you include a sample of this file and what the desired result should look like – EdChum Oct 18 '16 at 15:09
  • It is a dataset with 250 000 rows that represent people, and 12 features which represent their characteristics. So I need a simple 250000x12 dataframe –  Oct 18 '16 at 15:13
  • So have you tried `df = pd.read_csv('datafile')`? – EdChum Oct 18 '16 at 15:13
  • Yes. I get a UnicodeDecode error. –  Oct 18 '16 at 15:16
  • 1
    FITS is a mixed text and binary file format used primarily in astronomy. read_csv has nothing to do with it. – Iguananaut Oct 18 '16 at 22:10

2 Answers2

17

According to what you have in your question and the astropy docs (http://docs.astropy.org/en/stable/io/fits/), it looks like you just need to do: 

from astropy.io import fits
import pandas
with fits.open('datafile') as data:
    df = pandas.DataFrame(data[0].data)

Edit: I don't have much experience we astropy, but other have mentioned that you can read the fits files into a Table object, which has a to_pandas() method:

from astropy.table import Table
dat = Table.read('datafile', format='fits')
df = dat.to_pandas()

Might be worth investigating.

http://docs.astropy.org/en/latest/table/pandas.html

Paul H
  • 65,268
  • 20
  • 159
  • 136
  • Just a minor comment here: You should close the file if you explicitly `open` it, or use the contextmanager: `with fits.open('datafile') as data:` so you don't leave open file handles flapping around. – MSeifert Oct 19 '16 at 12:13
4

Note: the second option with Table is better for most cases, since the way FITS files store data is big-endian, which can cause problems when reading into a DataFrame object which is little-endian. See https://github.com/astropy/astropy/issues/1156

virginiadude
  • 51
  • 4