hdf5 file to pandas dataframe

Question

I downloaded a dataset which is stored in .h5 files. I need to keep only certain columns and to be able to manipulate the data in it.

To do this, I tried to load it in a pandas dataframe. I've tried to use:

pd.read_hdf(path)

But I get: No dataset in HDF5 file.

I've found answers on SO (read HDF5 file to pandas DataFrame with conditions) but I don't need conditions, and the answer adds conditions about how the file was written but I'm not the creator of the file so I can't do anything about that.

I've also tried using h5py:

df = h5py.File(path)

But this is not easily manipulable and I can't seem to get the columns out of it (only the names of the columns using df.keys()) Any idea on how to do this ?

I think my answe here and the provided links might be of help for you: https://stackoverflow.com/a/74127100/5838180 — NeStack, Oct 20 '22 at 11:53

score 15 · Answer 1 · answered Oct 03 '19 at 02:12

15

Easiest way to read them into Pandas is to convert into h5py, then np.array, and then into DataFrame. It would look something like:

df = pd.DataFrame(np.array(h5py.File(path)['variable_1']))

answered Oct 03 '19 at 02:12

im2527

391
3
4

2

What does 'variable_1' represent in this case? - I am having the same issue opening .h5 files – Patrick_Chong Jul 17 '22 at 12:31

score 7 · Answer 2 · edited May 23 '17 at 11:52

7

Pandas HDF support needs the HDF file to be formated very specifically. You can see https://stackoverflow.com/a/33644128/4128030 for more info.

edited May 23 '17 at 11:52

Community

1
1

answered Jan 11 '17 at 18:33

drj

146
2
8

1

Yes. More about this [here](https://stackoverflow.com/a/30787168/4653485) as well. – Jérôme Oct 18 '17 at 10:15

hdf5 file to pandas dataframe

2 Answers2