1

Let's say we have a dictionary

import numpy as np
d={}
d["s0"]=3
d["s1"]=np.int16(3)
d["s2"]=np.array("hello")
d["s3"]=np.array([2])
d["s4"]=np.linspace(0,2, 3)

One way to save this dictionary is to use json. Which means serializing and storing the data as a list. In this case there can be loss of precision.

Another way is to convert this into a pandas DataFrame and save that to hdf:

import pandas as pd
df=pd.DataFrame(d)
#dd.io.save("test.h5", d)
store = pd.HDFStore('store.h5')
store["data"]=df

But this failed. I get:

  ValueError: arrays must all be same length

A yet third way is to use deepdish:

dd.io.save("test.h5", d)

The problem with this method was it wants my keys to be strings and misses key data without throwing up error:

    $h5ls test.h5
    s3                       Dataset {1}
    s4                       Dataset {3}

Note that "s0", "s1" and "s2" were not saved to the file and no error was reported. So what is the safest way to store a python dictionary to an hdf file?

I don't want to use pickle dump because it will be hard to read back in Fortran. This question is not a duplicate of this question because it shown how those methods failed to store needed data.

wander95
  • 1,298
  • 1
  • 15
  • 22
  • Use `pd.Series` instead of `pd.DataFrame` – piRSquared Jun 21 '18 at 14:30
  • 1
    I have already covered various option so it is not a duplicate. I have shown how various answers failed. – wander95 Jun 21 '18 at 14:31
  • I think `pickle` is the best option in this case... – MaxU - stand with Ukraine Jun 21 '18 at 14:41
  • 1
    Can't read `pickle` in fortran – wander95 Jun 21 '18 at 14:46
  • 2
    With `h5py` you can save dictionaries and arrays. The `h5` groups are dictionary like, but limited to string keys. The datasets are numpy array like. While Python dictionaries can use any `hashable` as keys, it will hard to translate that across languages. While I've used Fortran on and off over many decades, I don't know what dictionary like data structures are available. Best stick with structures implemented by widely used file formats. – hpaulj Jun 22 '18 at 02:31

1 Answers1

0

So this isn't a cutting-edge answer that will impress any uber-geeks, but if you need portability, I'd suggest an INI file. There isn't a language or platform in existence that can't read or write it.

Here's a FORTRAN library for handling them.

https://github.com/szaghi/FiNeR

Terry Carmen
  • 3,720
  • 1
  • 16
  • 32