2

I am attempting to read in an npy file (numpys mmap format) directly in C++ but have had some difficulties.

The file was written using numpy and can be easily read back in using numpy (so it hasnt corrupted or anything).

My first attempt was the use the cnpy package (https://github.com/rogersce/cnpy) directly however this throws an assertion error regarding the size of the file header.

Does anyone have an example of creating a npy file in numpy and reading it in C++?

Cheers, Jack

JMzance
  • 1,704
  • 4
  • 30
  • 49
  • I think it would be wise to use (A) some widely supported matrix file format like [one of these](http://docs.scipy.org/doc/scipy/reference/io.html) or (B) write some customized code. – sascha Aug 23 '16 at 11:03
  • The code that writes a .npy is Python, and it should easy to determine the header size. Look for something like `lib/format.py`. If you don't have numpy, look at the github repository. – hpaulj Aug 23 '16 at 11:36

1 Answers1

3

I would check the first 10 bytes of your test file against the description in np.lib.format and the C++ code

A partial quote from the format.py file documentation header

Format Version 1.0
------------------

The first 6 bytes are a magic string: exactly ``\\x93NUMPY``.

The next 1 byte is an unsigned byte: the major version number of the file
format, e.g. ``\\x01``.

The next 1 byte is an unsigned byte: the minor version number of the file
format, e.g. ``\\x00``. Note: the version of the file format is not tied
to the version of the numpy package.

The next 2 bytes form a little-endian unsigned short int: the length of
the header data HEADER_LEN.

I haven't coded this in another language, but did look at the headers a bit for a SO that wanted to save several arrays in the same file.

loading arrays saved using numpy.save in append mode

Community
  • 1
  • 1
hpaulj
  • 221,503
  • 14
  • 230
  • 353