2

First of all, I use C++. I have a CV_32F cv::Mat and when I write it down to disk using FileStorage, the size of this Mat becomes around 4.5 times higher than the size of it when it was on RAM during the program execution. I do some experiments and each time it is like that. So, when I tried to read it again, obviously my RAM(6 GB) becomes insufficient ,though it was not during the program execution.

Here is how I write it down to the disk:

FileStorage fs( PATH, FileStorage::WRITE);
fs << "concatMat" << concatMat;
fs.release();

And this is how I calculate the occupied RAM size during the program execution:

size_t sz= sizeof( concatMat) + concatMat.total()*sizeof( CV_32F);

I wonder the reason behind this, especially why there is always 4.5 times difference?

EDIT: I save them with .bin extension, not YAML or XML. I need to save them efficiently and can take recommendations.

Aka
  • 137
  • 1
  • 12
  • Is `sizeof(CV_32F)` 4? At least `sizeof(CV_32FC3)` cannot be correct? – mainactual Jun 28 '16 at 08:01
  • 1
    `sizeof(CV_32F)` gives the same result as `sizeof(float)` which is 4 – Sunreef Jun 28 '16 at 08:09
  • OpenCV writes in YAML format if you specify a path ending in .bin... @Aka – Sunreef Jun 28 '16 at 08:20
  • `FileStorage` stores as YAML unless you specify XML. There's no option for storing binary data. – molbdnilo Jun 28 '16 at 08:25
  • Yes, I realised that it saves as YAML though it is a .bin file. How can I store it efficiently? – Aka Jun 28 '16 at 08:27
  • Here is a one way of saving it as binary: https://github.com/takmin/BinaryCvMat – Aka Jun 28 '16 at 08:40
  • To store/load matrix efficiently, have a look [here](http://stackoverflow.com/a/32357875/5008845) – Miki Jun 28 '16 at 10:22
  • @Miki Same approach with the link I shared but thanks. – Aka Jun 28 '16 at 10:29
  • @Aka, oh, I missed your link. Yeah, same stuff ;D. You can't do better than that (aside from compressing the binary stream, with [gzstream](http://www.cs.unc.edu/Research/compgeom/gzstream/) for example) – Miki Jun 28 '16 at 10:44

1 Answers1

2

Take a look at the contents of your XML or YML or .bin file with Notepad++. (By the way, if you specify a path ending in .bin, OpenCV will write it in a YAML format...)

You will see that each float from your CV_32F Mat has been written in a format like this 6.49999976e-001. This represents 15 bytes instead of the 4 bytes expected for a float. This is a ratio of 15 / 4 = 3.75. If you add to that all the characters for formatting like ',' '\n' or ' ', you may reach a size that is more than 4 times bigger than what you had on RAM.

If you try to save a Mat with only zeros inside, you will see that the size is quite similar to what you had in RAM because the zeros are written 0.. It is actually smaller if you save it in XML format.

Sunreef
  • 4,452
  • 21
  • 33
  • Hell, yea.. You're right, it saves them as a YAML format though it is a .bin file. How can I store them efficiently? – Aka Jun 28 '16 at 08:23
  • As far as I know (which may not be so far), `FileStorage` probably can't help you. You could try to find a library that writes real .bin files from the `float*` data pointer of your `Mat`. I'm not very knowledgeable here. @Aka – Sunreef Jun 28 '16 at 08:25