2

I store some JSON files on disk (each very small about 4kB), and load them into a var when required. I may have thousands of such files, and if the space they occupy in memory is the same as what they occupy on disk, I would rather load them all into memory at application start.

How can I calculate how much memory (RAM) the file occupies when I create an object from it ?

EDIT: I know how to save JSON files to disk. That is not my question. I am trying to understand how much memory (RAM), a JSON file will occupy when I load it into a var. If I know the size of the JSON file on disk, how can I calculate how much memory (RAM) it will occupy when I create a javascript object from the file in node.js or in a web browser ?

The files are saved as "plain text files". Eg:

{"foo": {
    "bar": 1,
    "foo": {
        "bar": [
            "foo",
            "bar"
        ], 
        "foo":[
             "bar",
             "foo"
        ]
     }
    }
}
Rahul Iyer
  • 19,924
  • 21
  • 96
  • 190
  • please have a look http://stackoverflow.com/questions/2202645/how-do-i-store-json-data-on-a-disk – Amit Agarwal Nov 11 '14 at 10:02
  • That is not relevant to my question. I know how to store data on disk. I'm trying to understand, how much memory a given JSON file will occupy in memory, and how to measure it. – Rahul Iyer Nov 11 '14 at 10:16
  • This might be a duplicate of http://stackoverflow.com/questions/1248302/javascript-object-size – Ajay Tatachar Nov 11 '14 at 11:26

2 Answers2

2

The amount of RAM occupied by the object is not the same as the size of the file. The size of the file is simply the number of bytes in it, but the amount of RAM occupied by the object depends on the values stored in it. For example, in an object like this:

{
    "bar": 1
}

When saved to a file, 1 is only a single byte. However, if you were to load this object into a var, 1 would occupy 8 bytes, since it is a number.

Take a look at this answer for the size of each datatype: https://stackoverflow.com/a/11900218/1611665

Also, you should take the size of the keys into account (but that's pretty simple, it's twice as much as the length property if your keys are strings).

Community
  • 1
  • 1
Ajay Tatachar
  • 383
  • 3
  • 16
  • So based on the code in the question you linked to, my 4KB JSON file, seems to become a 3748 byte object. Weird coincidence. Thanks for the explanation. But can you explain what you mean by taking the size of the keys into account ? Do you mean that I should because th e other code sample does not ? – Rahul Iyer Nov 11 '14 at 13:57
  • Yeah, I didn't see any of the other examples taking the size of the keys into account, even though the keys occupy RAM. – Ajay Tatachar Nov 12 '14 at 13:27
1

I would try to measure memory consumption in the JavaScript execution environment you use. E.g.for the chrome browser, you can use this memory profiling guide.

Your mentioning of data files suggests that your code runs on the server, probably node.js. I would have a look at this.

mvw
  • 5,075
  • 1
  • 28
  • 34