is there an easy way to estimate size of a json object?

Question

I'm working on a security service that will return a list of permissions and I'm trying to estimate the size of the json response object. Here's a piece of sample data:

ID=123 VariableName=CanAccessSomeContent

I'm looking for an easy way to estimate what size of the json response object will be with 1500 rows. Is there an online estimation tool or some other technique I can use to easily get a rough size estimate?

score 10 · Answer 1 · answered Aug 19 '16 at 21:26

Using Python you can estimate the size by creating the dictionary or just make one...

import json
import os
import sys

dict = {}

for a in range(0, 1500):
    dict[a] = {'VariableName': 'CanAccessSomeContent'}

output = json.dumps(dict, indent = 4)   

print ("Estimated size: " + str(sys.getsizeof(output) / 1024) + "KB")

with open( "test.json", 'wb') as outfile:
    outfile.write(output)

print ("Actual size: " + str(os.path.getsize('test.json') / 1024) + "KB")

Output:

Estimated size: 100KB
Actual size: 99KB

can you provide an example in c#? – user6604655 Aug 19 '16 at 22:08 — user6604655, Aug 19 '16 at 22:08

score 4 · Answer 2 · answered Oct 11 '18 at 23:11

I solved it, when I needed to, by adding a File-like object that just counted the characters and json.dump()ing into it:

# File-like object, throws away everything you write to it but keeps track of the size.
class MeterFile:
    def __init__(self, size=0):
        self.size = size

    def write(self, string):
        self.size += len(string)

# Calculates the JSON-encoded size of an object without storing it.
def json_size(obj, *args, **kwargs):
    mf = MeterFile()
    json.dump(obj, mf, *args, **kwargs)
    return mf.size

The advantage is that encoding is not stored in memory, which could be large especially in cases you care about the size to begin with.

score 2 · Answer 3 · answered Oct 16 '17 at 15:05

2

Function to estimate file size (Mash of JSON-Size & UTF-8 Length node repos)

function json_filesize (value) {  
    // returns object size in bytes
    return (~-encodeURI(JSON.stringify(value)).split(/%..|./).length)/1048576
}

json_filesize({foo: 'bar'}) >> 13

answered Oct 16 '17 at 15:05

Shankar ARUL

12,642
11
68
69

this isn't python right? Also using encodeURI, will it work for large json objects? – alancalvitti Jul 15 '21 at 16:36
this is JavaScript. didnt have issues on my browser in terms of file size. – Shankar ARUL Jul 16 '21 at 09:39

score 1 · Answer 4 · answered Aug 19 '16 at 21:13

I'm not sure if this is what you're after, as this seems extremely basic, but here goes:

First start with 0 rows, encode it and measure the size (We'll call this A).
Then, get a decent sample from your database and encode 1 row at a time.
For each of those outputs, calculate the size and then store the average (we'll call this B).

Now for X rows, the estimated json response will be X * (B-A) + A

So if A was 100 bytes, and B was 150 bytes, for 1500 rows we'll get:

1500 * (150-100) + 100 = 75100 bytes = 73 KB

is there an easy way to estimate size of a json object?

4 Answers4

Linked