Why and when do we need to flatten JSON objects?

Question

I am surprised that no one on StackOverflow asked this question before.

Looking through the JSON object documentation and a quick google search did not yield satisfactory results.

What's the advantage of it? How does it work?

---

Edit: To make it clear, take a look at this flatten/un-flatten example.

Fastest way to flatten / un-flatten nested JSON objects

Thank you.

Answer 1

There are many situations where you get JSON text that was automatically built by some library. Throughout the programming languages, there are many libraries that build JSON text (one example is here).

Whenever libraries add some additional object or array wrappings, you might want to get rid of them maybe because you send the JSON to the server and your code there crashes because it expects a primitive value instead of an object (or an array). Or, if your JSON is a server response, you don't want the resulting Javascript code having to differ between object/array or not object/array. In all these cases, flattening is helpful as it will save you time. You will have to implement lesser if/elses, and you can reliably expect your data structure to be as flat as possible.

The other approach to improve code for the scenario mentioned is to write the code in a maximal robust way so there is no way for it to crash by superfluous wrappings ever. So always expect some wrappers and get it's contents. Then, flattening is not needed.

You see, it depends on what is building the JSON and what is parsing it. The building may be out of your scope.

This leads also to data model questions. I've worked with XML code that needed to be parsed quiet a different way if there where 0 entries of some XY, or if there were >0 entries of some XY. Having a wrapper that is allowed to have 0 or more entries of some XY will make live easier. These are data model desicions.

In all cases where the JSON represents an object structure that I've combined manually, I expect it not to change. So flattening something I've designed in detail would be disturbing. Standard operations as far I've seen them do not need flattening (e.g. JSON.stringify(), json_encode() etc.)

Answer 2

Here's a simple scenario: In a web app you have an HTTP POST that is updating a complex relational object.

POST
update=1
&user.id=12345
&user.email=testmail@domain.tld
&user.profile.name=Mr. Test
&user.profile.age=42
&user.profile.friend.0.email=tom@domain.tld
&user.profile.friend.1.email=sally@domain.tld
&user.profile.friend.2.email=bob@domain.tld
&user.profile.skill.0.id=100
&user.profile.skill.0.name=javascript
&user.profile.skill.1.id=200
&user.profile.skill.1.name=piano

Everything is already in a flat structure, so why not have a simple one-to-one binding? If you had a list of constraints or security requirements that you needed to enforce you could validate them by searching directly on the sorted key list.

Flat structures are easier for people to understand and work with there's even some cross-over with database de-normalisation. It also allows for context specific security and constraints to be implemented in a readable, but more verbose way.

When showing a user's view in full you may want to hide the display of the primary key ID for the user's list of skills.

"user.profile.skill.#.id": { hidden: true, readonly: true }

But when looking directly at a skill (to possibly edit it as an administrator) you may want to see the ID.

"skill.id": { readonly: true }

If you were writing a user-centric/self-service type CMS application you'd get more users on board and able to contribute using a straightforward flat model (flat abstraction of the underlying nested relational model) than you would with just the nested model.

TLDR: Flat is easier to read than nested. While programmers can handle nested schemas, recursive parsing and processing; end-users and admins usually prefer that part abstracted away.

Answer 3

I realize this is a 5 year old question at this point, but I figured, I'd add my thoughts to it, in case someone runs into a similar use case and finds this useful.

One of the use cases why you would want to flatten a JSON object, is for dynamic template binding via Regular Expression (RegEx) string interpolation. Well wasn't that a mouthful ? It simply translates to "template filling a string without hardcoding".

Ok Imagine a scenario, you have a template string like this for an email:

Hello {{firstName}},

It is amazing you chose to join our site. We are happy to have you on board. 
To get started, we would really love it if you can confirm your email address
by clicking on the link: {{confirm_url}}.

Welcome aboard

The Team!

Given the following JSON object in memory:

{
   "user" : {
               "prefix"      : "Dr.",
               "firstName"   : "Awah",
               "lastName"    : "Teh",
               "email"       : "awah@superduperubercoolsite.com",
               "address"     : {
                                  "street": "100 Main St",
                                  "city"  : "PleasantVille",
                                  "state" : "NY",
                                  "phone" : "+1-212-555-1212"
                               }
            },
   "meta" : {
               "confirm_url" : "http://superduperubercoolsite.com/confirm/ABC123"
            }
}

it seems super simple to do a Regular Expression replace like so (assuming our email template string was stored in a variable named template and the json object was stored in a variable called templateData:

template = template.replace(new RegExp('{{firstName}}', 'g'), templateData.user.firstName);
template = template.replace(new RegExp('{{confirm_url}}', 'g'), templateData.meta.confirm_url);

Easy right? --> Actually yes! How about this email had 10 templated fields, or you wanted to decouple the template from the code, by storing it in a separate system like SendGrid, where your cool head of marketing can access the template and make changes to the copy-language, without having to call someone from engineering to make changes to the code, test the code and redeploy to production (what a hassle).

This is exactly where flattening of the JSON comes save the day!

Now there are many ways to flatten JSON, I have attached a link to a codepen I wrote that has logic to flatten JSON (actually, I demonstrate two similar but different approaches in the methods flattenJSONIntoKVP and flattenJSONIntoRAW check 'em out!).

That said, there are other implementations out there, and it is worth remembering that the focus on this post is to discuss the WHY JSON flattening could be useful, not the HOW.

Moving on! Assume you flattened the JSON from above (using my implementation that results in key value pairs) to something like this:

[
   { "key": "user.prefix",         "value": "Dr."},
   { "key": "user.firstName",      "value": "Awah"},
   { "key": "user.lastName",       "value": "Teh"},
   { "key": "user.email",          "value": "awah@superduperubercoolsite.com"},
   { "key": "user.address.street", "value": "100 Main St"},
   { "key": "user.address.city",   "value": "{PleasantVille"},
   { "key": "user.address.state",  "value": "NY"},
   { "key": "user.address.phone",  "value": "+1-212-555-1212"},
   { "key": "meta.confirm_url",    "value": "http://superduperubercoolsite.com/confirm/ABC123"},
]

Now, my friend, you are cooking with GAS!

Why, cause now you can dynamically interpolate the template string with values from the JSON object without giving too much worry to the structure of the JSON (if it changes due to the application evolving, you don't have to also remember to come down here and change this interpolation code -- you simply just have to update the email template itself, which mind you, is on SendGrid [per this example]).

So how to do it you say?: Simple, iteratively. Let's assume that flattened from above was stored in a variable called flatJSON:

///Notice how I use Javascripts native string interpolation to create my RegExp

///Also note that I am replacing the dot (.) in my flattened JSON variable names with a double underscore (__), I only do this because my intended target is SendGrid, and I don't believe it likes dots in its template placeholders.
flatJSON.forEach(kvp=>template = template.replace(new RegExp(`{{${kvp.key.replace(/\./g, '__'}}}`, 'g'), kvp.value));

That's it, one line of code to replace possibly 10 or even hundreds or even thousands (ok.. maybe not thousands, but you get the point).

Ohh! almost forgot, we need to update our template string.

Notice how now, in our new templated string we can use a somewhat FQDN style variable to map back to our original JSON (Ideally if SendGrid supported dots in their template placeholders, this would look super sweet but alas, can't always win everything!.

Hello {{user__firstName}},

It is amazing you chose to join our site. We are happy to have you on board. 
To get started, we would really love it if you can confirm your email address
by clicking on the link: {{meta__confirm_url}}.

Welcome aboard {{user__prefix}} {{user__lastName}}!

The Team!

Et Voila!

Just like that, we have accomplished some good here today; we have:

1. Answered the WHY of flattening JSON objects
2. We dibble-dabbled into the how, with the codepen example
3. And we even overviewed a use case where taking advantage of JSON flattening can help you write durable dynamic code, that evolves as your underlying object structures change -- and that doesn't require you to leverage the big bad ugly eval method (we can talk about big bad ugly eval on another post).