Storing hierarchical data in Google Datastore

Question

I have data in the following structure:

{
  "id": "1",
  "title": "A Title",
  "description": "A description.",

  "listOfStrings": ["one", "two", "three"]

  "keyA": {
    "keyB": " ",
    "keyC": " ",
    "keyD": {
      "keyE": " ",
      "KeyF": " "
    }
  } 
}

I want to put/get this in Google Datastore. What is the best way to store this?

Should I be using entity groups?

Requirements:

I do not need transactions.
Must be performant to read the whole structure.
Must be able to query based on KeyEs content.

This link (Storing hierarchical data in Google App Engine Datastore?) mentions using entity groups for hierarchical data, but this link (How to get all the descendants of a given instance of a model in google app engine?) mentions they should only be used for transactions, which I do not need.

Im using this library (which I do not think supports ReferenceProperty?). http://github.com/GoogleCloudPlatform/gcloud-ruby.git

score 2 · Accepted Answer · answered May 28 '15 at 15:15

2

If you want to be able to query by the hierarchy (keyA->keyB->keyC) -- use ancestors, or just a key which will look like this to avoid entity group limits. If you want to be able to query an entity which contains provided key -- make a computed property where you will store a flat list of keys stored inside. And store the original hierarchy in the JsonProeprty for example.

answered May 28 '15 at 15:15

Dmytro Sadovnychyi

6,171
5
33
60

So you and @BrentWashburne are suggesting: Flatten the hierarchy. _"keyA[keyD][keyE]" = " "_ Or/And store it as a JSON string. – zino May 28 '15 at 18:29

score 0 · Answer 2 · answered May 28 '15 at 16:49

In my experience, the more entities you request the slower (less performant) it becomes. Making lots of entities with small bits of data is not efficient. Instead, store the whole hierarchy as a JsonProperty and use code to walk through it.

If KeyE is just a string, then add a property KeyE = StringProperty(indexed=True) so you can query against it.

Storing hierarchical data in Google Datastore

2 Answers2