0

Planned App:

I want to make an web-app with Polymer & Firebase where users can create "card-sets" (like in Google Now on Android) and share them with groups/public. A card-set consists of a card-list (required to easily sort the list). A card-list contains of card-items which contain content in some way.

Firebase structure:

I was reading a lot about structuring data in Firebase and came up with the following structure (in manner of keeping it flat). The reason for the split is, that I might want to have a huge amount of "cardsets" and just load an overview of it.

|-user
   |-cardset
     |-UID
        |-title
        |-category
        |-reference-to-cardlist-UID
     |-UID
        |-....

   |-cardlist
     |-UID
        |-1
          |-reference-to-carditem-UID
          |-order
          |-visible
        |-2
          |-...

   |-carditem
     |-UID
        |-title
        |-content
     |-UID
        |....

If users want to share, I would duplicate this in a "public" or "group" node instead of "user".

Polymer element structure:

As for the element structure in polymer, I would create data loading elements in the same structure, with which I can load the nodes in the required way:

data-card-set.html
data-card-list.html
data-card-item.html

(even if I would combin them into one, it wouldn't reduce the amount of times I need to initiate an element in the parent) In the parent, they would interact with each other and load the data to display it.

Questions

  1. FB: Is this a reasonable flat structure for Firebase?
  2. FB: Is it the right approach for user/public/group data?
  3. Polymer: Is this the right way to load data? It seems like a lot of elements and a lot of requests sent to the database, but I want to avoid loading too much data from the DB.

In most of the examples I can find online (e.g. todo-data), all data for the user is loaded at once. This I want to avoid, as e.g. there might be plenty of cardsets for one user.

Thanks a lot for your feedback!

Tim

Frank van Puffelen
  • 565,676
  • 79
  • 828
  • 807
gerd hübner
  • 423
  • 3
  • 14

2 Answers2

0

What I found difficult to understand with NoSQL was that "writing is cheap and reading is expensive". In your example this means that your cardset, which has the reference to cardlist, also should include all the necessary data from cardlist (and related card item data if needed):

    Cardset
      Reference to cardlist uid
      Cardlist name
      Cardlist visible
      Card item uid
        Card item title
        Card item content
      Card item uid
        Card item title
        Card item content
      Reference to cardlist uid

  Cardlist name
  Cardlist visible
  Card item uid
    Card item title
    Card item content

This means that you must write same information in multiple location. For example card item's title is written to: Card item itself All cardlists that use this item All cardsets that use a cardlist, which contains card item

You write this information multiple times because "writing is cheap". Now with this kind of datastructure reading is simple, you just need to read cardset once and you'll have all the data you want to display. You must decide what data you want to display at top level (cardset) and what additional data you'll retrieve when a user clicks child (cardlist). When user selects a cardlist, then you'll retrieve a new data for that cardlist including it's children's data (card item). Again you must decide what data is needed to show about a child. If cardset doesn't need to show any data from grandchild (card item), then you don't need to store any data about grandchild to cardset, only data related to cardlist.

So, once more you must write data to multiple locations in order to read it fast with only one request. This is the main difference between SQL and NoSQL. Updating the data is much more difficult, because it's stored to many places. This is the cost you must pay for a fast app.

grohjy
  • 2,059
  • 1
  • 18
  • 19
  • You write this information multiple times because "writing is cheap". Now with this kind of datastructure reading is simple, you just need to read cardset once and you'll have all the data you want to display. ==> but that's what I want to avoid, right? e.g. if I just want to load an overview list of cardsets (without the list and items loaded), I explicity don't want to load this other stuff? – gerd hübner Dec 02 '16 at 12:43
  • Also still the question open, what is the best way to "load" the data then. As I understand, the best is, to have as little "read" operations as possible. So whenever I can just load one set of data it is better than 3 sets of data? (e.g. cardset, cardlist, carditem in this case). but then I would have to create even more nodes..."cardset-onlycardsets, cardset-withlist, cardset-withalldata" and same for the others. Still stays confusing :) – gerd hübner Dec 02 '16 at 12:44
0

Thanks grohjy. So what I understand is also I need to look at my "usecase". I re-structured it now:

cardset-TO-SHOW-LIST-ONLY => this is to load an overview list of the cardsets cardset-LOAD-CARDSET-WITH-CONTENT => this to load a cardset with all items - as I need all items anyway, I don't mind if they are "nested"

Does that make sense?

Also described here: Performance of Firebase with large data sets

|-user
   |-cardset-TO-SHOW-LIST-ONLY
     |-UID
        |-title
        |-category
        |-reference-to-cardlist-UID
     |-UID
        |-....

   |-cardset-LOAD-CARDSET-WITH-CONTENT
     |-UID
        |-title
        |-category
        |-carditems
          |-carditem-UID
             |-title
             |-content
          |-carditem-UID
             |....
     |-UID
        |-....

Load step by step?

The only performance problem I still see is, that I need to wait to get the whole "cardset-LOAD-CARDSET-WITH-CONTENT" and can not load "one by one" from the server, to at least show some items first. Or is there a way?

Community
  • 1
  • 1
gerd hübner
  • 423
  • 3
  • 14
  • Basically you have to decide what's a smallest amount of data needed to satisfy users requirements and then let him to ask more information. These further requests usually contains a much smaller amount of data plus there is more time to fulfill the request (ie. user is expecting a reasonable waiting time). If you have initially a very long list to show, you should slice it into a smaller chunks (I'm not sure which is the best way to do it: save these lists separately in database or make a filtered query), by smaller chunks I mean for example "next page" or "more". – grohjy Dec 02 '16 at 13:54
  • And in the initial list you can use a "+" for expandable row/info and then by clicking the "+" a user asks more information. If you have a typical filtered cardset (f. ex. "Important", "recent"), each of these most used filters should be saved to database as a node. Easy to read, slow to write. And it's same principle when you have a user related cardset. I believe you are starting to see a difference between SQL and NoSQL. You would never use these kind of structures in SQL. – grohjy Dec 02 '16 at 14:04