0

I am very new to Firebase.We have an application which creates notifications for users.The same notification might be delivered to multiple users. So we have structured the data as follows to optimize the data storage cost:

notifications
{
    message1
    {
        tiltle:"abc",
        desc:"desc"
        user1 : true
        user2 : true
        user3 : true
    }

    message2
    {
        tiltle:"efg",
        desc:"desc"
        user1 : true
        user2 : true
        user3 : true
    }
}

the users query the messages using orderByChild("<userId>").equalTo(true).

We cannot use an array of users attached to messages as the user can mark the message as read in which case we need to remove the user from the message.

Now the problem is that we see the queries are slow. Further we see a warning from firebase which says us to use indexes while performing queries on child nodes. The number of users will keep increasing, which would mean we need to create an index whenever a new user signs up.

Given this situation, i was wondering if there is any limit on the number of indexes that can be created?Also what will be the impact of creating a large number of indexes?

Frank van Puffelen
  • 565,676
  • 79
  • 828
  • 807
Neela Mauli Vangipurapu
  • 1
  • 1
    When you say the queries are slow, what does that mean? We have a similar structure with some of our data and with a few hundred (sometimes thousand) messages and a dozen or so children in each (or more), our queries are returning almost instantly. For indexing, you should define the keys you will be indexing on via the .indexOn rule in the [Security and Firebase Rules](https://www.firebase.com/docs/security/guide/indexing-data.html). The documentation is a good read. – Jay Feb 06 '16 at 15:44
  • 1
    The data structure shown doesn't lend itself to .index by user1 user2 etc so an alternate structure would take care of that. However, the slow response issue should probably be addressed first. – Jay Feb 06 '16 at 15:51
  • Cross-post: https://groups.google.com/forum/#!topic/firebase-talk/HP3c3YGSo6k – Frank van Puffelen Feb 07 '16 at 05:41
  • 1
    I wish someone had answered the question as laid out in the title: "Is there any limit on the number of indexes?". Does anyone have an answer for this? – TheAddonDepot Dec 14 '16 at 06:07

1 Answers1

2

When using NoSQL you'll often end up modeling your data for the use-cases that you need in your application. For your current use-case "get messages for user", this is not an ideal data structure. You're making the database consider all messages for each query. While this may be a normal practice in relational databases, with NoSQL it's better to model differently.

Given the use-case "get messages for user", this data model makes sense:

notifications: {
    message1: {
        tiltle:"abc",
        desc:"desc"
        user1 : true
        user2 : true
        user3 : true
    }
    message2: {
        tiltle:"efg",
        desc:"desc"
        user1 : true
        user2 : true
        user3 : true
    }
},
messages_per_user: {
    user1: {
        message1: true
        message2: true
    }
    user2: {
        message1: true
        message2: true
    }
    user3: {
        message1: true
        message2: true
    }
}

Now if you want to load the messages for a user, you do:

ref.child('messages_per_user').child('user1').on('value', function(snapshot) {
    snapshot.forEach(function(messageSnapshot) {
        ref.child('notifications').child(messageSnapshot.key).once('value', function(notificationSnapshot) {
            console.log(notificationSnapshot.val());
        })
    });
});

This way the database can directly access the correct nodes and ignore everything else.

Recommended reading:

Community
  • 1
  • 1
Frank van Puffelen
  • 565,676
  • 79
  • 828
  • 807
  • Can you please update the answer to include info about what node(s) to .indexOn in the data structure in your answer (from the OP's question), I think it will make it a more complete answer. – Jay Feb 06 '16 at 16:30
  • In my approach you don't need any indexes, since there are no calls to `orderBy...`. In the initial data structure you'd need an infinite number of indexes, since you'd need one for every user-id (I think). Or am I mistaken there? – Frank van Puffelen Feb 06 '16 at 16:57
  • Nope - you're on point. My thought was more about ordering of the data. If there were three messages in the structure (as they would appear in firebase): message1, message10, message2 there may be a need to work with them in the correct sequence: message1, message2, message10 which would require some way to order them. – Jay Feb 07 '16 at 13:26
  • Ah, got it. I was thinking of them as push ids. But indeed otherwise you'd need some value (instead of true) to order them by. This was necessary for the [Udacity Firebase course](https://www.udacity.com/course/firebase-essentials-for-android--ud009) and they ended up getting rid of the top-level "list" of items and just storing items per user only. – Frank van Puffelen Feb 07 '16 at 15:44
  • The structure proposed works for us for reads.But we have a problem with the number of writes that we need to make to firebase.Lets say there is a message which is applicable for about 1000 users.Looks like we need to patch each user path once (which means 1000 calls) to Firebase.Is there a way that we can request a bulk patch of multiple locations in a single HttpRequest? (The reason we want to avoid multiple HttpRequest is due to the restriction we have with our infrastructure.) – Neela Mauli Vangipurapu Feb 13 '16 at 16:56
  • You can do [multi-location updates with the Firebase SDKs](https://www.firebase.com/blog/2015-09-24-atomic-writes-and-more.html). If you're using REST, you can accomplish the same with a [REST PATCH call](https://www.firebase.com/blog/2015-09-24-atomic-writes-and-more.html#comment-2276358402). – Frank van Puffelen Feb 13 '16 at 17:00