Firebase for complex query. A no go?

Question

I've moved from parse-server to firebase for my new project, but reached a point in the project where I beginning to think it was a bad idea.

Basically, I'm making an app where people can post information about concerts going on in their town.

My first challenge was to filter the events, so a user only get events in his/her own town. I did this by structure the data after cities:

{
    concerts: {
        "New york": {
            ...,
            ...
        }, 
        "Chicago": {
            ...,
            ...
        }
    }
}

Then I figure I need another filter for the type of concert, e.g rock, pop, etc. So I though I did another restructure. However, there probably need to be 5-10 more filters, and it will become very hard to structure the database in a good way.

I though about multiple query, but this wasn't allowed:

firebase.database().ref("concerts")
.orderByChild("type").equalTo("rock")
.orderByChild("length").equalTo("2")
.orderByChild("artist").equalTo("beatles")

I thought about fetching everything from the server, and then filter the result in the client. I see however two problems with this:

1. There might be a ton of unnecessarily data being downloaded.
2. Some concerts will be locked only to certain users (e.g users who have gone to at least 10 other concerts), and there might be a security aspect of pulling home these concerts to user not being allowed to see them.

I thought about combining filters to create query keys, like this this, but with over 10 filters, it will become to complex.

Is there a solution to this or should I forget about firebase for this use case?

Thanks in advance

Answer 1

Incredibly complex queries can be crafted in Firebase. The data needs to be stored in a structure that lends itself to being queried and most importantly, don't be afraid of duplicate data.

For example, lets assume we have an app that enables a user to select a concert for a particular year and month, a specific city, and in a particular genre.

There are 3 parameters

year_month city genre

The UI first queries the user to select a City

Austin

then the UI asks to select a year and month

201704

then a genre

Rock

Your Firebase structure looks like this

concerts
  concert_00
   city: Memphis
   year_month: 201706
   genre: Country
   city_date_genre: Memphis_201606_Country
  concert_01
   city: Austin
   year_month: 201704
   genre: Rock
   city_date_genre: Austin_201704_Rock
  concert_02
   city: Seattle
   year_month: 201705
   genre: Disco
   city_date_genre: Seattle_201705_Disco

Your UI has already polled the user for the query info and with that, build a query string

Austin_201704_Rock

and then query the 'city_date_genre' node for that string and you have your data.

What if the user wanted to know all of the concerts in Austin for April 2017

queryStartingAt("Austin_201704").queryEndingAt("Austin_201704")

You could easily expand on this by adding another query node and changing the order of the data

concerts
  concert_00
   city: Memphis
   year_month: 201706
   genre: Country
   city_date_genre: Memphis_201606_Country
   city_genre_date: Memphis_Country_201606

And depending on which order the user selects their data, you could query the associated node.

Adding additional nodes is a tiny amount of data and allows for very open ended queries for the data you need.

Answer 2

I see this is an old post, but I'd like to take this opportunity to point others running into a similar Firebase issues to AceBase, which is is a free and open source alternative to the Firebase realtime database. The lack of proper querying and indexing options in Firebase was one of the reasons AceBase was built. Using AceBase would enable you to query your data like so:

const snapshots = await db.ref('concerts')
  .query()
  .filter('city', '==', 'New York')
  .filter('date', 'between', [today, nextWeek]) // today & nextWeek being Dates
  .filter('genre', 'in', ['rock', 'blues', 'country'])
  .get();

Because AceBase supports indexing, adding 1 or more indexes to the the queried fields will make those queries run incredibly fast, even with millions of records. It supports simple indexes, but also FullText and Geo indexes, so you could also query your data with a location and keywords:

.filter('location', 'geo:nearby', { lat: 40.730610, long: -73.935242, radius: 10000 }) // New York center with 10km radius
.filter('title', 'fulltext:contains', '"John Mayer" OR "Kenny Wayne Shepherd"')

If you want to limit results to allow paging, simply add skip and limit: .skip(80).limit(20)

Additionally, if you'd want to make the query deliver realtime results so any newly added concert will immediately notify your app - simply adding event listeners will upgrade it to a realtime query:

const results = await db.ref('concerts')
  .filter('location', 'geo:nearby', { lat: 40.730610, long: -73.935242, radius: 10000 })
  .on('add', addConcert)
  .on('remove', removeConcert)
  .get();

function addConcert(match) {
  results.push(match.snapshot);
  updateUI();
}
function removeConcert(match) {
  const index = results.findIndex(r => r.ref.path === match.ref.path);
  results.splice(index, 1);
  updateUI();
}

If you want to know more about AceBase, check it out at npm: https://www.npmjs.com/package/acebase. AceBase is free and its entire source code is available on GitHub. Have fun!