185

I thought I read that you can query subcollections with the new Firebase Firestore, but I don't see any examples. For example I have my Firestore setup in the following way:

  • Dances [collection]
    • danceName
    • Songs [collection]
      • songName

How would I be able to query "Find all dances where songName == 'X'"

Doug Stevenson
  • 297,357
  • 32
  • 422
  • 441
Nelson.b.austin
  • 3,080
  • 6
  • 37
  • 63

11 Answers11

202

Update 2019-05-07

Today we released collection group queries, and these allow you to query across subcollections.

So, for example in the web SDK:

db.collectionGroup('Songs')
  .where('songName', '==', 'X')
  .get()

This would match documents in any collection where the last part of the collection path is 'Songs'.

Your original question was about finding dances where songName == 'X', and this still isn't possible directly, however, for each Song that matched you can load its parent.

Original answer

This is a feature which does not yet exist. It's called a "collection group query" and would allow you query all songs regardless of which dance contained them. This is something we intend to support but don't have a concrete timeline on when it's coming.

The alternative structure at this point is to make songs a top-level collection and make which dance the song is a part of a property of the song.

Frank van Puffelen
  • 565,676
  • 79
  • 828
  • 807
Gil Gilbert
  • 7,722
  • 3
  • 24
  • 25
  • Would you reference the dance from the song using a firestore reference? Can you query by those references? – Siôn Oct 04 '17 at 20:45
  • Whether or not you use a Firestore reference is up to you. You can query by them. If you only want to use the references to look up the dance then a reference is sufficient. If you want to be able to display something about the dance from within a song without looking up the dance then I wouldn't use a reference. This is the classic nosql denormalization problem though. – Gil Gilbert Oct 04 '17 at 20:49
  • So my problem is I want to query by dance name AND by song name. A single dance can have multiple songs associated with it. Even further I need to be able to rate the dance, AND the song associated to the dance. Ex: Song A and Song B go to Dance A. Dance A is fun, 5 star - BUT Song A doesn't really fit (down vote), but Song B does (up vote). So I'll definitely be utilizing Firebase Functions to keep track of an average rating of both. But I still need to be able to query WHERE songName = 'X' and also WHERE danceName = 'Y'. Will Firestore be able to support this in the future? – Nelson.b.austin Oct 05 '17 at 04:14
  • In general Firestore does not support joins so you can't query WHERE songs.songName = 'X' AND dances.danceName = 'Y'. Just like any other NoSQL store, the traditional way to deal with this is to denormalize your data such that each song contains enough data that you query just on songs. Alternatively do the query in two steps, manually joining for yourself. In your case this could be viable since it sounds like dance names may be unique or close to it. – Gil Gilbert Oct 05 '17 at 04:39
  • @GilGilbert Can you show an example how you query by Reference in javascript? I can't find any docs showing that. – cdock Oct 05 '17 at 05:13
  • 2
    @cdock you can just use a simple query. If you store a reference to the dance in the song, then you can.. `songsCollectionRef.where('dance', '==', danceDocument.ref)` – Siôn Oct 05 '17 at 06:43
  • 2
    It would not be a good solution when instead songName you have a dynamic value, like wind speed or temperature. How you suggest to approach that kind of challenge? – norgematos Oct 05 '17 at 12:21
  • 172
    It would be MUCH better if the Firestore dev team implemented subcollection queries ASAP. After all, 'more powerful queries' is one of the major selling points according to the Firestore manual. Right now, Firestore is like a Porsche without wheels. – Arne Wolframm Nov 04 '17 at 00:38
  • 23
    We agree! There's just a limited number of hours in the day :-). – Gil Gilbert Nov 04 '17 at 20:54
  • 25
    I don't understand, for what do people pay, if the firebase is limited? Seems like even Backendless has more functionality than Firebase. And why is Firebase so popular? Seems people got crazy – nzackoya Nov 10 '17 at 21:56
  • @GilGilbert : Q1. If and when the "collection group query "join" like feature is implemented by the FS team, would you think that it would be implemented so that also sibling collections and not just subcollections could be joined? (I.e, the "join" would also work if Songs was not nested under Dances) Q2. Any non-concrete ballpark timing on the release of this feature? – JLS Nov 15 '17 at 08:18
  • Collection group queries work with all collections of the same name, regardless of location. The only ballpark I can offer on when is "not soon". – Gil Gilbert Nov 22 '17 at 22:21
  • 16
    This feature is highly needed or else people will start finding alternatives, even we have deadlines to meet. :P – JD-V Nov 30 '17 at 13:11
  • 5
    Since months pass, do we have a more precise release date because we need to choose our technology in next weeks and would very like to use Firestone. – HKoncept Feb 03 '18 at 17:23
  • 13
    We need this feature . At lease the timeline to release this will help us to be prepared. – sanjaya Kumar panigrahy Feb 08 '18 at 11:40
  • 8
    I am beginning to regret my decision to use Firestore :( I love it for some features, but when I hit walls like this and I'm left restructuring my data again, I ask why I'm not using something else. I understand it's in beta... just sad I guess. – stevejboyer Apr 03 '18 at 00:12
  • Hi, is there any timeline or issue tracker where this is being tracked? – Yashovardhan99 Jul 02 '18 at 16:51
  • 1
    Sorry we don't have a public roadmap that we can commit to on this and other issues regarding future releases. – Gil Gilbert Jul 10 '18 at 22:11
  • @GilGilbert I know it's hard to answer questions about unreleased features. Our biggest concern is: would collection group queries, as they are currently envisioned, be sufficient to perform an efficient "map/reduce" operation across a dataset? For instance, scaling out a huge data migration of Songs by splitting the work efficiently between multiple machines. This was a major headache with Datastore's entity groups. – Nick Farina Jul 16 '18 at 19:56
  • 1
    Collection group queries are a query just like any other. It's not going to be any better or worse for that purpose than if you had all your songs in a single top-level collection and ran a query on that. – Gil Gilbert Jul 20 '18 at 18:58
  • @GilGilbert - I have a game with seats. In order to listen to seat occupation efficiently, they need to be their own documents in a sub-collection of seats in the games collection. Having seats in the top collection will cause the seat listeners to fire needlessly. But now I cannot query games with available seats. I understand that our complaints sound ungrateful but you charge per read. It is easy to assume that Firestore is incentivized to delay sub-collection queries in order to bill more reads. That doesn't sit well in my craw. If I am wrong please explain. – Adrian Bartholomew Dec 30 '18 at 09:12
  • 1
    Any subcollection arrangement can be transformed into an equivalent flat collection with attributes for what would have been parent keys. So instead of loading db.collection('games/game-1/seats').where('available', '==', true) do db.collection(seats).where('game', '==', 'game-1').where('available', '==', true). The schemes are equivalently powerful. We want to support collection group queries though to give flexibility--so that you don't have to rewrite your data when you realize you need to query across parent collections. – Gil Gilbert Jan 31 '19 at 19:15
  • Happened to come across this on the day you added collection group queries! That's awesome, thanks Firebase team! – Matt Palmerlee May 08 '19 at 02:36
  • Wow. That's great! Just when i needed this update. I've been stuck for 3 days trying to query a collection group. – T double May 08 '19 at 08:49
  • @GilGilbert Any idea when the firebase main library will be updated with this - https://github.com/firebase/firebase-js-sdk/tree/master/packages/firestore ? – seaders May 09 '19 at 16:36
  • 1
    It's there https://github.com/firebase/firebase-js-sdk/blob/master/packages/firebase/index.d.ts#L6008. Note that this was released with firebase 6, so you need to change the major version you're depending upon. – Gil Gilbert May 09 '19 at 17:47
  • Is this available yet on iOS? The documentation is updated with code snippets for every client/language except for swift and objective c... @GilGilbert – claramelon May 13 '19 at 22:37
  • It is available: [API docs](https://firebase.google.com/docs/reference/swift/firebasefirestore/api/reference/Classes/Firestore.html#collectiongroup_:). I'm not sure why the snippets haven't been added for Swift/Objective-C, but this is a docs bug. – Gil Gilbert May 14 '19 at 17:46
  • Please allow us to easily make copy/paste of document IDs from Firestore GUI to mock up manually some sample to query. – camillo777 Oct 22 '19 at 12:33
  • 1
    @GilGilbert you have mentioned "for each Song that matched you can load its parent" . How to load parent of matched Song? – sachin rathod Jan 26 '20 at 11:00
  • 1
    DocumentReference and CollectionReference both have parent methods. Use those to navigate to the level you want and then load that document. – Gil Gilbert Jan 27 '20 at 15:45
  • 2
    This is crazy, to have nested collections where you need the parent to understand the path and depth you are at... Why do we have nested collections at all? Firestore is no database, a DOM with XPath gives you more flexibility at this point – dnhyde Mar 13 '20 at 10:23
  • 1
    @GilGilbert - Can I get all `Songs` subcollections but not from **all** documents, but only if the document ID is in a documents ID list that I supply? – Alaa M. Jul 18 '20 at 10:32
26

UPDATE Now Firestore supports array-contains

Having these documents

    {danceName: 'Danca name 1', songName: ['Title1','Title2']}
    {danceName: 'Danca name 2', songName: ['Title3']}

do it this way

collection("Dances")
    .where("songName", "array-contains", "Title1")
    .get()...

@Nelson.b.austin Since firestore does not have that yet, I suggest you to have a flat structure, meaning:

Dances = {
    danceName: 'Dance name 1',
    songName_Title1: true,
    songName_Title2: true,
    songName_Title3: false
}

Having it in that way, you can get it done:

var songTitle = 'Title1';
var dances = db.collection("Dances");
var query = dances.where("songName_"+songTitle, "==", true);

I hope this helps.

norgematos
  • 610
  • 8
  • 18
  • 2
    what is `songName_Title3: false` useful for? if I am not wrong, it can only be used to search for dances that don't have a specific song name assuming that us need a `songName_Title3: false` to make `dances.where("songName_"+songTitle, "==", false); ` return such results it would not make sence for every dance to have boolean flags for every possible song name... – epeleg Mar 26 '18 at 19:54
  • 1
    This is great but documents are limited to 1MB, so if you need to associate a long list of strings or whatever with a specific document, then you can't use this approach. – Supertecnoboff Feb 13 '19 at 22:04
  • @Supertecnoboff That seems like it would have to be an awfully large and long list of strings. How performant is this "array_contains" query and what are more performant alternatives? – Jay Ordway Mar 05 '19 at 05:12
19

UPDATE 2019

Firestore have released Collection Group Queries. See Gil's answer above or the official Collection Group Query Documentation


Previous Answer

As stated by Gil Gilbert, it seems as if collection group queries is currently in the works. In the mean time it is probably better to use root level collections and just link between these collection using the document UID's.

For those who don't already know, Jeff Delaney has some incredible guides and resources for anyone working with Firebase (and Angular) on AngularFirebase.

Firestore NoSQL Relational Data Modeling - Here he breaks down the basics of NoSQL and Firestore DB structuring

Advanced Data Modeling With Firestore by Example - These are more advanced techniques to keep in the back of your mind. A great read for those wanting to take their Firestore skills to the next level

Matthew Mullin
  • 7,116
  • 4
  • 21
  • 35
18

What if you store songs as an object instead of as a collection? Each dance as, with songs as a field: type Object (not a collection)

{
  danceName: "My Dance",
  songs: {
    "aNameOfASong": true,
    "aNameOfAnotherSong": true,
  }
}

then you could query for all dances with aNameOfASong:

db.collection('Dances')
  .where('songs.aNameOfASong', '==', true)
  .get()
  .then(function(querySnapshot) {
    querySnapshot.forEach(function(doc) {
      console.log(doc.id, " => ", doc.data());
    });
   })
   .catch(function(error) {
     console.log("Error getting documents: ", error);
    });
Christopher Moore
  • 15,626
  • 10
  • 42
  • 52
dmartins
  • 261
  • 2
  • 2
  • 11
    This solution would work but it is not scalable in case the number of songs is large or can grow dynamically. This would increase the document size and affect the read/write performance. More about this can be found in Firebase documentation linked below (see the last section 'Limitations' on the page) https://firebase.google.com/docs/firestore/solutions/arrays – Nouman Hanif Feb 13 '18 at 21:49
10

NEW UPDATE July 8, 2019:

db.collectionGroup('Songs')
  .where('songName', isEqualTo:'X')
  .get()
Nhật Trần
  • 2,522
  • 1
  • 20
  • 20
4

You can always search like this:-

    this.key$ = new BehaviorSubject(null);

    return this.key$.switchMap(key =>
      this.angFirestore
        .collection("dances").doc("danceName").collections("songs", ref =>
          ref
            .where("songName", "==", X)
        )
        .snapshotChanges()
        .map(actions => {
          if (actions.toString()) {
            return actions.map(a => {
              const data = a.payload.doc.data() as Dance;
              const id = a.payload.doc.id;
              return { id, ...data };
            });
          } else {
            return false;
          }
        })
    );
raaaay
  • 496
  • 7
  • 14
Ankur
  • 177
  • 3
  • 15
4

I have found a solution. Please check this.

var museums = Firestore.instance.collectionGroup('Songs').where('songName', isEqualTo: "X");
        museums.getDocuments().then((querySnapshot) {
            setState(() {
              songCounts= querySnapshot.documents.length.toString();
            });
        });

And then you can see Data, Rules, Indexes, Usage tabs in your cloud firestore from console.firebase.google.com. Finally, you should set indexes in the indexes tab.enter image description here

Fill in collection ID and some field value here. Then Select the collection group option. Enjoy it. Thanks

Happy Son
  • 81
  • 4
  • 1
    This do not answer the question. Query mentioned above just fetches all the Songs with songName = 'X' . This won't provide the dances where songName = 'X' . – sachin rathod Jan 26 '20 at 10:54
3

Query limitations

Cloud Firestore does not support the following types of queries:

  1. Queries with range filters on different fields.

  2. Single queries across multiple collections or subcollections. Each query runs against a single collection of documents. For more information about how your data structure affects your queries, see Choose a Data Structure.

  3. Logical OR queries. In this case, you should create a separate query for each OR condition and merge the query results in your app.

  4. Queries with a != clause. In this case, you should split the query into a greater-than query and a less-than query. For example, although the query clause where("age", "!=", "30") is not supported, you can get the same result set by combining two queries, one with the clause where("age", "<", "30") and one with the clause where("age", ">", 30).

Community
  • 1
  • 1
ggDeGreat
  • 1,098
  • 1
  • 17
  • 33
3

I'm working with Observables here and the AngularFire wrapper but here's how I managed to do that.

It's kind of crazy, I'm still learning about observables and I possibly overdid it. But it was a nice exercise.

Some explanation (not an RxJS expert):

  • songId$ is an observable that will emit ids
  • dance$ is an observable that reads that id and then gets only the first value.
  • it then queries the collectionGroup of all songs to find all instances of it.
  • Based on the instances it traverses to the parent Dances and get their ids.
  • Now that we have all the Dance ids we need to query them to get their data. But I wanted it to perform well so instead of querying one by one I batch them in buckets of 10 (the maximum angular will take for an in query.
  • We end up with N buckets and need to do N queries on firestore to get their values.
  • once we do the queries on firestore we still need to actually parse the data from that.
  • and finally we can merge all the query results to get a single array with all the Dances in it.
type Song = {id: string, name: string};
type Dance = {id: string, name: string, songs: Song[]};

const songId$: Observable<Song> = new Observable();
const dance$ = songId$.pipe(
  take(1), // Only take 1 song name
  switchMap( v =>
    // Query across collectionGroup to get all instances.
    this.db.collectionGroup('songs', ref =>
      ref.where('id', '==', v.id)).get()
  ),
  switchMap( v => {
    // map the Song to the parent Dance, return the Dance ids
    const obs: string[] = [];
    v.docs.forEach(docRef => {
      // We invoke parent twice to go from doc->collection->doc
      obs.push(docRef.ref.parent.parent.id);
    });
    // Because we return an array here this one emit becomes N
    return obs;
  }),
  // Firebase IN support up to 10 values so we partition the data to query the Dances
  bufferCount(10),
  mergeMap( v => { // query every partition in parallel
    return this.db.collection('dances', ref => {
      return ref.where( firebase.firestore.FieldPath.documentId(), 'in', v);
    }).get();
  }),
  switchMap( v => {
    // Almost there now just need to extract the data from the QuerySnapshots
    const obs: Dance[] = [];
    v.docs.forEach(docRef => {
      obs.push({
        ...docRef.data(),
        id: docRef.id
      } as Dance);
    });
    return of(obs);
  }),
  // And finally we reduce the docs fetched into a single array.
  reduce((acc, value) => acc.concat(value), []),
);
const parentDances = await dance$.toPromise();

I copy pasted my code and changed the variable names to yours, not sure if there are any errors, but it worked fine for me. Let me know if you find any errors or can suggest a better way to test it with maybe some mock firestore.

Eduardo
  • 22,574
  • 11
  • 76
  • 94
2
var songs = []    
db.collection('Dances')
      .where('songs.aNameOfASong', '==', true)
      .get()
      .then(function(querySnapshot) {
        var songLength = querySnapshot.size
        var i=0;
        querySnapshot.forEach(function(doc) {
           songs.push(doc.data())
           i ++;
           if(songLength===i){
                console.log(songs
           }
          console.log(doc.id, " => ", doc.data());
        });
       })
       .catch(function(error) {
         console.log("Error getting documents: ", error);
        });
Alok Prusty
  • 330
  • 3
  • 4
1

It could be better to use a flat data structure.
The docs specify the pros and cons of different data structures on this page.

Specifically about the limitations of structures with sub-collections:

You can't easily delete subcollections, or perform compound queries across subcollections.

Contrasted with the purported advantages of a flat data structure:

Root-level collections offer the most flexibility and scalability, along with powerful querying within each collection.

MattCochrane
  • 2,900
  • 2
  • 25
  • 35