309

I'm just exploring the new Firebase Firestore and it contains a data type called reference. It is not clear to me what this does.

  • Is it like foreign key?
  • Can it be used to point to a collection that is located somewhere else?
  • If reference is an actual reference, can I use it for queries? For example can I have a reference that points directly to the user, instead of storing the userId in a text field? And can I use this user reference for querying?
Super Kai - Kazuya Ito
  • 22,221
  • 10
  • 124
  • 129
Jürgen Brandstetter
  • 7,066
  • 3
  • 35
  • 30
  • 31
    I think this video from firebase team breaks it down for you: youtube.com/watch?v=Elg2zDVIcLo (watch from 4:36) – Adarsh Jan 24 '19 at 05:03
  • 23
    https://youtu.be/Elg2zDVIcLo?t=276 – Trevor Jan 17 '20 at 20:50
  • 1
    I don't like to nest collections in firebase for multiple reasons. If for some reason you had another root level collection that you need to drill all the way down on a sibling root collection; let's say 4 levels to get to a document. This is made a lot easier by using refs and just using db.doc('some_saved_ref') vs matching all the ids out again... from the other root collection. – JustDave Jun 22 '21 at 16:34

9 Answers9

202

Adding below what worked for me using references in Firestore.

As the other answers say, it's like a foreign key. The reference attribute doesn't return the data of the reference doc though. For example, I have a list of products, with a userRef reference as one of the attributes on the product. Getting the list of products, gives me the reference of the user that created that product. But it doesn't give me the details of the user in that reference. I've used other back end as a services with pointers before that have a "populate: true" flag that gives the user details back instead of just the reference id of the user, which would be great to have here (hopefully a future improvement).

Below is some example code that I used to set the reference as well as get the collection of products list then get the user details from the user reference id given.

Set a reference on a collection:

let data = {
  name: 'productName',
  size: 'medium',
  userRef: db.doc('users/' + firebase.auth().currentUser.uid)
};
db.collection('products').add(data);

Get a collection (products) and all references on each document (user details):

db.collection('products').get()
    .then(res => {
      vm.mainListItems = [];
      res.forEach(doc => {
        let newItem = doc.data();
        newItem.id = doc.id;
        if (newItem.userRef) {
          newItem.userRef.get()
          .then(res => { 
            newItem.userData = res.data() 
            vm.mainListItems.push(newItem);
          })
          .catch(err => console.error(err));
        } else {
          vm.mainListItems.push(newItem);  
        }
        
      });
    })
    .catch(err => { console.error(err) });
starball
  • 20,030
  • 7
  • 43
  • 238
Ben Cochrane
  • 3,317
  • 1
  • 14
  • 16
  • 4
    Thanks for sharing! I think there is a typo in the first line of Get part and it should be `db.collection('products').get()`. Have you tried getting user directly? I'm guessing `newItem.userRef.get()` should work instead of `db.collection("users").doc(newItem.userRef.id).get()` – Sergey Nefedyev Dec 09 '17 at 01:11
  • Thanks @CRG good pickup on the typo and yep good point on the userRef.get() instead - that’s a simpler cleaner way to do it - thanks – Ben Cochrane Dec 09 '17 at 03:09
  • 84
    First of all thank you for the example. I hope they will add a "populate: true" for the future. Otherwise saving a reference is somewhat pointless. The same could have been done by simply saving the `uid` and reference via it. – Jürgen Brandstetter Dec 09 '17 at 15:05
  • 1
    @BenCochrane please edit your answer to include the @Sergey's suggestion regarding `newItem.userRef.get()` so that no one misses it if they don't read comments. I thought of editing but I wasn't sure if I should edit the answer to improve it from the code's perspective. – Sandip Fichadiya Dec 18 '17 at 11:24
  • 7
    Thanks for the example! But what is the point of storing the reference type if there is no "populate" kind of option when we query the document? If there is a populate option that anyone knows of, please let me know. – Harshil Shah Dec 22 '17 at 15:17
  • It seems that the admin-sdk [allows fetching multiple references at once](https://cloud.google.com/nodejs/docs/reference/firestore/0.12.x/Firestore#getAll). This could improve the code example to fetch all the refs in a single request (of course you pay per doc, not per request). I can't find the same function in the client sdk... strange! – codyzu Mar 21 '18 at 16:07
  • 29
    So in fact it's not like a foreign key. To me it does basically nothing - what's the point of having `reference` if we can't use it as a true foreign key should work? – jean d'arme May 15 '18 at 21:29
  • 1
    @jeand'arme agreed - not sure why they would have it without being able to populate that reference automatically – Ben Cochrane May 16 '18 at 21:50
  • 1
    Does `vm.mainListItems.push(newItem);` returns the userData in the object? I ask since its outside the `then` where you set up the value. – Johhan Santana Sep 06 '18 at 13:38
  • Hey @JohhanSantana great pick up! you are right - it wouldn't be adding it how it was. I've updated the code to now add it in - thanks :) – Ben Cochrane Sep 07 '18 at 06:16
  • 1
    Does getting a userRef count as a new operation, aka increases the firestore bill? – Miguel Stevens Nov 29 '18 at 13:33
  • 1
    @Notflip yes it does count as another operation as it's another call to firestore to get each user in the example above – Ben Cochrane Nov 30 '18 at 04:17
  • can I use populate with Firestore SDK on ios? – Yahya Tabba Dec 11 '18 at 10:00
  • Yahya Tabba populate isn’t something that is available in the firestore sdk (iOS or otherwise) as I understand - I was more referring that populate is available in other no sql backends ive used previously. There is a similar question with the same outcome of needing to request the data of the reference doc in a separate call https://stackoverflow.com/questions/49241477/how-can-i-populate-the-reference-field-using-firestore – Ben Cochrane Dec 12 '18 at 10:15
  • 28
    So the only advantage of a `reference` over a `string` is, that you can call `get()` on the reference directly. Not very useful yet. Hope they add an option to automatically populate references with the corresponding objects! – morgler Dec 12 '18 at 16:01
  • 1
    @morgler yep agreed – Ben Cochrane Dec 13 '18 at 07:57
  • 1
    @BenCochrane so this still hasn't changed? I must manually get actual value from the reference, and until then I'm stuck with just a string? – Sebastijan Dumančić Dec 13 '18 at 13:56
  • Any possibility for doing this automatically with annotations? – softmarshmallow Jan 15 '19 at 10:56
  • hey @uzu I'm not across annotations - would you have an example of what this is? – Ben Cochrane Jan 16 '19 at 01:58
  • 2
    For me using it like this will be more problematic than it should be. We can use ref or add some field like "userID: string" which will allow us to find it other way. The reason why I dont like ref approach is coupling data model (which is schema-less), so you will not DRY. I think its better to have a single service which manages the collection and maps data, instead of referencing differenct collections in single service. It will cause problems if data changes. – Łukasz Ostrowski Jun 24 '19 at 10:45
  • This is strange. I have the same approach on my code, but when I try to get the same thing like you with `vm.mainListItems[0].userData` it return `undefined`. @BenCochrane, how is it possible? – C-lio Garcia Dec 12 '20 at 10:22
  • Thanks for the Answer. This will be very useful for the database design I am trying to form for my app I am working in. – Gautham Vijayan May 25 '22 at 15:50
138

References are very much like foreign keys.

The currently released SDKs cannot store references to other projects. Within a project, references can point to any other document in any other collection.

You can use references in queries like any other value: for filtering, ordering, and for paging (startAt/startAfter).

Unlike foreign keys in a SQL database, references are not useful for performing joins in a single query. You can use them for dependent lookups (which seem join like), but be careful because each hop will result in another round trip to the server.

jub0bs
  • 60,866
  • 25
  • 183
  • 186
Gil Gilbert
  • 7,722
  • 3
  • 24
  • 25
  • 12
    Please, can you share possible use cases? Is it possible to query fields in that reference? E.g. I have a `friends` collection listing all my friends (`friends/myId`). Then, I reference this document in the `friends` field of another document (`group/groupId`). I'd like to display only my friends who are in that group, doing something like this: `where('friends.myId', '==', true)`. – Will Oct 04 '17 at 23:19
  • 143
    Btw, it might be useful to update the [docs](https://firebase.google.com/docs/firestore/manage-data/add-data) to include an example of adding a reference type. – Will Oct 04 '17 at 23:32
  • 13
    I can't find any info about this? This will change my whole database structure, I need to know more ... – Ruben Nov 07 '17 at 21:02
  • 3
    do you have an example (preferably in swift) on how to query using reference? right now, I can do it by storing the raw uid as string, but that's doesn't seems right. – Mickey Cheong Feb 21 '18 at 09:40
  • 7
    I'm needing to change all my reference types to strings because the query's will always fail with a reference type. I literally cannot find anything about how to query by reference type :( if anyone finds out how to query by reference types let me know... – Sam Trent May 25 '18 at 20:59
  • 1
    @SamTrent Did you get any solutions for reference type i also have the same issue. – rajlaxmi_jagdale Jan 31 '19 at 05:28
  • @rajlaxmi_jagdale The reference types never wound up working for me, so instead I just typed them as strings :\ – Sam Trent Jan 31 '19 at 05:31
  • @SamTrent I found a solution if you have reference in arraylist you need to write code as ArrayList group = (ArrayList) objectMap.get("abc"); DocumentReference documentReference = (DocumentReference) group.get(0); documentReference.getPath(); and if you have just reference in field just call DocumentReference documentReference = (DocumentReference) objectMap.get("abc"); documentReference.getPath(); – rajlaxmi_jagdale Jan 31 '19 at 06:29
  • @MickeyCheong, Google API explorer is helpful here: https://developers.google.com/apis-explorer/#p/firestore/v1beta1/ You can, for example, use the reference in the `batchGet` operation (in my use case, I'm using the REST API). – rm.rf.etc Jul 22 '19 at 17:16
  • Can you see my same question https://stackoverflow.com/questions/68442405/how-to-set-where-condition-on-array-of-reference-selection-in-flutter-using-clou – Sakthi Karthik Jul 22 '21 at 10:26
  • I have the same question from the OP and I still don't think I understood. If there is no populate:true in firestore, what is the point to saving a reference if I will have the same trouble to pull the reference data as if I just saved a string with the reference id? – Fred Guth Aug 18 '22 at 13:00
  • A reference is similar to a string containing the reference id. A string ID is smaller to store but requires extra code to reconstruct the reference. A reference is fully qualified, so it's larger, but easier to use for fetching the related document. – Gil Gilbert Oct 01 '22 at 02:03
33

For those looking for a Javascript solution to querying by reference - the concept is that, you need to use a 'document reference' object in the query statement

teamDbRef = db.collection('teams').doc('CnbasS9cZQ2SfvGY2r3b'); /* CnbasS9cZQ2SfvGY2r3b being the collection ID */
//
//
db.collection("squad").where('team', '==', teamDbRef).get().then((querySnapshot) => {
  //
}).catch(function(error) {
  //
});

(Kudos to the answer here: https://stackoverflow.com/a/53141199/1487867)

Aswin Kumar
  • 5,158
  • 5
  • 34
  • 39
11

According to the #AskFirebase https://youtu.be/Elg2zDVIcLo?t=276 the primary use-case for now is a link in Firebase console UI

Pavel Shastov
  • 2,657
  • 1
  • 18
  • 26
11

A lot of answers mentioned it is just a reference to another document but does not return data for that reference but we can use it to fetch data separately.

Here is an example of how you could use it in the firebase JavaScript SDK 9, modular version.

let's assume your firestore have a collection called products and it contains the following document.

{
  name: 'productName',
  size: 'medium',
  userRef: 'user/dfjalskerijfs'
}

here users have a reference to a document in the users collection. we can use the following code segment to get the product and then retrieve the user from the reference.

import { collection, getDocs, getDoc, query, where } from "firebase/firestore";
import { db } from "./main"; // firestore db object

let productsWithUser = []
const querySnaphot = await getDocs(collection(db, 'products'));
querySnapshot.forEach(async (doc) => {
  let newItem = {id: doc.id, ...doc.data()};
  if(newItem.userRef) {
    let userData = await getDoc(newItem.userRef);
    if(userData.exists()) {
      newItem.userData = {userID: userData.id, ...userData.data()}
    }
    productwithUser.push(newItem);
  } else {
    productwithUser.push(newItem);
  }
});

here collection, getDocs, getDoc, query, where are firestore related modules we can use to get data whenever necessary. we use user reference returned from the products document directly to fetch the user document for that reference using the following code,

let userData = await getDoc(newItem.userRef);

to read more on how to use modular ver SDK refer to official documentation to learn more.

Vikum Dheemantha
  • 764
  • 8
  • 24
5

If you don't use Reference data type, you need to update every document.

For example, you have 2 collections "categories" and "products" and you stored the category name "Fruits" in categories to every document of "Apple" and "Lemon" in products as shown below. But, if you update the category name "Fruits" in categories, you also need to update the category name "Fruits" in every document of "Apple" and "Lemon" in products:

collection | document | field

categories > 67f60ad3 > name: "Fruits"
collection | document | field

  products > 32d410a7 > name: "Apple", category: "Fruits"
             58d16c57 > name: "Lemon", category: "Fruits"

But, if you store the reference of "Fruits" in categories to every document of "Apple" and "Lemon" in products, you don't need to update every document of "Apple" and "Lemon" when you update the category name "Fruits" in categories:

collection | document | field

  products > 32d410a7 > name: "Apple", category: categories/67f60ad3
             58d16c57 > name: "Lemon", category: categories/67f60ad3

This is the goodness of Reference data type.

sideshowbarker
  • 81,827
  • 26
  • 193
  • 197
Super Kai - Kazuya Ito
  • 22,221
  • 10
  • 124
  • 129
  • 8
    The discussion is not so much about storing the static name vs. a 'Foreign-Key-Like' id; but rather the benefit of using a doc reference vs. just using the doc ID as a string. – egalvan10 Jan 19 '22 at 21:30
1

Belatedly, there are two advantages from this blog:

enter image description here

if I expect that I'll want to order restaurant reviews by rating, or publish date, or most upvotes, I can do that within a reviews subcollection without needing a composite index. In the larger top level collection, I'd need to create a separate composite index for each one of those, and I also have a limit of 200 composite indexes.

I wouldn't have 200 composite indices but there are some constraints.

Also, from a security rules standpoint, it's fairly common to restrict child documents based on some data that exists in their parent, and that's significantly easier to do when you have data set up in subcollections.

One example would be restricting to insert a child collection if the user doesn't have the privilege in the parent's field.

Dharman
  • 30,962
  • 25
  • 85
  • 135
Taku
  • 5,639
  • 2
  • 42
  • 31
1

UPDATE 12/18/22 - I put this in a package.

Original Blog Post

What this package does is use RXJS to loop through each field in a document. If that document type is a Reference type, then it grabs that foreign document type. The collection version grabs the foreign key value for each reference field in all documents in your collection. You can also input the fields manually that you wish to parse to speed up the searching (see my post). This is definitely not as efficient as doing manual aggregations with Firebase Functions, as you will get charged for lots of reads for each document you read, but it could come in handy for people that want a quick way to join data on the frontend.

This could also come in handy if you cache data and really only need to do this once.

J

install

npm i j-firebase

import

import { expandRef, expandRefs } from 'j-firebase';

https://github.com/jdgamble555/j-firebase


Original Post


Automatic JOINS:

DOC

expandRef<T>(obs: Observable<T>, fields: any[] = []): Observable<T> {
  return obs.pipe(
    switchMap((doc: any) => doc ? combineLatest(
      (fields.length === 0 ? Object.keys(doc).filter(
        (k: any) => {
          const p = doc[k] instanceof DocumentReference;
          if (p) fields.push(k);
          return p;
        }
      ) : fields).map((f: any) => docData<any>(doc[f]))
    ).pipe(
      map((r: any) => fields.reduce(
        (prev: any, curr: any) =>
          ({ ...prev, [curr]: r.shift() })
        , doc)
      )
    ) : of(doc))
  );
}

COLLECTION

expandRefs<T>(
  obs: Observable<T[]>,
  fields: any[] = []
): Observable<T[]> {
  return obs.pipe(
    switchMap((col: any[]) =>
      col.length !== 0 ? combineLatest(col.map((doc: any) =>
        (fields.length === 0 ? Object.keys(doc).filter(
          (k: any) => {
            const p = doc[k] instanceof DocumentReference;
            if (p) fields.push(k);
            return p;
          }
        ) : fields).map((f: any) => docData<any>(doc[f]))
      ).reduce((acc: any, val: any) => [].concat(acc, val)))
        .pipe(
          map((h: any) =>
            col.map((doc2: any) =>
              fields.reduce(
                (prev: any, curr: any) =>
                  ({ ...prev, [curr]: h.shift() })
                , doc2
              )
            )
          )
        ) : of(col)
    )
  );
}

Simply put this function around your observable and it will automatically expand all reference data types providing automatic joins.

Usage

this.posts = expandRefs(
  collectionData(
    query(
      collection(this.afs, 'posts'),
      where('published', '==', true),
      orderBy(fieldSort)
    ), { idField: 'id' }
  )
);

Note: You can also now input the fields you want to expand as a second argument in an array.

['imageDoc', 'authorDoc']

This will increase the speed!

Add .pipe(take(1)).toPromise(); at the end for a promise version!

See here for more info. Works in Firebase 8 or 9!

Simple!

J

Jonathan
  • 3,893
  • 5
  • 46
  • 77
  • 2
    Please could you explain your code a bit more? I'm finding it difficult to understand what the pipes and map-reducers are doing. Thanks! – dylan-myers Jul 09 '22 at 23:16
  • I agree with Dylan. This code is super hard to read and that makes it difficult to receive any benefit from it. Can you try commenting what some parts of it are doing. It also looks like you have nested `map` and `reduce` that would add significant amounts of runtime complexity. – Abir Taheer Dec 19 '22 at 01:44
  • this is not an update, this is someone trying to build a third party library on top of firebase's mistake... – Rafael Lima Feb 02 '23 at 15:33
  • @AbirTaheer - It only goes through every field with map if you do not input the specific fields you want to expand. @-RafaelLima - Agreed this a problem with Firestore, but I'm just trying to share the code that I wrote and find useful. Even if Firestore had joins, it would sill charge you to read the other documents. I can make comments on the code itself, but it really just goes through every field that is of reference type and expands it, or only the fields you input. – Jonathan Feb 03 '23 at 15:13
1

2022 UPDATE

let coursesArray = [];
const coursesCollection = async () => {
    const queryCourse = query(
        collection(db, "course"),
        where("status", "==", "active")
    )
    onSnapshot(queryCourse, (querySnapshot) => {
        querySnapshot.forEach(async (courseDoc) => {

            if (courseDoc.data().userId) {
                const userRef = courseDoc.data().userId;
                getDoc(userRef)
                    .then((res) => {
                        console.log(res.data());
                    })
            }
            coursesArray.push(courseDoc.data());
        });
        setCourses(coursesArray);
    });
}
Enrique Flores
  • 716
  • 4
  • 13