0

Using a MEAN stack and Mongoose (5.9), please consider this model of a User document:

{
   "_id" : ObjectId,
   "Username" : "SamWhatever",
   "Followers" : [                 //may potentially grow to a huge number
      ForeignObjectId,
      ForeignObjectId,
      ForeignObjectId,
      ...
   ],
   "FollowersAmount": 20000001 //is it wise to substitute this by some native MongoDB functionality?
}

I am keeping track of followers in an array to possibly list all of them (via populate). On the profile page of a user, I want to display the amount of followers. Of course, I don't want to transmit the entire followers array but rather just send the plain amount of followers to the browser. I am not sure whether I should keep an additional FollowersAmount field in each user document which - of course - also requires further logic to always keep the value up-to-date or if I should rather use a native Mongoose functionality (which will also always be fast enough, irrespective of the future amount of items in that array) to count the amount of followers on-the-fly when querying a user and then append the result to the result set. Something like (pseudocode):

searchquery = User.findById({_id: id});

searchquery.populate({
    count: 'Followers'
});

Any best practice advice what to use? Again, I just want to count the number of items in ONE array of ONE document, NOT ACROSS MANY documents. Besides, I expect the array Followers to grow to a huge number and would prefer a solution that scales.

Igor P.
  • 1,407
  • 2
  • 20
  • 34
  • Does this answer your question? [MongoDB: count the number of items in an array](https://stackoverflow.com/questions/21387969/mongodb-count-the-number-of-items-in-an-array) You don't need to use `.populate`, Now you've two options either use aggregation as stated in above link or once you get result in code - you can just do `thatArray.length` ! – whoami - fakeFaceTrueSoul Jun 02 '20 at 18:23
  • Not entirely. As stated, I was also seeking best practice advice whether native MongoDB counting methods also scale as the Followers array may potentially rise to several million items or if I just should have redundant fields which contain this meta-data? – Igor P. Jun 02 '20 at 18:31
  • Besides, I am counting the items of an array in just ONE document, not across MANY documents. – Igor P. Jun 02 '20 at 18:32
  • @IgorP. IMO best way to go about this will be to precompute the number of array elements when you're pushing or popping elements from the array, and storing the result. – Ayush Gupta Jun 02 '20 at 18:36
  • 1
    @Igor P. You can have a field to inc or dec count or You need to use `$match` as first stage to filter for required document ! After that accepted answer does also counts from only one document as it's not doing any grouping !! Regarding best practice no one would suggest to create an array with millions of items either in MongoDB or in any programming language ! think of indexing on array with millions of items !! **best practice advice**-It really depends on your environment you're working on server size/ram/cpu we can't advice on it, it's actually self determined by research on own system. – whoami - fakeFaceTrueSoul Jun 02 '20 at 18:38
  • @AyushGupta In other words, always keep track of this meta-data using additional fields inside the user document (as proposed in my pseudo-code) and thus make sure, that querying a user will always be performed at computational low cost, irrespective of the size of the Followers array. – Igor P. Jun 02 '20 at 18:42
  • @IgorP. yes, that's right. – Ayush Gupta Jun 02 '20 at 18:45
  • @whoami Hm, ok. So how should I then keep track of followers just in case I need to list all followers of a specific user? Querying all users whether they follow a specific user would be other way round and IMHO will most likely also perform poor as the total number of users rises significantly. Do you see what my problem is? There is heaps of tutorials out there how to set up a simple MongoDB structure, but pretty much all turorials lack information about crucial ingredients to set up web applications that may exceed a certain size. Very frustrating. – Igor P. Jun 02 '20 at 18:45
  • @IgorP. your problem depends on your data. What's the average number of followers you expect a person to have, and what's the average number of people any person will follow become important parts of the problem, among a lot of things. – Ayush Gupta Jun 02 '20 at 18:47
  • As an example, Think of it this way, instead of saving all followers of a user in the `user` document, why not store the users that user is following? – Ayush Gupta Jun 02 '20 at 18:48
  • @AyushGupta Just to get things right and avoid any misunderstandings. My Followers array just stores the documentIDs of the followers, NOT ALL contents of those "foreign" documents. These IDs in the array are just references to documents in a separate collection. I thought, by doing that I am doing exactly what you proposed: "store the users that user is following" -> by storing their documentIDs in an array or what other way were you thinking of? – Igor P. Jun 02 '20 at 18:50
  • @IgorP. I understand that. What I'm saying is that high-level "best practices" are really situation and data dependent. There's rarely any one answer which fits all cases. – Ayush Gupta Jun 02 '20 at 18:52
  • Ok, to sum things up, I think I will then stick to additional META-DATA fields. Thanks guys. – Igor P. Jun 02 '20 at 18:58

0 Answers0