8

I'm creating a social site using mean stack and I need some suggestions regarding mongoDB and mongoose. I'm part of a startup and we decided to use these amazing technologies to fulfil our task.

Basically, I need some suggestions.

Currently, I have finished creation of simple CRUD and implemented local passport JS. I have currently one single collection in my mongoDB called users.

Our social site will have a blog, marketplace and many other pages (features) that will be related to a single user.

Since I never worked with mongoDB before, I'm curious if mongoDB should use one collection per user or have multiple collections for each feature.

To clarify it, let's say I use User model for user registration, blog model for blogs etc etc.

This would really mean a lot to me if you would shortly explain me how to structure my mongoose models, if all data should be inside one collection or if one user should have separate collections for different features. And if you recommend multiple collections, how do I then link these collections together and make sure that all data is saved for one user etc.

Thanks a lot in advance!

Matthew James Davis
  • 12,134
  • 7
  • 61
  • 90
Aleksandar Grbic
  • 203
  • 1
  • 3
  • 13

2 Answers2

10

I will explain partitioning/dividing into two level.

  1. Of course, you're going to create different collections for different models. Such as Users, Blogs, Messages etc.

  2. Now comes the 2nd part, if we are talking about millions of data. How you partition them for faster data lookup. For example, you have 1M users, which you are going to put in one big collection of 'Users'. But if you look for a user whose first name is 'Imdad' and age is 28, Now your query looks through these 1M items in your single Users collection, which will take a good amount of time.

To solve this problem, users collection can be divided into multiple collections through horizontal partition (Users1 (age between 10-20), Users2 (age between 20-30), Users3 (age between 30-40)). Now based on your query predicate monggoDB is to look up into a different collection/s. This is the idea that MongoDB has applied like other SQL DB. You don't have to explicitly execute your query to the chunk collection but the mongoDB itself take care of that.

Shard key generation

Mongoose shard key

Imdadul Huq Naim
  • 364
  • 5
  • 10
  • 1
    Sharding is not Partition, and MongoDB does not support Partition on collection currently. See: https://stackoverflow.com/a/46450166/2195426 – backslash112 Feb 27 '19 at 17:12
  • Sharding is one technique of partitioning. https://stackoverflow.com/a/20772742/1187316 & yes mongodb has sharding support. https://docs.mongodb.com/manual/sharding/ – Imdadul Huq Naim Mar 22 '20 at 20:31
5

If you are using mongoDB as a backend for a REST interface, the best practice is to create on collection per resource. For example, if you intend on having a /api/users endpoint, you should have users collection and it should contain any and everything you intend to return on that endpoint.

If you are using node to compile server-side templates, structure can be more flexible. In this case, the above still applies (as you will probably eventually want to expose a REST service), but there is more flexibility. In fact, if a many-to-many style relationship is appropriate, it is easier to separate these collections and load them together in the same page.

As an aside, you mention having users and a marketplace. The bigger issue than the separation of data into collections is the use of transactions. Any time you intend on performing a transaction of data, it should be performed within a SQL transaction. There are no notions of transactions in MongoDB. This is by design, as MongoDB is designed to be a fast, scaleable data store. It is not unreasonable to amalgamate SQL and noSQL data, in this case.

Matthew James Davis
  • 12,134
  • 7
  • 61
  • 90
  • We are building business to business social network and we gonna have many features here, not only blog, but marketplace, adverts and many others services. So as far as I understood, you are suggesting that I should have one collection per one resource (collection for posts, collection for users and any other resources). Could you please provide me some guide or reference on how do I link all these collections so my authenticated user can save all this and log out. I still have no idea how to link all these collections to work for one single user. Thanks a lot in advance again. – Aleksandar Grbic Nov 16 '14 at 01:05
  • 3
    You do not link resources in noSQL, this is the SQL way of doing things. noSQL, and MongoDB, is a JSON document store. It is relationship agnostic. You can still create links by adding attributes that reference primary keys, but those links are weak and not enforced. See here: http://stackoverflow.com/questions/4067197/mongodb-and-joins – Matthew James Davis Nov 16 '14 at 01:07
  • 1
    The way I handle this is to store transactional and relational data in SQL (typically PostgreSQL) and JSON documents, where relationships are weak and performance trumps integrity, in noSQL (typically MongoDB) – Matthew James Davis Nov 16 '14 at 01:09
  • The biggest issue that I'm facing now is not understanding on how to make my authenticated user save his stuff and simply log out. I know how would I do it by using one single collection since I would declare all pages there but how do I make my authenticated user fetch all his data from all these collections. For example, I use collection Users for registration and logging in and I could just as well declare my posts there and all data but how do I fetch data from different collections for different users. This is the only issue now I m facing. Thanks a lot :) – Aleksandar Grbic Nov 16 '14 at 01:11
  • 3
    It is not unreasonable to store users in both SQL and noSQL. For example, create a user in SQL, then create the same user in noSQL with _id set to the SQL pk. From there, you can add say a purchase record in SQL, and you could store a list of recent searches in noSQL. – Matthew James Davis Nov 16 '14 at 01:12
  • don't forget to upvote this answer (and these comments) if you've found them helpful :) – Matthew James Davis Nov 16 '14 at 01:12
  • I probably didn't explain my problem well. Basically now, I have 2 collections, one is called posts and another users. Now when I log in with my user, I'm adding posts via my CRUD normally and they are being saved to my collection posts, whatsoever, this collection posts is not connected to my passportJS and generally, I don't know how to make these collections work together so I save my posts for the user I log in with. This is what I'm confused about. – Aleksandar Grbic Nov 16 '14 at 01:15
  • Matthew James Davis - Would you please be kind to answer this last question, still haven't made this working :( ? – Aleksandar Grbic Nov 16 '14 at 23:34