1

for learning purposes, i am trying to reconstruct some TikTok functionalities within my MERN App.

Some requirements i would like to implement are:

  1. A many-to-many relationship between posts and hashtags: → A user can create a post and add multiple hashtags → A user can search for a specific hashtag and get posts sorted by timestamp or "likesCount"

  2. Follow/-Unfollow System

  3. Like/-Unlike System

The Design Pattern should avoid following limitations:

  • 16 MB Document Size Limitation
  • Slow Performance when paginating due to "limit" and "skip" restrictions
  • Anything else?

I`ve read about the "Bucket Pattern" (https://www.mongodb.com/blog/post/paging-with-the-bucket-pattern--part-1) and the "Outlier Pattern" (https://www.mongodb.com/blog/post/building-with-patterns-the-outlier-pattern).

When i take a look at TikTok's query responses, it looks like they have implemented the "Outlier Pattern" for retrieving their content.

Whether it's in the feed, where the users is only displayed personalised data or in the hashtag "page", where the data is sorted by the "likesCount"

Query Response when fetching the Feed

Unfortunately, I can't find anything on the internet about the "Outlier Pattern", which makes it difficult for me as a beginner to think my way into this pattern and to implement it correctly with MongoDB.

Like it's described in this article above, i obviously need a document with a structure like this:

{
   "_id": ObjectID("507f191e810c19729de860ea"),
   "hashtagName": "football",
   "hashtag_posts": ["post01", "post02", "post03", ..., "post50"],
   "hasMore": true
}

{
   "_id": ObjectID("507f191e810c19729de860eb"),
   "hashtagName": "football",
   "hashtag_posts": ["post51", "post51", "post52", ..., "post100"],
   "hasMore": false
}

My questions are:

  • How do i sort my list of posts based on attributes like "likesCount" if only a certain amount of data gets saved into a documents array? What about the second document with the other 50 posts?
  • How do i get the count attribute of a hashtags posts?

My Mongoose Schema for a Post looks like this:

const postSchema = new Schema(
{
    desc: {
        type: String,
        required: true
    },
    hashtags: [
        {
            type: [Schema.Types.ObjectId],
            ref: 'Hashtag'
        }
    ],
    stats: {
        likesCount:{
            type: Number,
            required: true,
            default: 0
        },
    }
});
Toni
  • 21
  • 3
  • 1
    I wouldn't recommend using a NoSQL database for creating a social network. It's just not a good practice and it will need a lot of manual handling by the server, not to mention the code that you need to write. You can look into Graph and SQL databases instead such as Neo4j(graph database) and CockroachDB(SQL database) – michaelgrigoryan25 Sep 10 '21 at 15:32
  • what kind of "manual handling by the server" ? I mean there are a lot of companies doing the same with NoSQL databases. I`m just wondering how did they solve this problem? – Toni Sep 10 '21 at 15:37
  • What do I mean by manual handling by the server is that you'll have to handle the joins manually. Like, you'll need to query every piece of data that you want separately and keep it inside of a variable and then append it to your object. There's no such thing as joins in a NoSQL database. – michaelgrigoryan25 Sep 10 '21 at 15:39
  • 1
    Also, I don't think that there are any social network websites that use a NoSQL database, and even if there are they might use something like [CQL(Cassandra Query Language)](https://docs.datastax.com/en/cql-oss/3.1/cql/cql_intro_c.html) which is something completely different – michaelgrigoryan25 Sep 10 '21 at 15:46
  • Or if you really want to go with NoSQL, you may check out this question. – michaelgrigoryan25 Sep 10 '21 at 15:47
  • Isn't there this concept of population in mongodb that is basically doing almost the same thing as a join in relational databases? Maybe not in one query but in two queries. – Toni Sep 10 '21 at 15:49
  • You may want to check out [this thread](https://stackoverflow.com/questions/35813854/how-to-join-multiple-collections-with-lookup-in-mongodb) – michaelgrigoryan25 Sep 10 '21 at 15:50
  • Check out [https://highlyscalable.wordpress.com/2012/03/01/nosql-data-modeling-techniques/](this post) for a better idea of how to model things using NoSQL db's like mongo. It is doable, and arguments can be made for and against it, depending on the complexity of the networking app. That post should help guide your modeling if you do choose NoSQL. – Bogie Sep 10 '21 at 17:45
  • @michael.grigoryan would you stick with a graph database or a relational database ? – Toni Sep 10 '21 at 20:43
  • That document with `hashTagName` and `hashTagPosts` looks a lot like an index. You could probably just store the hashTagNames directly in the post document, index that field, and query on it directly. – Joe Sep 11 '21 at 04:01
  • 1
    @Toni I'd recommend you to play around with them both. Graph database seems a much better way to do things in the case of social media. SQL databases would work fine too but I think that Graph databases are more suited for things like this. – michaelgrigoryan25 Sep 11 '21 at 04:24

0 Answers0