4

I've been using MongoDB changeStreams with Node.js and socket.io;

I'm using a sort of pub-sub architecture, where for each subscription he's supposed to replace the previous changeStream with the new subscription.

The logic goes something like this:

  1. Client requests new subscription(it's related to a collection changeStream)
  2. If the client has a changeStream, the server closes such changeStream and replaces with the updated changeStream.

pseudoCode for subscription

// If it has a changeStream
if (socket.dbWatcher["users"]) {
  // close changeStream
  socket.dbWatcher["users"].removeAllListeners("change");
  socket.dbWatcher["users"].close();
}

// Replace with new changeStream and pipeline
socket.dbWatcher["users"] = db.collection("users").watch(pipeline, {resumeAfter})
.on("change", (change) => {
  socket.emit("users", change)
})

Things were getting slow so I checked the number of connections and it was way over the number of collections (16 collections vs 29 connections).

Once a socket disconnects, I close all changeStreams but the connctions are still there(database). I increased the poolSize, but having 29 connections is not normal.

This is the mongodb command that I use to check connections: db.serverStatus().connections

Lucas Gomes
  • 316
  • 5
  • 16
  • What is the topology of the MongoDB deployment ? Remember the value of `db.serverStatus().connections` include all incoming connections including any `mongo` shell connections and/or connections from other servers, i.e. replica set or `mongos` instances. – Wan B. Dec 16 '18 at 23:12
  • I use Replica Set with the default configuration. Just execute the command `mongod --maxConns 64000 --replSet rs0` – Lucas Gomes Dec 17 '18 at 09:57

1 Answers1

0

You can see documentation for closing a changestream here: https://docs.mongodb.com/manual/changeStreams/#resume-a-change-stream

As you can see in the node driver API, there is a close() method on the changestream.

However, I'm not so sure that rapidly closing/reopening connections with MongoDB changestreams will be performant. It may be better just to watch on the entire user collection.

The changestream feature as designed isn't really intended for the traditional pubsub pattern. See this question for more details: Severe performance drop with MongoDB Change Streams

hpx7
  • 41
  • 1
  • 3