socket.io disconnects due to the size of data

Question

In the code below:

The client is supposed to start emitting images as long Base64 Strings to the server, one after another at close intervals.

As soon as the first emit with a Base64 String takes place, the socket is disconnected. I tried with hardcoded 2-character strings and this doesn't happen.

I believe it's either the size of the data that is the problem or the fact that socket hasn't finished emitting the first Base64 string when the 2nd and 3rd etc.. emit comes.

const io = new Server(httpServer, {
  cors: {
    credentials: true,
    origin: 'http://localhost:4200',
    methods: ["GET", "POST"]
  }
})

io.on('connection', function(socket) {
  console.log('connected')

  socket.on('newDataFromClient', async (newDataFromTheClient) => {
    // will work only after I refresh the page after a login
   })

  socket.on('disconnect', (reason) => {
    // the reason is "transport error"
    console.log('disconnected due to = ', reason)
  })

})

httpServer.listen(4200)

This is the client-side code:

let socket

// this is only executed once, on page load
function setup() {
  fetch('/setup')
    .then((response) => response.json())
    .then((setup) => {
      doSomething(setup)
      
      socket = io('http://localhost:4200', {
        withCredentials: true,
        transports: ['websocket']
      })
    })
}

// this is executed repeatedly at close intervals 
async function sendToServer(imageAsBase64) {
 socket.emit('newDataFromClient', imageAsBase64)
}

What am I doing wrong?

You're asking why the client does something, but you don't show or describe anything about the client. That's probably where the issue is. — jfriend00, Nov 01 '21 at 20:34
Here's a problem. You create `httpServer`. an http server object. You then hook up both socket.io and Express to that http server. Then, you start that server with `httpServer.listen(SOCKET_PORT)`. That's all fine. Then, you call `app.listen(EXPRESS_PORT)` which starts another server. That's likely not right. You have two separate servers which both have your Express code hooked up to them. And, one of those servers has socket.io on it. Typically, you would have ONE http server that both Express and socket.io use. — jfriend00, Nov 01 '21 at 20:38
I've updated the OP with the client side code and the values of the PORTS — Kawd, Nov 01 '21 at 20:42
I use 2 different ports because when I used the same port for both it didn't work. My httpServer/app/socket setup is copied/pasted from a tutorial, I don't know if it's right, but it seems to work.. sort of.. — Kawd, Nov 01 '21 at 20:43
Why is the client setting options: `upgrade: false` and `transports: ['websocket']`? An upgrade is required to get to `webSocket` so I don't even know what socket.io will do with those conflicting options set. — jfriend00, Nov 01 '21 at 20:46
Two separate ports and thus two separate servers is NOT required at all. You can use ONE server for both Express and socket.io. Socket.io and the underlying webSocket transport is specifically designed for that and most people use it that way. — jfriend00, Nov 01 '21 at 20:47
@jfriend00 I removed `upgrade: false` and it didn't solve my problem. I will now try the one-server/port setup. — Kawd, Nov 01 '21 at 20:49
There's still not enough information shown here for us to have any idea why your client disconnects. It must have to do with something else the client is doing or the double http servers. — jfriend00, Nov 01 '21 at 20:49
My server/socket setup at the top of the file actually comes from socket.io: https://socket.io/get-started/chat#integrating-socketio so it's correct wrong. It's the two `.listen(..)` at the bottom that I have to maybe fix. — Kawd, Nov 01 '21 at 20:52
Yes, and notice that there's NO `app.listen()` in that documentation example code. It is not starting a 2nd server because in that example both socket.io and express are using the same http server. You should run everything on one server and use just one port. — jfriend00, Nov 01 '21 at 20:53
Sorry but there's nothing I'm hiding from the client side code, it's pretty basic. `setup` gets called on page load and `sendToServer()` starts getting called at various intervals after the page loads. In my case, it never has chance to get called anyway, as the client disconnects, but this is irrelevant. — Kawd, Nov 01 '21 at 20:55
So at the bottom, do I keep `app.listen()` or `httpServer.listen()` ? — Kawd, Nov 01 '21 at 20:56
I've update the OP with my latest setup with just one `httpServer.listen(4200)` and still nothing. I've also added an important (I believe) `EDIT`. — Kawd, Nov 01 '21 at 20:59
You get rid of the app.listen and put everything on the port you want to use on the browser. — jfriend00, Nov 01 '21 at 21:22
Thanks. I realised that it's the first `emit` coming from the client that triggers the disconnect. The data it sends is not corrupted. I know that. I've updated the top of the OP once again. I will look more into this. — Kawd, Nov 01 '21 at 21:27
I checked again. I believe it's the size of the data being emitted that breaks the connection. It's supposed to be a an image as Base64 string. When I hardcode a small 2 character string, the connection doesn't break. — Kawd, Nov 01 '21 at 21:32

score 4 · Accepted Answer · answered Nov 01 '21 at 21:56

4

The problem was socket.io's maxHttpBufferSize limit, set to 1Mb by default.

Each Base64 string I'm sending is around 2.5Mb.

I had to update my server-side code to

const io = new Server(httpServer, {
  cors: {
    origin: 'http://localhost:4200',
    methods: ["GET", "POST"]
  },
  maxHttpBufferSize: 4e6 // 4Mb
})

And now everything works. I found the answer here.

answered Nov 01 '21 at 21:56

Kawd

4,122
10
37
68

1

Here's a thought for next time: [Logging and Debugging socket.io](https://socket.io/docs/v3/logging-and-debugging/). This would presumably have logged something helpful. – jfriend00 Nov 02 '21 at 00:18

socket.io disconnects due to the size of data

1 Answers1

Linked