0

I have a websocket server made with express in nodejs

Websocket documentation states that websocket.send() can pass string or array buffer. Now I want to send a big array of objects (200k lines when formatted) over websocket to clients. What is the best way to send up data such as this.

What I've tried: I've tried sending it directly by stringifying it, Now this works completely fine but delay is very long.

So is there any way to send such a massive array data to clients while keeping speed intact? I think array buffers might help but I couldn't find any suggested examples

Also if this issue is code specific then let me know in comment so that I can share code snippets as well.

Mr.Online
  • 306
  • 4
  • 15
  • What kind of app is your client? – JRichardsz Nov 21 '21 at 05:34
  • @JRichardsz Client is android app made in kotlin – Mr.Online Nov 21 '21 at 05:41
  • Send the data in multiple smaller chunks rather than one really large piece of data. Include a chunk number and total chunks for the client to expect so the client will know when it has received all the chunks. Another option is to send the client a unique web link and have the client then do a large download from your web server using that unique web link as http is well rehearsed at large downloads whereas webSocket, not so much. – jfriend00 Nov 21 '21 at 05:42
  • @jfriend00 I'm in fact very newbie if it comes to binary stuff like those but I think I can go with link idea, That might be the solution as well. Let me post update after I test this one – Mr.Online Nov 21 '21 at 05:46
  • Usually sockets are used to handle real time messaging, sacrificing http features with the goal of to be light and fast. Transfer large amount of data goes against this. Why Do you need sockets to that feature? Why a classic http route is not an option for you? – JRichardsz Nov 21 '21 at 05:48
  • @JRichardsz Infact I'm having crypto related api. Which send tickers info (all coins) to multiple clients every seconds, I thought to make it api endpoint at first but spamming API every second can give a load on server right? – Mr.Online Nov 21 '21 at 05:52
  • #1 Interesting. Maybe just the notification of some change in crypto ticker should use socket. But when client knows (thank to the socket) that there are new/update crypto ticker (which is large) could download it using common http instead socket? – JRichardsz Nov 21 '21 at 05:59
  • @JRichardsz yeah it's indeed a good idea I guess – Mr.Online Nov 21 '21 at 07:20

1 Answers1

1

Usually sockets are used to handle real time messaging, sacrificing common http features with the goal of to be light and fast. Check:

Transfer large amount of data goes against this. Check :

Advice 1

Use socket to detect the real time notification of some change in crypto ticker should use socket.

After tha when client knows (thank to the socket) that there are a new or and updated crypto ticker (which is large), download it using common http instead socket as @jfriend00 also recommends in comments.

Also if data is large, you should use an approach to split the data and send chunk by chunk using common http, not sockets.

Advice 2

As @jfriend00 said and also the major file host services do, implement and algorithm to split the data and send part by part to the client.

If the chunk or part is small, maybe you could use sockets but this is a common feature , so use the known way: common http

JRichardsz
  • 14,356
  • 6
  • 59
  • 94