5

Overview

Is it possible to use VNC (RFB) with WebRTC to implement remote screen control using Node Js? I get remote screen frames from RFB and I want to transform it to MediaStream and then send to the client side. I was trying to search for any solution in the net but found nothing I can use.

Possible solutions I've found

  1. ffmpeg frame encoding (I don't sure I can encode frames to something suitable for MediaStream)
  2. put frames into canvas element and then capture to the MediaStream

Main question

How to encode rfb frames to be suitable for Mediastream and WebRTC

What I've been using until now

I just transform rfb frames to png pictures, send to the client and render it using canvas. Problem - poor fps, quite big latency

Is there any other solutions except WebRTC?

Valery Magadeeva
  • 143
  • 2
  • 6

1 Answers1

4

I think WebRTC is a great solution for this, this Open Source project neko does it already. They aren't using VNC (but instead using GStreamer to capture X11), but that totally possible to change.

Since png is lossless you are wasting a lot of bandwidth on that, if possible I would encode to VPx or H264.

Are you transporting these png via the DataChannel? I would also use RTP if possible. The browser will discard late frames (and other optimizations) to make sure you get the best experience.

Sean DuBois
  • 3,972
  • 1
  • 11
  • 22
  • Thanks for your answer! Yeah, I guess, h.264 is a great solution, but I haven't found any solutions to encode rfb frames to h.264. Is there anything? Also, I transporting pngs via socket connection – Valery Magadeeva Mar 18 '20 at 18:54
  • You can use GStreamer's rfbsrc maybe? That will take care of decoding, then you can encode to whatever you want! – Sean DuBois Mar 18 '20 at 19:15
  • Does using GStreamer to capture X11 also work to control Windows / macOS operating systems, like VNC? – baptx Mar 06 '21 at 18:18