12

Overview

I would like to use a custom video source to live stream video via WebRTC Android implementation. If I understand correctly, existing implementation only supports front and back facing cameras on Android phones. The following classes are relevant in this scenario:

Currently for using front facing camera on Android phone I'm doing the following steps:

CameraEnumerator enumerator = new Camera1Enumerator(false);
VideoCapturer videoCapturer = enumerator.createCapturer(deviceName, null);
VideoSource videoSource = peerConnectionFactory.createVideoSource(false);
videoCapturer.initialize(surfaceTextureHelper, this.getApplicationContext(), videoSource.getCapturerObserver());
VideoTrack localVideoTrack = peerConnectionFactory.createVideoTrack(VideoTrackID, videoSource);

My scenario

I've a callback handler that receives video buffer in byte array from custom video source:

public void onReceive(byte[] videoBuffer, int size) {}

How would I be able to send this byte array buffer? I'm not sure about the solution, but I think I would have to implement custom VideoCapturer?

Existing questions

This question might be relevant, though I'm not using libjingle library, only native WebRTC Android package.

Similar questions/articles:

Jernej Jerin
  • 3,179
  • 9
  • 37
  • 53
  • You want to stream a video file to WebRTC (like fake person in chat video? just for the example) or broadcast an exiting broadcaster ? do you need it as android app ? if I will provide a way to hook an exiting app ? is android tag related ? because you provided source code from chromium – whoopdedoo Apr 20 '20 at 17:54
  • I want to stream an existing video source, for which I receive data via `onReceive` method. I will need to handle it inside Android app, as the device is connected to the phone via USB cable. Source code is hosted on chromium, but it's for Android app. It seems that the whole WebRTC project is hosted there. – Jernej Jerin Apr 20 '20 at 18:03
  • Please explain, you want to stream an existing video source to where ? you receive bytes from a URL for example, your android phone decrypt them and so you will see them as frames, where do you want to send them ? – whoopdedoo Apr 20 '20 at 18:06
  • @JimWest it's a P2P connection using WebRTC. It's not an issue of streaming to the client on the other side, it's an issue with WebRTC Java implementation apparently not supporting external video source for processing apart from existing front/back cameras. – Jernej Jerin Apr 20 '20 at 19:54
  • you want to stream a local video instead of camera to a running video chat ? it's not clear what you want to achieve – whoopdedoo Apr 20 '20 at 21:42
  • @whoopdedoo , Yes I want to stream a local video instead of a camera to a running video chat. correct. – kiran boghra Jun 23 '23 at 05:36

1 Answers1

16

There are two possible solutions to this problem:

  1. Implement custom VideoCapturer and create VideoFrame using byte[] stream data in onReceive handler. There actually exists a very good example of FileVideoCapturer, which implements VideoCapturer.
  2. Simply construct VideoFrame from NV21Buffer, which is created from our byte array stream data. Then we only need to use our previously created VideoSource to capture this frame. Example:
public void onReceive(byte[] videoBuffer, int size, int width, int height) {
    long timestampNS = TimeUnit.MILLISECONDS.toNanos(SystemClock.elapsedRealtime());
    NV21Buffer buffer = new NV21Buffer(videoBuffer, width, height, null);

    VideoFrame videoFrame = new VideoFrame(buffer, 0, timestampNS);
    videoSource.getCapturerObserver().onFrameCaptured(videoFrame);

    videoFrame.release();
}
Jernej Jerin
  • 3,179
  • 9
  • 37
  • 53
  • A bit late to the party, but can you elaborate? The onReceive(byte[],int,int,int) is not the onReceive from DJI from my understanding. So which event is this? – user15227672 Oct 07 '22 at 16:25
  • I mean the onReceive event from DJI does not provide width and height, so I have a hard time following where you obtained those from all of the sudden. – user15227672 Oct 08 '22 at 08:53