8

I am trying to get a video to load from a stream. I would like to be able to "play" the video even if the entire video has not "downloaded" yet. The base64 data is coming in as a stream of buffer chunks.

const videoTag = document.getElementById('videoTag');
const mimeCodec = 'video/mp4; codecs="avc1.42E01E, mp4a.40.2"';
const mediaSource = new MediaSource();

videoTag.src = URL.createObjectURL(mediaSource);

await new Promise((resolve, reject) => {
    mediaSource.addEventListener('sourceopen', function (_) {
        resolve();
    });
});

const sourceBuffer = mediaSource.addSourceBuffer(mimeCodec);


for await (const chunk of ipfsNode.cat(CID)) {//Chunk is coming in as an array buffer
    sourceBuffer.appendBuffer(chunk);
}

sourceBuffer.addEventListener('updateend', async function (_) {
    mediaSource.endOfStream();
    $('#videoTag')[0].load();
});

I've tried using new MediaSource() and sourceBuffer.appendBuffer(chunk); to push new downloaded chunks to the video, but not only does the video not ever play, the method I am using requires that the entire video downloads before it can play.

How can I get my chunks of base64 data into a video?

EDIT (in response to comments): Regardless of whether or not I use a String or binary steam, I am a little confused on how I get a stream into a video.

now_world
  • 940
  • 7
  • 21
  • 56
  • Why base64? (And, where exactly is that coming from anyway...) – Brad Nov 23 '20 at 02:28
  • It has to be either a base64 encoded string or a dataURL :/ And it is coming from my IPFS server that I own. – now_world Nov 23 '20 at 03:00
  • Why do you believe it has to be base64/data URI? – Brad Nov 23 '20 at 03:04
  • Well I guess it doesn't have to be in a base64/data format, I just need it in either a string or a bufferarray so that I can encrypt it. – now_world Nov 23 '20 at 03:49
  • Encrypt it, how, and for what purpose? – Brad Nov 23 '20 at 03:49
  • For privacy and with [Salsa20 256](https://en.wikipedia.org/wiki/Salsa20) – now_world Nov 23 '20 at 03:55
  • 7
    Ok, so base64 is not what you want, nor strings. Keep everything in binary, otherwise you're adding 33% overhead in storage, memory, CPU, everywhere. – Brad Nov 23 '20 at 04:03
  • For everyone else, I still need to know how to stream data into a video though. Even if the data is in binary and not a string. – now_world Nov 24 '20 at 20:45
  • 2
    If your video file is available on your server as an mp4 then you just need to make sure your server supports range requests and that the mp4 has the 'MOOV' atom, the header, at the start and using the standard HTML5 tag will automatically request the video in 'chunks' and play it for you. If you want a more sophisticated streaming solution then you may want to use a dedicated streaming protocol like HLS or DASH - there are quite a few overviews available online and you can take a look at live examples on YouTube etc - see here: https://stackoverflow.com/a/42365034/334402 – Mick Nov 26 '20 at 10:22
  • _"...I still need to know how to stream data into a video"_ Provide a testable link to such data and can try show you how to display it. – VC.One Dec 02 '20 at 05:33

2 Answers2

3

You have two problems: 1. how to add arbitrary data to a video stream? 2. how to stream?

  1. Given that you use a mp4 container (const mimeCodec = 'video/mp4; codecs="avc1.42E01E, mp4a.40.2"';) you can add arbitrary number of private streams, hence your encrypted stream (that can be base64 or bytes[] as well).

    From the MP4 registration authority the codecs encm (Encrypted/Protected metadata), encs (Encrypted Systems stream), enct (Encrypted Text) seems to be good candidates for your encrypted private stream.

  2. ffmpeg have the ability to mux multiple streams into the final stream; this might be the solution on the backend side. Here there is no need to have the input/private streams finished in order to be able to mux and broadcast into the final stream; the reading should also happen without reaching the end of the stream.

Soleil
  • 6,404
  • 5
  • 41
  • 61
1

Use a data: URL and directly insert that into the video src property.

9pfs
  • 560
  • 5
  • 17