1

I have a ReadableStream received from a WebWorker and I would like to play it with a HTMLVideoElement (Blob is not an option and ArrayBuffer is available but not usable, see below). The format can be anything as the video is picked by user and the best I can do is the general MIME type like video/mp4 or video/webm.

Right now the only solution I can find is this answer but I need to know the codec (passing video/mp4 would fail):

// The codec for SourceBuffer is not known:
const sourceBuffer = mediaSource.addSourceBuffer('audio/webm; codecs="opus"');
// ...

The alternatives is to load the whole thing into an ArrayBuffer instead but with large files (I tested with 4.5GB file) it would hang the browser.

Is there anyway to play a ReadableStream in HTMLVideoElement?

Luke Vo
  • 17,859
  • 21
  • 105
  • 181
  • 2
    How is ArrayBuffer available and why can't it be read in small chunks (_ie:_ Are you using FileReader at any point in the _"Video is picked by user"_ part)? Anyways the solution is that you need to somehow read a starting chunk (like the first 1000 bytes) and simply look for bytes of text like "VP8", "VP9", "avc" "hevc" etc... Note that MP4 can put its metadata at the back end of file, so you need to be able to read from any position. – VC.One Apr 14 '23 at 13:28
  • @VC.One It's provided by Blazor (.NET/WebAssembly). That's why I do nto want to use `ArrayBuffer` because the moment I request an ArrayBuffer everything is read into memory (.NET implementation is my guess). The only other available method is `stream()` that gives me a `ReadableStream` that doesn't load everything into memory. And yes I can read small chunk of data through the Stream though I don't think I can simply "jump" to the end of the file. Anyway if that is the case, I guess I will parse the Stream at .NET/WebAssembly side instead of JS. – Luke Vo Apr 15 '23 at 02:22
  • I don't use Blazor so I'm not sure if that still means FileReader is involved or not? If the user picks a JS "File" object then you can first use FileReader on it to first check the bytes (in chunks if needed) before sending the File object to the WebAssembly part as usual (I assume File is selected --> sent to Workers/Assembly? So add a bytes checking middle step)... Alternatively yes, you can also do it on the .NET side if you can jump to offsets within the user file. I can only help with JS side to check bytes, and that's if you're using File and FileReader. – VC.One Apr 16 '23 at 15:39
  • 1
    @VC.One Hi, no `FileReader` is involved unfortunately. I solved my issue by not processing it on JS side. Reading the source code of Blazor, `ReadableStream` is the only thing that I can get without loading stuff into memory. The `ArrayBuffer` is simply a wrapped Stream (`new Response(stream).arrayBuffer()`) and if I use `.blob()` instead, it's still loaded into memory. – Luke Vo Apr 17 '23 at 08:09

0 Answers0