2

There was an interesting discussion over here on StackOverflow, and in some ways this question is a followup. I've also asked a similar question in the past, but I feel this is more generally a question about object URLs.

There have been a number of times where I would like to implement a streaming version of a ".src" for image or video elements in JS, perhaps from a stream of bytes. Unfortunately, I only see two main options that are more controllable by JS:

  1. Create a Blob and then use URL.createObjectURL(). Unfortunately, this seems to be static - but perhaps there is a way to mutate the contents?
  2. Create a MediaSource. However, this only works for video and is much pickier than just using a video element, which is really the level of support I need.

Any thoughts on how I can create some type of streaming object URL? And/or if not, does anybody know why JS hasn't implemented this type of streaming long, long ago?

J Trana
  • 2,150
  • 2
  • 20
  • 32
  • For those interested in the use case, I am streaming in bytes over the network in a WebExtension plugin for Firefox. My current working pattern is to create an array of ArrayBuffers and continuously swap out Blobs constructed from the current set of ArrayBuffers and set it as the .src. Gross! – J Trana Feb 01 '20 at 17:32
  • It got a bit buried in the comments to @Brad's answer, so I'll comment again here. There is an [interesting discussion over here at the WHATWG Github page around this](https://github.com/whatwg/streams/issues/480#issuecomment-271110536) that is worth reading! – J Trana Feb 01 '20 at 22:18

1 Answers1

4

There have been a number of times where I would like to implement a streaming version of a ".src" for image or video elements in JS, perhaps from a stream of bytes.

Use a Service Worker to respond with a Response with a ReadableStream as the body.

but I feel this is more generally a question about object URLs.

Object URLs really only represent immutable Blobs. The MediaStream object URL is a special case, not really applicable here, and a deprecated API as srcObject exists for media elements these days.

Create a Blob and then use URL.createObjectURL(). Unfortunately, this seems to be static - but perhaps there is a way to mutate the contents?

No, Blobs are immutable.

Create a MediaSource. However, this only works for video...

... or audio.

Brad
  • 159,648
  • 54
  • 349
  • 530
  • Hum, I fail to understand the ServiceWorker concept here. Could you explain a bit more how this would work? As I already said in [the previous Q/A](https://stackoverflow.com/a/58254537/3702797), the only format that would allow an HTMLImageElement to fetch a resource as a stream is the MJPEG format. From here, your service worker idea would just make it stall without ever being able to completely fetch the resource, it may be able to display the first image, but would never fire a load event, and would not display any subsequent one. – Kaiido Feb 01 '20 at 05:27
  • @Kaiido The question implies that they want to either generate or retrieve a resource in a streaming manner. Seems like a perfectly reasonable request to me, and it *doesn't* have to be one of streaming media. Perhaps they want to ensure a JPEG can render progressively. Or, more generically get around the limitations of MediaSource Extensions. – Brad Feb 01 '20 at 06:48
  • @Brad Yes, you have hit on my use case. Specifically, I am working in the context of WebExtensions for a plugin, and I would like to stream in video (and possibly images) progressively. I am receiving the data in a streaming fashion using Firefox's StreamFilter. – J Trana Feb 01 '20 at 17:22
  • And yes, I'm quite excited about your answer! I'd like to see if that could work. I didn't think I could essentially host a server in the browser but that approach looks quite promising. – J Trana Feb 01 '20 at 17:23
  • For my specific case, I have [a slight problem - I can't use ServiceWorkers](https://bugzilla.mozilla.org/show_bug.cgi?id=1344561). But now that I know that bit about Response maybe I can find some other way to hack it... – J Trana Feb 01 '20 at 18:59
  • @JTrana Relay the data via the postMessage API from your extension to a service worker on the page? – Brad Feb 01 '20 at 21:24
  • Well, I suppose I could tunnel it through but I think the inefficiency there would start to outweigh the benefit... However, I have seen reference to your approach elsewhere so I believe others have gotten it working well. It answers the question, so I will award it to you. [Here is an interesting link and commentary](https://github.com/whatwg/streams/issues/480#issuecomment-271110536) with regards to this approach for any others who are curious! – J Trana Feb 01 '20 at 22:15