6

In the past, you could use URL.createObjectURL() and pass it a MediaStream. However, this has been removed (see https://www.fxsitecompat.dev/en-CA/docs/2017/url-createobjecturl-stream-has-been-deprecated/).

The replacement functionality was to instead use HTMLMediaElement.srcObject. This does a good job of covering the video case.

However, HTMLImageElement does not inherit from HTMLMediaElement. It does not have srcObject, either.

In my specific case, I am developing a FireFox plugin that utilizes the WebRequest filter stream functionality to do image transformations. With that API I get ArrayBuffer chunks of data. I would like to be able to stream these as I receive them to an Image() that is decoding them on the fly rather than simply accumulating them, turning them into a Blob, and then converting into a URL via URL.createObjectURL(blob).

Is there a way I can accomplish this in a streaming fashion?

(Note 1: I'm ok with a FireFox specific solution if need be.)

(Note 2: I tried setting HTMLVideoElement src to e.g. PNG but it appears that the video element is indeed picky and only supports video formats rather than stills. If I could get HTMLVideoElement.srcObject to load image stills, that might make a MediaStream-based solution possible, too).

Kaiido
  • 123,334
  • 13
  • 219
  • 285
J Trana
  • 2,150
  • 2
  • 20
  • 32
  • Note: the bounty states that I'm looking for a replacement for URL.createObjectURL(blob). This is a typo and it should have stated URL.createObjectURL(MediaStream) as the question hints. I apologize for the confusion. – J Trana Oct 07 '19 at 02:11

1 Answers1

7

URL.createObjectURL( MediaStreamInstance ) has been deprecated, not URL.createObjectURL( BlobInstance ) nor other valid inputs for this method (like MediaSource).

MediaStreams are only able to expose video and audio media data, I couldn't find a quote were it's been said exactly, but reading the specs makes it pretty clear, along with the fact a MediaStreamTrack.kind can only be "audio" or "video".

Just like you can't set <img src="video.mp4">, you never could pass a MediaStream to an <img>, so it is not a problem for you, and URL.createObjectURL will work as you wish:

const img = document.getElementById( 'img' );
fetch( "https://upload.wikimedia.org/wikipedia/commons/4/47/PNG_transparency_demonstration_1.png" )
.then( (resp) => resp.arrayBuffer() )
.then( (buf) => {
  const blob = new Blob( [buf], { type: 'image/png' } );
  img.src = URL.createObjectURL( blob );
} );
<img id="img">

The only way to have a true stream in an <img> is through the MJPEG format.

It has been decided that MediaStreams should not be allowed anymore, because too often authors didn't revoke it when they should have, and a MediaStream that got linked by a blobURI had to be kept alive, meaning that even the hardware source in case of a local stream had to be kept running for the whole life of this blobURI (which can be quite long in some circumstances).

When all major browsers finally did support the MediaElement.srcObject property for MediaStreams, that made little sense to keep this.

Here is a long discussion that occurred on the specs' repo.
Here is Firefox's bug report.
Here is Chrome's one.

(note that the only one thing that got removed is the ability to make an <iframe> or <object> point to such a MediaStream through a blobURI, but there is no real use case to use these instead of the <video> element.)

Kaiido
  • 123,334
  • 13
  • 219
  • 285
  • You are correct that URL.createObjectURL(blob) has not been deprecated - this represents a typo between my original question and the bounty declaration. It should have read URL.createObjectURL(MediaStream) as you have likely surmised already. The intent is: how can I make a Image load via streaming? Was it not possible to create a URL.createObjectURL(MediaStream) and pass it to an Image()? – J Trana Oct 07 '19 at 02:08
  • Just like you can't do ``. The data passed from the MediaStream to the MediaElement is video data, img doesn't provide any video decoding. The only true stream you can have to an `` has always been and still is through mjpeg. But this has nothing to do with MediaStream API. – Kaiido Oct 07 '19 at 02:45
  • Well, I'm not so sure about that. `` supports animated image formats such as GIF and APNG, too. So it seems entirely reasonable to me that all formats - animated or not - might have been (might be?) supported by MediaStream, and that `` only places further restrictions on e.g. the registered decoders. So for example, perhaps `` decodes only JPEG's, MJPEG's, and GIF's but ` – J Trana Oct 07 '19 at 04:48
  • ... that both provides an authoritative source for at least part of your statement on revoking as well as showing clear intent for MediaStream only being used for audio and video elements. If you can incorporate that as a link into your answer (and, if possible, provide the source you had for the blob revoke info) then I will grant the bounty since authoritative info is in the answer. I'm still bummed that the original question about Image and streaming loading seems a bit open though. I was hoping there'd be something there. – J Trana Oct 07 '19 at 04:50
  • Neither APNG nor GIF are "true stream", under the hood they are just a succession of still images all packed in the same container with some metadata to set the frequency. Decoders need the whole file in order to play the animation => the data can't be "streamed". I'll edit my answer with links to different intents to deprecation when I'll find time, and maybe for the ones about MediaStream being only for video media (btw I didn't say MediaStreams are only for MediaElements, other APIs also use it, like WebAudioAPI and WebRTC, what I said is that its data is audio/video only). – Kaiido Oct 07 '19 at 05:11
  • Hehe, well video containers aren't always optimized for the web; sometimes the layout is [optimized for the muxer instead](https://www.matroska.org/technical/order/index.html). But even in that case if the frame is independent, the frame itself can be decoded in a streaming fashion once it is known it can be displayed. I must say that I'm now quite curious to see how Firefox actually implements Image.src under the hood. At any rate, look forward to your updates and thanks for taking the time..! – J Trana Oct 07 '19 at 05:37
  • Thanks for the additions! Here's the bounty. I'm going to leave this open to see if anybody else can come up with some oddball creative answer on the byte streaming action. – J Trana Oct 07 '19 at 22:52