2

According to the HTML spec section on media elements:

Media elements that are potentially playing while not in a document must not play any video, but should play any audio component. Media elements must not stop playing just because all references to them have been removed; only once a media element is in a state where no further audio could ever be played by that element may the element be garbage collected.

However, I have tested creating video elements in js and playing them in Chrome, Safari, and Firefox and they have all worked without them being appended to the document. Are these browsers not following the spec? Or am I interpreting the spec wrong?

I ask because I want to extract frames from a video by seeking through it and painting the frames to a canvas. Ideally, this would all be done in js without having to append the video to the DOM. However, I do not want to do this if it is against the spec, as it might change in the future and my implementation might break.

frownyface
  • 153
  • 1
  • 8
  • not sure but if you want the code to take screens [see this](https://stackoverflow.com/questions/60992142/snapshot-video-as-a-preview/61012704#61012704) – Lawrence Cherone Jan 06 '22 at 01:08

1 Answers1

2

I agree that this wording is a bit unclear, and the whole section seems a bit outdated anyway, but after further inspection that doesn't seem to cause a bug.

While the term "play" is not precisely defined, this part actually doesn't really matter. Consumers will define their own way of grabbing the video data, for instance the mediacapture-from-element specs do specify that

Whether a media element is actively rendering content (e.g., to a screen or audio device) has no effect on the content of captured streams.

In fact, implementations hook to the VideoTracks before it reaches the <video> element's renderer.

Similarly, for when the <video> is being used as a CanvasImageSource, the specs ask that

When a CanvasImageSource object represents an HTMLVideoElement, then the frame at the current playback position when the method with the argument is invoked must be used as the source image when rendering the image for CanvasRenderingContext2D APIs

So once again, what is rendered by the <video> element doesn't matter, what matters here is its current playback position, which doesn't stop being incremented when the element is disconnected.

So yes, you are safe to use disconnected <video> elements to grab frames from it, most Web Platform Tests that do draw a <video> do expect this behavior (e.g), so it can be considered as "standard".

Kaiido
  • 123,334
  • 13
  • 219
  • 285
  • Thank you so much for the very informative answer. I did want to respond that the spec does [give a description of potentially playing](https://html.spec.whatwg.org/multipage/media.html#potentially-playing), but I'm not sure if that is the "not precisely defined" description you are talking about? And would you please be able to further clarify what you mean by "Consumers will define their own way of grabbing the video data"? The mediacapture-from-element was helpful but I had trouble connecting it to that statement. – frownyface Jan 06 '22 at 17:21
  • And in the CanvasImageSource spec, you say what is rendered by the – frownyface Jan 06 '22 at 17:21
  • 1
    I mean the term "play" from "must not play". This has no precise description. To "play" any video in this context apparently actually means to render the video frames. So there is no video data being generated by the element, and the consumers (like captureStream or the canvas API) have to define how this data is generated. And that's the parts I quote in my answer, how both APIs do generate the video data without relying on the – Kaiido Jan 06 '22 at 23:24