0

I am trying to understand how DRM systems work so my journey begins by trying to play a cenc encrypted mp4 video using the Clear Key DRM system without using any library like dash.js or Shaka Player.

The first problem I encountered is that I do not always receive the "encrypted" event. I receive the "encrypted" only on Safari, but not on Google Chrome nor on Firefox.

Interestingly I do receive the "waitingforkey" on Google Chrome and Safari, but not on Firefox.

This fact confuses me the most, because if Google Chrome knows it needs a key, I assume it has to know that the media is encrypted, so why does it not fire the "encrypted" event?

Below you can find the code I use. I am using some convenience functions. I hope it is clear what they do. If not you see their definitions here. Also my example is online here for you to test and debug right in the browser.

async function playClearkeyVideoFromUrls(videoElement, initUrl, urls) {
    // for debugging purposes
    videoElement.addEventListener(`waitingforkey`, () => console.log(`Event: waitingforkey`))
    videoElement.addEventListener(`encrypted`, () => console.log(`Event: encrypted`))
    videoElement.addEventListener(`error`, function () {
        console.log(`Event: HTMLMediaElement.onerror`)
        console.log(this.error)
    })

    // we create a MediaSource
    const mediaSource = new MediaSource()

    // attach the MediaSource to the Video tag, only then it will fire the "sourceopen" event
    videoElement.src = URL.createObjectURL(mediaSource)

    // add a SourceBuffer to the MediaSource, we need to specify the MIME type of the video we want to play
    const sourceBuffer = await mediaSource.asyncAddSourceBuffer(`video/mp4;codecs="avc1.64001f"`)

    // for debugging purposes
    sourceBuffer.addEventListener(`error`, e => {
        console.log(`Event: SourceBuffer.onerror`);
        console.log(e)
    })

    // append the first (init) segment
    console.log(`Appending the first (init) segment`)
    await sourceBuffer.asyncAppendBuffer(await fetchArrayBuffer(initUrl), videoElement)

    // here I expect the "encrypted" AND "waitingforkey" event to fire

    // now append the rest of the segments
    for (let i = 0; i < urls.length; i++) {
        const url = urls[i]
        console.log(`Appending a segment ...`)
        if (!await sourceBuffer.asyncAppendBuffer(await fetchArrayBuffer(url), videoElement)) {
            console.log(`Canceling playback as an error has occurred.`)
            console.log(videoElement.error)
            break
        }
    }
}

The cenc encrpyted mp4 files I have are from the dash.js examples page, so I assume that this is not the root of my problem.

To sum up my main question is: Why is the "encrypted" event not fired or is my assumption wrong that it should be fired?

I also thought that my fancy util functions could be the cause of the problem. Sadly this is not the case. You can check out my version without the utils file here. It behaves just like the other version.

let initUrl
let urls
let segmentIndex = 0


Number.prototype.toStringPadded = function(size) {
    let thisString = this.toString();
    while (thisString.length < size) thisString = "0" + thisString;
    return thisString;
}

async function fetchArrayBuffer(url) {
    return await (await (await fetch(url)).blob()).arrayBuffer()
}

async function updateend() {
    console.log(`Event: updateend`)

    this.appendBuffer(await fetchArrayBuffer(urls[segmentIndex]))
    segmentIndex++
    if (segmentIndex === urls.length) {
        this.removeEventListener(`updateend`, updateend)
    }
    console.log(`Appended segment with id ${segmentIndex}.`)
}

async function sourceopen() {
    console.log(`Event: sourceopen`)

    // add a SourceBuffer to the MediaSource, we need to specify the MIME type of the video we want to play
    const sourceBuffer = this.addSourceBuffer(`video/mp4;codecs="avc1.64001f"`)

    // for debugging purposes
    sourceBuffer.addEventListener(`error`, e => {
        console.log(`Event: SourceBuffer.onerror`);
        console.log(e)
    })

    sourceBuffer.addEventListener(`updateend`, updateend)
    sourceBuffer.appendBuffer(await fetchArrayBuffer(initUrl))
}

async function playClearkeyVideoFromUrls(videoElement) {
    // for debugging purposes
    videoElement.addEventListener(`waitingforkey`, (event) => {
        console.log(`Event: waitingforkey`)
        console.log(event)
    })
    videoElement.addEventListener(`encrypted`, (mediaEncryptedEvent) => {
        console.log(`Event: encrypted`)
        console.log(mediaEncryptedEvent)
    })
    videoElement.addEventListener(`error`, function () {
        console.log(`Event: HTMLMediaElement.onerror`)
        console.log(this.error)
    })

    // we create a MediaSource
    const mediaSource = new MediaSource()
    mediaSource.addEventListener(`sourceopen`, sourceopen)

    // attach the MediaSource to the Video tag, only then it will fire the "sourceopen" event
    videoElement.src = URL.createObjectURL(mediaSource)
}

async function testPlayClearkeyVideoFromUrls() {
    // video urls are from here https://reference.dashif.org/dash.js/nightly/samples/drm/clearkey.html
    // and here https://media.axprod.net/TestVectors/v7-MultiDRM-SingleKey/Manifest_ClearKey.mpd
    const streamId = 1
    initUrl = `https://media.axprod.net/TestVectors/v7-MultiDRM-SingleKey/${streamId}/init.mp4`
    const videoUrlPrefix = `https://media.axprod.net/TestVectors/v7-MultiDRM-SingleKey/${streamId}/`
    const videoUrlSuffix = `.m4s`
    const numberOfSegments = 4

    // first we generate our urls we will download
    urls = []
    for (let i = 0; i < numberOfSegments; i++) {
        const url = `${videoUrlPrefix}${(i + 1).toStringPadded(4)}${videoUrlSuffix}`
        urls.push(url)
    }
    const videoElement = document.querySelector(`video`)

    await playClearkeyVideoFromUrls(videoElement)
}

testPlayClearkeyVideoFromUrls()
Blubberlase
  • 29
  • 1
  • 8

2 Answers2

1

This fact confuses me the most, because if Google Chrome knows it needs a key, I assume it has to know that the media is encrypted, so why does it not fire the "encrypted" event?

I believe that's because your initialization segment does not contain a pssh atom. Chrome seems to not fire an encrypted event when PSSH is not embedded in the media file.

You can use https://gpac.github.io/mp4box.js/test/filereader.html to view the init segment's MP4 boxes and atoms.

In your case the PSSH data is not included in the media file, but in the manifest itself – https://media.axprod.net/TestVectors/v7-MultiDRM-SingleKey/Manifest.mpd – you could extract the PSSH info from it and not rely on the encrypted event firing at all, since you would already have the initialization data you need.

Alternatively you would need to package your media in a way that generates the PSSH atom. This is how I encrypt a single fragmented MP4 (1 video and 1 audio track) using mp4encrypt to use with a cenc encryption and a clearkey "DRM":

mp4encrypt --method MPEG-CENC --key 1:eecdb2b549f02a7c97ce50c17f494ca0:random --property 1:KID:c77fee35e51fd615a7b91afcb1091c5e --key 2:9abb7ab6cc4ad3b86c2193dadb1e786c:random --property 2:KID:045f7ecc35848ed7b3c012ea7614422f --global-option mpeg-cenc.eme-pssh:true source.mp4 target-encrypted.mp4

(You are, however, using DASH sources with Widevine, so this doesn't really apply to your case. I'm including it for inspiration only and maybe for other people that run into the same issues using clearkey and a single file playback.)

Dero
  • 143
  • 6
  • the pssh atom included in the manifest bit was really helpful. I was facing a similar problem on my end where the `encrypted` event just won't trigger, realised that I've got a cenc:pssh in the manifest and shaka player handles it from there. – Subhan Jun 21 '23 at 05:52
0

I think the issue you are seeing is that some of these events are 'Experimental' status and so support may not be consistent - e.g.:

From experience the browser compatibility changes quite frequently so you may have to experiment as you have done.

To add a bit more complexity there are actually multiple places that a stream or a container may indicate that a track or even a section of a track is encrypted.

Different players in the past, in particular, have given different results depending on where they looked.

For example, one common player used to only look at the manifest for encryption information and if it saw nothing there it would assume it was unencrypted even if the media stream itself had 'atoms' in the mp4 indicating that it was encrypted - this caused the playback to fail.

More specifically looking at the events fired when playing back encrypted media in a browser that supports Encrypted Media Extensions. From the standard, the high level flow is as in the following diagram:

enter image description here

It can be seen that reporting the encrypted event to the application is optional. The 'waiting for key' event is not shown unfortunately, but it is included in the detail of the EME spec.

As you saw, and from a quick check I did here also based on DASH.js samples, the encrypted event is not fired on Chrome but is on Safari and the waiting for key event is fired on both. On chrome you can look in much more detail at the events and messaging, if you want, with a Chrome extension to view EME messaging - again this will show the encrypted event does not appear to be sent to the app.

Unfortunately, browser implementaions are different in some of these details and if you look through the open source players encryption handling code you will see this reflected - for example for videojs:

initializeMediaKeys()

player.eme.initializeMediaKeys() sets up MediaKeys immediately on demand.

This is useful for setting up the video element for DRM before loading any content. Otherwise, the video element is set up for DRM on encrypted events. This is not supported in Safari.

(https://github.com/videojs/videojs-contrib-eme#initializemediakeys)

The reason the player can do this before it sees one of the several indications that may be present in the media stream flagging that the content is encrypted, is because it can also read information in the manifest file, which it reads before loading the media streams, which indicates the encryption scheme(s) that are being used for the media streams.

It's worth noting also the CDMs, the element that does the actual decryption and (optionally) the output to the display are proprietary per browser or DRM usually also.

Going back to your original goal of understanding how EME and encryption works, I think your approach is good. You will probably see other browser and CDM differences also. The solutions do evolve quite quickly sometimes, especially in response to a particular attack or vulnerability, so you do need to be aware of this. Again, the open source player issue and discussion lists are a great resource for understanding the latests changes, as well as the history.

Mick
  • 24,231
  • 1
  • 54
  • 120
  • I doubt that Chrome or Firefox do not support this event because even though it is marked as "experimental" it is not very new and/or special. See [this](https://www.html5rocks.com/en/tutorials/eme/basics/) article which is now almost 8 years old talking about the event. Furthermore all browser need to be aware that the file is encrypted as they fire the _waitingforkey_ event. I suspect it is more likely I am doing a mistake and the browsers react differently to it. But if you have any other hint that Chrome and Firefox do not support this event please let me know. – Blubberlase Dec 03 '21 at 02:22
  • @Blubberlase One quick note - your 'sourceBuffer.addEventListener' line is causing an error when I look at your example. Did you mean to use an 'await' here also? – Mick Dec 03 '21 at 13:21
  • Good observation. I fixed the small bug (see diff [here](https://pastebin.com/5bmdKb9s)). It is also fixed now in my my online example. Sadly this does not seem to be the cause of the odd behaviour I described in my question. – Blubberlase Dec 03 '21 at 19:35
  • The quote from videojs confuses me a little. It says that the encrypted event is not supported in Safari although my experiments clearly show that it does. (Maybe the quote is outdated?) – Blubberlase Dec 20 '21 at 00:59
  • The picture nicely shows that the encrypted event is optional. Partially answering my question. But the picture it is non normative. I have been looking around the spec but was unable to find anything normative that shows that the encrypted and/or the waitingforkey event is optinal. If you could find the sections of the spec which clearly show how the encrypted event is optional that would be great. I assume that also the waitingforkey element is optinal, but I was unable to find a hint in the spec too. This would then also describe the behaviour I see on Firefox and fully answer my question. – Blubberlase Dec 20 '21 at 01:08