How to predict or undo failure of createMediaElementSource

Question

I'm developing a WebExtension that uses the createMediaElementSource function.

The problem is that this operation can fail, and when it does, it does so without throwing an error. It simply returns an audio node that produces no output, and displays the following warning in the browser console:

The HTMLMediaElement passed to createMediaElementSource has a cross-origin resource, the node will output silence.

Further, the affected <audio>/<video> element will no longer output any sound.

---

The following snippet demonstrates the problem - as soon as the "Create audio node" button is pressed, the audio becomes permanently muted.

function createAudioNode() {
  const audioElement = document.querySelector('audio')
  const audioContext = new AudioContext()
  const audioNode = audioContext.createMediaElementSource(audioElement)
  audioNode.connect(audioContext.destination)
}

<audio controls src="https://upload.wikimedia.org/wikipedia/commons/4/40/Toreador_song_cleaned.ogg">
  Your browser does not support the <code>audio</code> element.
</audio>
<br>
<button onclick="createAudioNode()">Create audio node</button>

This is an unacceptable user experience - if something goes wrong, I want to display an error message, not just silently (literally) break the media playback.

---

So, my question is: How can I prevent this from happening? I can think of two ways to handle this:

• Predict that createMediaElementSource will fail, and not execute it at all.
• Detect that createMediaElementSource has failed, and undo it.

Is either one of these possible? Or is this simply not doable with the current Web Audio API?

Answer 1

I tried to take a detour and attach using a stream of the audio element. Also suggested here. One of my ideas was also to connect it to oscillator to detect if sound is muted.

But it threw an Exception already when trying to capture the stream. But the audio played on so this is my solution. It is my understanding that createMediaElementSource will not work for same reason that createMediaStreamSource which is the same reason that MediaElement.captureStream didn't work.

function createAudioNode() {
  const audioElement = document.querySelector('audio')
  const audioContext = new AudioContext()

  try {
    var stream = audioElement.captureStream()
  } catch (ex) {
    console.log("createMediaElementSource() will not work because " + ex.message)
    return
  }


  const audioNode = audioContext.createMediaElementSource(audioElement)
  audioNode.connect(audioContext.destination)
}

<audio controls src="https://upload.wikimedia.org/wikipedia/commons/4/40/Toreador_song_cleaned.ogg">
  Your browser does not support the <code>audio</code> element.
</audio>
<br>
<button onclick="createAudioNode()">Create audio node</button>

Answer 2

The error message I get in my console when trying to run your example shows that the example fails due to cross-origin resource sharing permissions:

Cross-Origin Resource Sharing (CORS) is an HTTP-header based mechanism that allows a server to indicate any origins (domain, scheme, or port) other than its own from which a browser should permit loading resources. CORS also relies on a mechanism by which browsers make a "preflight" request to the server hosting the cross-origin resource, in order to check that the server will permit the actual request. In that preflight, the browser sends headers that indicate the HTTP method and headers that will be used in the actual request.

The server can permit such resource loading via the Access-Control-Allow-Origin header.

This GitHub issue on the web-audio-api repo looks very relevant: A way to a) detect if MediaElementAudioSourceNode is CORS-restricted & b) revert createMediaElementSource

Within that issue thread, there is a comment saying:

To check if the HTMLAudioElement is a cross-origin resource served without Access-Control-Allow-Origin header listing the origin requested from is to set the crossorigin attribute on the element then observe both loadedmetadata (fired when the header is set) and error (fired when the header is not set) events.

See the full comment for a code snippet that prints those errors to the console.

Also possibly related: Firefox WebAudio createMediaElementSource not working

---

To check if the resource can be fetched, you can send a fetch request for the src of the media element, and use the success and failure callbacks of the Promise.then method.

function createAudioNode() {
  const audioElement = document.querySelector('audio')
  fetch(audioElement.src, { method: "GET", mode: "cors" }).then(
    success => {
      const audioContext = new AudioContext()
      const audioNode = audioContext.createMediaElementSource(audioElement)
      audioNode.connect(audioContext.destination)
    },
    failure => console.log(failure)
  );
}

So if you are intent on not adding a crossorigin attribute to the audio element, you can detect whether createMediaElementSource by checking that the fetch succeeds (ie. the server supports CORS), and that the crossorigin attribute exists.

I don't know whether doing an extra fetch results in consuming network resources on the client side. I presume that caching (if configured) could prevent wasteful network resource consumption.

Answer 3

According to audio element https://developer.mozilla.org/en-US/docs/Web/HTML/Element/audio#attr-crossorigin , not setting crossorigin attribute could limit the usage of audio cors.

This enumerated attribute indicates whether to use CORS to fetch the related audio file. CORS-enabled resources can be reused in the element without being tainted. The allowed values are:

anonymous Sends a cross-origin request without a credential. In other words, it sends the Origin: HTTP header without a cookie, X.509 certificate, or performing HTTP Basic authentication. If the server does not give credentials to the origin site (by not setting the Access-Control-Allow-Origin: HTTP header), the image will be tainted, and its usage restricted.

use-credentials Sends a cross-origin request with a credential. In other words, it sends the Origin: HTTP header with a cookie, a certificate, or performing HTTP Basic authentication. If the server does not give credentials to the origin site (through Access-Control-Allow-Credentials: HTTP header), the image will be tainted and its usage restricted.

When not present, the resource is fetched without a CORS request (i.e. without sending the Origin: HTTP header), preventing its non-tainted used in elements. If invalid, it is handled as if the enumerated keyword anonymous was used. See CORS settings attributes for additional information.

So try to add crossorigin="anonymous" or as you wish to tell explicitly to use the cors.

function createAudioNode() {
  const audioElement = document.querySelector('audio')
  const audioContext = new AudioContext()
  const audioNode = audioContext.createMediaElementSource(audioElement)
  audioNode.connect(audioContext.destination)
}

<audio crossorigin="anonymous" controls src="https://upload.wikimedia.org/wikipedia/commons/4/40/Toreador_song_cleaned.ogg">
  Your browser does not support the <code>audio</code> element.
</audio>
<br>
<button onclick="createAudioNode()">Create audio node</button>