Media Notifications using the Media Session Web API doesn't work with Web Audio API

Question

I'm trying to implement custom notifications into my PWA for the currently playing audio content.

As the title states; I'm using Android v8.1.0 with the Google Chrome App v68.0.x. According to this article: The Media Session API is supported in Chrome 57. I assume the version I'm using should be more than up-to-date to work with these notifications.

First, a code snipped for the audio content I play:

let context = getContext();
await context.resume().catch(console.error);
let source = context.createBufferSource();
source.connect(context.destination);
source.start(0);

The playing of audio content works just fine, no issue here. I tried updating the metadata of the Media Session API after the source.start(..) call, like this:

let updateMediaSession = (payload) => {
  if (!('mediaSession' in navigator)) {
    return;
  }

  navigator.mediaSession.metadata = new MediaMetadata({
    title: payload.title,
    artist: payload.artist,
    album: payload.album,
    artwork: [
      {
        src: '/static/logos/android-chrome-36x36.png',
        sizes: '36x36',
        type: 'image/png',
      },
      {
        src: '/static/logos/android-chrome-48x48.png',
        sizes: '48x48',
        type: 'image/png',
      },
      {
        src: '/static/logos/android-chrome-72x72.png',
        sizes: '72x72',
        type: 'image/png',
      },
      {
        src: '/static/logos/android-chrome-96x96.png',
        sizes: '96x96',
        type: 'image/png',
      },
      {
        src: '/static/logos/android-chrome-144x144.png',
        sizes: '144x144',
        type: 'image/png',
      },
      {
        src: '/static/logos/android-chrome-192x192.png',
        sizes: '192x192',
        type: 'image/png',
      },
      {
        src: '/static/logos/android-chrome-256x256.png',
        sizes: '256x256',
        type: 'image/png',
      },
      {
        src: '/static/logos/android-chrome-384x384.png',
        sizes: '384x384',
        type: 'image/png',
      },
      {
        src: '/static/logos/android-chrome-512x512.png',
        sizes: '512x512',
        type: 'image/png',
      },],
  });

  navigator.mediaSession.setActionHandler('play', () => ...);
  navigator.mediaSession.setActionHandler('pause', () => ...);
  navigator.mediaSession.setActionHandler('seekbackward', () => ...);
  navigator.mediaSession.setActionHandler('seekforward', () => ...);
  navigator.mediaSession.setActionHandler('previoustrack',() => ...);
  navigator.mediaSession.setActionHandler('nexttrack', () => ...);
};

I event update the playing state like so:

  if ('mediaSession' in navigator) {
    navigator.mediaSession.playbackState = 'paused';
  }

and

  if ('mediaSession' in navigator) {
    navigator.mediaSession.playbackState = 'playing';
  }

The Problem The notification doesn't show up on my smartphone.

I've deployed my source code to a proper server with domain. Navigated to the site on my smartphone, cleared all the data, uninstalled any potential PWAs and refreshed the page before conducting my tests.

It seems as if everything gets executed like one would expect but nothing is being displayed. I expected something like this.

Does anyone know why this doesn't work?

Any chance you can link to the site you're hosting it on? Tried it locally and it works fine for me. — JediBurrell, Sep 18 '18 at 12:15
No, sorry, because there is private data involved, however you can read into the code if you want to: [repo](https://github.com/tobiaswuerth/mux-www) \ [file](https://github.com/tobiaswuerth/mux-www/blob/dev/src/ecosystems/vuex/modules/GlobalModule.js) — товіаѕ, Sep 18 '18 at 13:31
I tried to clone your repo, and I got it to run, but I can't seem to get past the login/register page. Are there any internal functions to create an account (locally)? — JediBurrell, Sep 19 '18 at 03:50

JediBurrell · Accepted Answer · 2018-09-19T13:14:52.310

7

You have two issues at play here. The first one is a really simple fix.

updateNotification is preventing updateMediaSession from running, and would cause a double notification otherwise. You can fix this by simply wrapping updateNotification with a condition checking if(!('mediaSession' in navigation)). So if the device supports mediaSession, use mediaSession, otherwise (it's a computer) create a Notification.

Second (and most important), using mediaSession requires an audio (or video) element to play for the notification to show up. It appears you're not using that.

You can either change to using the audio element in the background and just rework your controls to modify that on interaction, or you can work around it by playing a silent audio clip in an invisible audio element and immediately pausing it (the notification will dismiss if it plays through). I would recommend using audio.

If you decide to use the workaround, I found this repository with silent audio clips of varying lengths. The shortest one I could get to work was 5 seconds, any shorter would not show a notification for me.

edited Sep 19 '18 at 13:14

answered Sep 19 '18 at 13:05

JediBurrell

1,069
10
24

so what you are saying is that calling `new Notification(..)` suppresses the `navigator.mediaSession.metadata = new MediaMetadata(..)` call? .. as you can see [here](https://github.com/tobiaswuerth/mux-www/blob/dev/src/ecosystems/vuex/modules/GlobalModule.js), the `updateNotification` and `updateMediaSession` both are executing conditionally, so there should never be the situation where both of them are actually executed. Regarding the required element; you're right, I don't have an `audio` _element_, but I have an `AudioContext`, I cannot use an audio element bc of custom authentication – товіаѕ Sep 19 '18 at 14:24
using the workaround seems a litte messy, I'm hoping I don't have to use this – товіаѕ Sep 19 '18 at 14:25
It's not that it's suppressing it, rather it's throwing an error only on Android that's preventing it to run. Check the console on mobile. – JediBurrell Sep 19 '18 at 14:27
ahh I see. .. I wasn't able to emulate an android device on PC easily, and I have no clue how to open the console on mobile. How do I do that? – товіаѕ Sep 19 '18 at 14:29
Plug your phone into your computer and go to chrome://inspect, authorize adb, and you'll get a list of your tabs where you can click to inspect it. – JediBurrell Sep 19 '18 at 14:31
oh really? this would be awsome! I'll try it as soon as possible, try to fix the issue and then check back to see if it works. Thanks already for the help – товіаѕ Sep 19 '18 at 14:32
So I tested chrome://inspect - this is awsome! .. I fixed the issue and updated my beta site. Everything looks good but no controls (pause, skip, ..) are shown. Any idea why? – товіаѕ Sep 20 '18 at 06:22
If you push it to the site I can check it out. Currently not getting a notification, so I assume you're running it local? – JediBurrell Sep 20 '18 at 06:24
It is already updated. Remember to disable network cache and force service worker reinstall – товіаѕ Sep 20 '18 at 06:27
I see it now. The problem is that that's from `createNotification`, and not the `mediaSession`. `mediaSession.metadata` is actually getting set now, but without an audio element starting, that notification won't show up. – JediBurrell Sep 20 '18 at 06:31
ah, I see. so `Notification` is not only for PC but also for mobile, makes sence I guess. .. even if I update the mediaSession playingstate manually (e.g. [here](https://github.com/tobiaswuerth/mux-www/blob/dev/src/ecosystems/vuex/modules/AudioModule.js#L55))? Isn't there a way with which I do not have to add an 'artificial'/unnecessary audio element? – товіаѕ Sep 20 '18 at 06:42
Unfortunately not. The notification is being made by the media (`audio`, `video`) element and the metadata is just extending that by adding extra control/style to it. – JediBurrell Sep 20 '18 at 06:46
do you have a source for your information? so your workaround suggests to create a real audio element which plays silence .. hm – товіаѕ Sep 20 '18 at 06:58
Let us [continue this discussion in chat](https://chat.stackoverflow.com/rooms/180412/discussion-between-jediburrell-and-tobias-wurth). – JediBurrell Sep 20 '18 at 06:59
In the above answer, `if(!('mediaSession' in navigation))` is wrong. It should be `if(!('mediaSession' in navigator))`. – TOG Jan 14 '23 at 09:01

Media Notifications using the Media Session Web API doesn't work with Web Audio API

1 Answers1

Linked