Understanding How to Store Web Push Endpoints

Question

I'm trying to get started implementing Web Push in one of my apps. In the examples I have found, the client's endpoint URL is generally stored in memory with a comment saying something like:

In production you would store this in your database...

Since only registered users of my app can/will get push notifications, my plan was to store the endpoint URL in the user's meta data in my database. So far, so good.

The problem comes when I want to allow the same user to receive notifications on multiple devices. In theory, I will just add a new endpoint to the database for each device the user subscribes with. However, in testing I have noticed that endpoints change with each subscription/unsubscription on the same device. So, if a user subscribes/unsubscribes several times in a row on the same device, I wind up with several endpoints saved for that user (all but one of which are bad).

From what I have read, there is no reliable way to be notified when a user unsubscribes or an endpoint is otherwise invalidated. So, how can I tell if I should remove an old endpoint before adding a new one?

What's to stop a user from effectively mounting a denial of service attack by filling my db with endpoints through repeated subscription/unsubscription?

That's more meant as a joke (I can obvioulsy limit the total endpoints for a given user), but the problem I see is that when it comes time to send a notification, I will blast notification services with hundreds of notifications for invalid endpoints.

---

I want the subscribe logic on my server to be:

1. Check if we already have an endpoint saved for this user/device combo
2. If not add it, if yes, update it

The problem is that I can't figure out how to reliably do #1.

Answer 1

I will just add a new endpoint to the database for each device the user subscribes with

The best approach is to have a table like this:

endpoint | user_id

• add an unique constraint (or a primary key) on the endpoint: you don't want to associate the same browser to multiple users, because it's a mess (if an endpoint is already present but it has a different user_id, just update the user_id associated to it)
• user_id is a foreign key that points to your users table

if a user subscribes/unsubscribes several times in a row on the same device, I wind up with several endpoints saved for that user (all but one of which are bad).

Yes, unfortunately the push API has a wild unsubscription mechanism and you have to deal with it.

The endpoints can expire or can be invalid (or even malicious, like android.chromlum.info). You need to detect failures (using the HTTP status code, timeouts, etc.) when you try to send the push message from your application server. Then, for some kind of failures (permanent failures, like expiration) you need to delete the endpoint.

What's to stop a user from effectively mounting a denial of service attack by filling my db with endpoints through repeated subscription/unsubscription?

As I described above, you need to properly delete the invalid endpoints, once you realize that they are expired or invalid. Basically they will produce at most one invalid request. Moreover, if you have high throughput, it takes only a few seconds for your server to make requests for thousands of endpoints.

My suggestions are based on a lot of experiments and thinking done when I was developing Pushpad.

Answer 2

Another way is to have a keep alive field on you server and have your service worker update it whenever it receives a push notification. Then regularly purge endpoints which haven't been responded to recently.

Answer 3

I'm guessing that in the real world, most users don't unsubscribe and re-subscribe very often. However, Push subscriptions getting stale can be a problem. And users with more than one device can get notified on the wrong device if your server is using the most recent Push subscription and the user has meanwhile returned to a device with a subscription which is older, but still valid.

One way to deal with this, which I have considered and dismissed, would be to re-subscribe the user every time they fire up a client. However, I could find no information from the Apple/Google/etc (the big browser push service providers) as to whether there are any limits on this or whether it's a bad practice. Don't want to get black-listed.... [BTW, Apple gives errors to push requests as JSON in the response content, whereas Google sends plain text. Broke my server code with Chrome client after writing it while testing on Safari.]

So, rather than simply inserting the latest subscription into, say, a column in a users db table, perhaps one should insert it into a new table with a primary key consisting of user_id /and/ browser_id. I'll show an example of what browser_id might be below. Then, when the user comes from a particular browser on a particular device, look up the appropriate subscription and fire away at the endpoint.

[Another option would be to store the complete subscription JSON in a cookie or localStorage and send it to your server very often, but a few hundred extra bytes on every HTTP request seems like a waste. Thoughts anyone?]

Anyway, here's my take on making a smaller browser_id and using it along with user_id to look up the subscription JSON on the server side (the comment is a little redundant; you may just want to look at the code):

    /*
        This is to help the server use the right web Push API subscription if the user switches
        between different browsers; e.g. say a user is on browser A and the server caches a push
        subscription for them, and then they go to browser B and we cache a new subscription for
        them, and then they go back to browser A and we don't cache a new subscription for them
        because browser A already has a subscription going because the service worker is still
        running from their previous session; then the server wants to send them a notification
        and it uses the browser B subscription instead of the browser A one; clearly the 
        subscriptions need to be cached per user, per browser -- but how to identify each browser
        uniquely?  Upon first visiting the site, we'll create a browser_id that combines the
        users initial IP address with a timestamp; if the visitor is later issued a user_id, 
        we will store any Push subscriptions in a db table with primary key of user_id and 
        browser_id.     Sample browser_id: 10.0.0.255-1680711958431
    */
    async function set_browser_id() {

        let browser_id = Cookies.get('browser_id') || localStorage.browser_id;

        if ( ! browser_id) {

            /*
            const response = await fetch("https://api.ipify.org?format=json");
            const jsonData = await response.json();
            const ip_addr = jsonData.ip;
            */
            const timestamp = Date.now();

            browser_id = `${ip_addr}-${timestamp}`; // server set JS const ip_addr earlier

            console.log(`NEW browser_id: ${browser_id}`);
        }
        else { console.log(`EXISTING browser_id: ${browser_id}`); }

        // set whether new or existing to keep it from ever expiring
        Cookies.set('browser_id', browser_id, 
                { path: '/', domain: `.${hostname}`, expires: 365, secure: true });
        
        localStorage.browser_id = browser_id; // backup in persistent storage
    }

    window.onload = set_browser_id; // do it after everything else

The browser_id should be pretty persistent given the localStorage backup of the cookie (and might be useful for analytical purposes). Now your server can get a short little browser_id cookie with every request and can use it to look up the longer Push subscription JSON. In fact, you could probably skip the user_id in the db table since the chance of a browser_id namespace collision is infinitesimal. Of course, whenever the user generates a new subscription, you'll need the client to send that over so the server can update the db table.

I'd be curious what others think about this plan; I'm implementing it presently. And also about any security concerns with passing the complete subscription JSON around in cookies repeatedly -- your private VAPID key is needed to send a notification, right? But....

p.s. I'm on about this now since Apple /finally/ turned on the Push API with iOS 16.4 -- yay! About time.