Is it possible to use Firebase Realtime Database to implement a distributed mutex?

Question

I was thinking of using a transaction like so to implement a kind of distributed lock:

const lockId = 'myLock';
const lockRef = firebaseAdmin.database().ref(`/locks/${lockId}`);
lockRef.transaction(function(current) {
  if (current === null) {
    return '1';
  }
}, function(error, committed) {
  if (committed) {
    // .... Do the synchronized work I need ...
    lockRef.remove();
  }
});

The question I have is: will the update function be called with null only if the data does not exist?

Generally, is this a valid way to implement a distributed lock?

Answer 1

Your initial attempt would not work for reasons @FrankvanPuffelen stated in their answer.

But it is possible (although not really that straightforward) to accomplish this. I battled quite a long with different edge cases, and finally came up with this solution that passes a myriad of different tests which verify that this prevents all possible race conditions and deadlocks:

import crypto from 'crypto';
import { promisify } from 'util';

import * as firebaseAdmin from 'firebase-admin';

const randomBytes = promisify(crypto.randomBytes);

// A string which is stored in the place of the value to signal that the mutex holder has
// encountered an error. This must be unique value for each mutex so that we can distinguish old,
// stale rejection states from the failures of the mutex that we are currently waiting for.
const rejectionSignal = (mutexId: string): string => `rejected${mutexId}`;

const isValidValue = (value: unknown): boolean => {
  // `value` could be string in the form `rejected...` which signals failure,
  // using this function makes sure we don't return that as "valid" value.
  return !!value && (typeof value !== 'string' || !value.startsWith('rejected'));
};

export const getOrSetValueWithLocking = async <T>(id: string, value: T): Promise<T> => {
  const ref = firebaseAdmin.database().ref(`/myValues/${id}`);

  const mutexRef = firebaseAdmin.database().ref(`/mutexes/myValues/${id}`);

  const attemptingMutexId = (await randomBytes(16)).toString('hex');

  const mutexTransaction = await mutexRef.transaction((data) => {
    if (data === null) {
      return attemptingMutexId;
    }
  });

  const owningMutexId = mutexTransaction.snapshot.val();

  if (mutexTransaction.committed) {
    // We own the mutex (meaning that `attemptingMutexId` equals `owningMutexId`).
    try {
      const existing = (await ref.once('value')).val();
      if (isValidValue(existing)) {
        return existing;
      }
      /*
        --- YOU CAN DO ANYTHING HERE ---
        E.g. create `value` here instead of passing it as an argument.
      */
      await ref.set(value);
      return value;
    } catch (e) {
      await ref.set(rejectionSignal(owningMutexId));
      throw e;
    } finally {
      // Since we own the mutex, we MUST make sure to release it, no matter what happens.
      await mutexRef.remove();
    }
  } else {
    // Some other caller owns the mutex -> if the value is not yet
    // available, wait for them to insert it or to signal a failure.
    return new Promise((resolve, reject) => {
      ref.on('value', (snapshot) => {
        const val = snapshot.val();
        if (isValidValue(val)) {
          resolve(val);
        } else if (val === rejectionSignal(owningMutexId)) {
          reject(new Error('Mutex holder encountered an error and was not able to set a value.'));
        } // else: Wait for a new value.
      });
    });
  }
};

My use case for this was that I had Next.js API routes running in Vercel, where the only shared state of the parallelly executing serverless functions was a Firebase Realtime Database.

Answer 2

A transaction will be called initially with the client's best guess to the current value. If the client doesn't have a current value in memory, its best guess is that there is no current value.

That means that there is no guarantee whatsoever that no value actually exists in the database if you get a null.

Also see:

• Firebase transaction api call current data is null
• Firebase runTransaction not working - MutableData is null