0

Our c# .net wpf application connects to Google Cloud Firestore using custom tokens. The custom token contains embedded metadata that our Cloud Firestore Rules use to restrict access. The token is generated by a node.js Firebase Function on the same server.

This was working 100% for the past 18 months. But in the last week our users have started receiving an Unauthenticated error about 40% of the time when reading/writing data. It still works about 60% of the time.

E.G. The line await batch.CommitAsync(); will generate the error. The error never occurs while authenticating or creating the token or FirestoreDb, only when using the FirestoreDb to perform actions, and only sometimes.

The error is always the same: Grpc.Core.RpcException: Status(StatusCode="Unauthenticated", Detail="Request had invalid authentication credentials. Expected OAuth 2 access token, login cookie or other valid authentication credential. See https://developers.google.com/identity/sign-in/web/devconsole-project.") at Google.Cloud.Firestore.WriteBatch.<CommitAsync>d__15.MoveNext() in /_/apis/Google.Cloud.Firestore/Google.Cloud.Firestore/WriteBatch.cs:line 226"

If the user immediately reconnects, the data can be read/written about 90% of the time. 10% of the time the same error will then thrown again.

It seems like something has recently changed at Google that is stopping this from working. Yet it still works 60% of the time, which is very strange...

The relevant c# code is:

using Firebase.Auth;
using Flurl.Http;
using Google.Cloud.Firestore;
using Google.Cloud.Firestore.V1;
using Grpc.Core;

dynamic response = await GetAuthenticationToken(requestToken);
FirestoreDb db = CreateFirestoreDbWithToken(response.token, FIREBASE_WEB_API, FIREBASE_PROJECT_ID);

private FirestoreDb CreateFirestoreDbWithToken(string token, string firebaseApiKey, string firebaseProjectId)
{
    FirebaseAuthProvider authProvider = new FirebaseAuthProvider(new FirebaseConfig(firebaseApiKey));
    firebaseAuth = authProvider.SignInWithCustomTokenAsync(token).Result;
    CallCredentials callCredentials = CallCredentials.FromInterceptor(async (context, metadata) =>
    {
        if (firebaseAuth.IsExpired())
            firebaseAuth = await firebaseAuth.GetFreshAuthAsync();
        if (string.IsNullOrEmpty(firebaseAuth.FirebaseToken))
            return;

        metadata.Clear();
        metadata.Add("authorization", $"Bearer {firebaseAuth.FirebaseToken}");
    });
    ChannelCredentials credentials = ChannelCredentials.Create(new SslCredentials(), callCredentials);
    FirestoreClientBuilder builder = new FirestoreClientBuilder { ChannelCredentials = credentials };
    return FirestoreDb.Create(firebaseProjectId, builder.Build());
}

public async Task<dynamic> GetAuthenticationToken(string requestToken)
{
    var functionPath = FIREBASE_FUNCTION_NAME + ".cloudfunctions.net/verifyClient";
    var response = await functionPath.WithHeader("Content-Type", "application/json; charset=utf-8")
    .PostJsonAsync
    (
        new
        {
            requestToken,
        }
    )
    .ReceiveJson();
    return response;
}

public WriteData()
{
    ...
    WriteBatch batch = db.StartBatch();
    batch.Set(dataRef, dataObject);
    await batch.CommitAsync();
    ...
}

Our node.js Firebase Function which verifies the client and returns the access code is:

const functions = require('firebase-functions');
const admin = require('firebase-admin');
admin.initializeApp(functions.config().firebase)

exports.verifyClient = functions
    .https.onRequest((request, response) => {
    
    ...
    // BUSINESS LOGIC
    ...
    
    return createAuthToken(requestEmail, groupID).then(authToken => {
        response.send({ result: "OK", token: authToken });
        return;
    })
});

function createAuthToken(requestEmail, groupID) {
    return admin.auth()
        .createCustomToken(requestEmail, { adminID: groupID })
        .then(customToken => {
            return customToken;
        })
        .catch(err => {
            console.error('createAuthToken Error: ' + err);
            return "";
        });
}

Does anyone have any ideas what the problem could be? Or how we can fix?

Clemens
  • 123,504
  • 12
  • 155
  • 268
Damian
  • 119
  • 1
  • 10
  • Can you refer to these links: [1](https://stackoverflow.com/questions/56828674/request-had-invalid-authentication-credentials-expected-oauth-2-access-token-l), [2](https://stackoverflow.com/questions/63000214/expected-oauth-2-access-token-login-cookie-or-other-valid-authentication-creden) and [3](https://stackoverflow.com/questions/48720306/request-had-invalid-authentication-credentials-expected-oauth-2-access-token-er) – Sandeep Vokkareni Jul 12 '22 at 12:43
  • Hi Sandeep, yes I've looked at all those links. None are relevent. 1: The token is not expired and the code already has a Refresh function that works. 2: We are not using Google-Auth. How would that work with FirestoreDb? We're generating the token ourselves, which is working fine. 3: Perhaps "Make sure your date and time are correct and also your timezone is correct." could be a problem, but again that doesn't make sense, since we can't adjust the Firestore server time. – Damian Jul 12 '22 at 22:38
  • Interestingly, we have 7 different Firebase servers in 7 different regions of the world. The Central USA is still working 100%, but the Pacific (Sydney) is failing 50% of the time. The code is EXACTLY the same. Also remember is still works 50% of the time. – Damian Jul 12 '22 at 22:40
  • It seems to be a known issue, according to the link: https://github.com/googleapis/nodejs-firestore/issues/1748 the issue got resolved itself. Can you check once to let me know if the issue is solved? – Sandeep Vokkareni Jul 14 '22 at 06:42
  • The problem still remains. These are production servers supporting over 30,000 users. We had no choice but to create 2 new Firebase servers and divert users to the new servers. This "resolved" the issue. – Damian Jul 15 '22 at 07:15

1 Answers1

0

It seems to be a known issue, according to the link and the issue has been resolved now. I can see you have made a comment in the same Github link. As it is still open you may follow up there.. The work around “By creating 2 new Firebase servers and diverting users to the new servers” seems fine to me and as to figure out the exact root cause for your issue although the respective github issue is fixed i would suggest you to contact Google Support Team.