Calling Firestore Database from another Cloud Project shows "Permission Error"

Question

After reading every piece of question, documentation available online, we couldn't find a solution, so posting a question here.

Our Setup :

1. Project FB: Used only for Firestore in Dev mode. We want to access data from this project in our dev backend server, hosted on a different cloud project.

2. Project GCP: A GCP Project with app deployed in GAE that makes a simple get() call to Project FB's document. The app is in Python, and it works perfectly in localhost, but not after it's deployed.

Sample Flask code that we are testing with:

route('/test-fb', methods=['POST', 'GET'])
def test():
    doc = doc_ref.get()
    if doc.exists:
        print(u'Document data: {}'.format(doc.to_dict()))
    else:
        print(u'No such document!')
    return make_response("Firestore worked!")

Note:

1. This is not a question on Firestore Rules, because we are using service account key. To be 100% sure in any case, we have rules set to: true always

   match /{document=**} { allow read, write: if true

2. We generate the Service Account private key by going to "Settings -> Service Accounts", generate new private key. Then use the Admin SDK config snippet code in Python above our code. This works perfectly in localhost.

3. Aware of the Service Account permissions needed, we added many permissions including 'Editor', 'Storage Admin' and 'Cloud Datastore Owner' to the Project FB IAM account for GAE service Account of Project GCP (@appspot.gserviceaccount.com)

4. All the Firestore packages, any other dependency are updated to the latest version.

5. Created new keys to test again. For Project FB, updated Credentials -> Key restrictions and set them to unrestricted so any domains can access them.

6. Deleted the versions, and tried again many times at different times of the day as well. The deployments happen through triggered cloud builds in Project GCP. The cloud builds are successful. Also, all routes function perfectly except the one in which we are reading the Firestore document (code above).

7. Deleted cookies, and tried different browsers.

8. Instead of using the snippet code, also tried the google-cloud-firestore package: https://pypi.org/project/google-cloud-firestore/

9. Both the projects are in the same location (US multilocation)

Please advise on what we could be doing wrong, and what else we can try? We are lost at this point, and this simple task has taken us several days, and we've tried all permutations of above steps multiple times to double check.

GAE Response on a request to the server:

Answer 1

I tried this with a minimal example and it worked. Make sure your GAE app is using the application default credentials.

Deploying a Flask app in project-foo with access to a Firestore DB in project-bar:

main.py

from flask import Flask
import firebase_admin
from firebase_admin import credentials
from firebase_admin import firestore

app = Flask(__name__)

# Use the application default credentials
cred = credentials.ApplicationDefault()
firebase_admin.initialize_app(cred, {
  'projectId': 'project-bar',
})

db = firestore.client()

@app.route('/test-fb', methods=['POST', 'GET'])
def test():
    doc = db.collection('users').document('123').get()
    if doc.exists:
        print(u'Document data: {}'.format(doc.to_dict()))
    else:
        print(u'No such document!')
    return 'Firestore worked!'

if __name__ == '__main__':

Deploy app in project-foo

gcloud set project project-foo
gcloud app deploy

Visit project-foo.uc.r.appspot.com/test-fb. As expected, see permission denied error in logs.

Give project-foo's default service account access to project-bar's Firestore DB

gcloud set project project-bar
gcloud projects add-iam-policy-binding project-bar \
  --member serviceAccount:project-foo@appspot.gserviceaccount.com \
  --role roles/datastore.user

Wait a few minutes for IAM binding to take hold, refresh project-foo.uc.r.appspot.com/test-fb. See Firestore worked!.