Creating new collection and document with Firestore REST API returning HTTP 400

Question

Hello stackoverflow community,

I am using Firestore and not Realtime database for a test project I am working on.

As a starter, I am trying to create a new document inside an existing collection and project which I manually created through the web interface. My project id is newproject-30f72 and the already created collection name is testcollection.
I have enabled the Email/Password authentication method, and I have registered a new email-id/password user. The rule I have for auth enabled write is as follows.

service cloud.firestore {
  match /databases/{database}/documents {
    match /{document=**} {
      allow read, write: if request.auth.uid != null;
    }
  }
}

At this point, I am not sure, a barer token alone passed in the header is enough to validate the request.auth.uid != null, so most probably my rule could also be wrong, as I was using the simulator to auto generate the rule, but I verified that the rule is not causing the problem I am facing at the moment by disabling the rule completely.

I have enabled rules so that only authenticated users can write to firestore. (and by authentication, I mean the idToken for a specific email/password based user and not the apiKey for a web app. I don't know if this is possible though.)
I have enabled the Add Firebase to your web app / register app and that gave me a json object named firebaseConfig with the following details.


   var firebaseConfig = {
      apiKey: "some_api_key",
      authDomain: "newproject-30f72.firebaseapp.com",
      databaseURL: "https://newproject-30f72.firebaseio.com",
      projectId: "newproject-30f72",
      storageBucket: "newproject-30f72.appspot.com",
      messagingSenderId: "111122223333",
      appId: "1:1111:web:2222"
   };

I am using the apiKey from the above json object to get an idToken valid for some time, to write to the firestore database. The code for that is as below in python3.

import json
from urllib import request, error
from collections import defaultdict

firebase_apikey = 'some_api_key'
auth_request_url = "https://www.googleapis.com/identitytoolkit/v3/relyingparty/verifyPassword?key={}".format(firebase_apikey)

class auth:
    def __init__(self, email: str, password: str):
        self.email = email
        self.password = password
        self.post_data = defaultdict()
        self.post_data['email'] = self.email
        self.post_data['password'] = self.password
        self.post_data['returnSecureToken'] = True
        self.headers = {"Content-Type": "application/json"}
        # print("POST DATA IS:: " + json.dumps(self.post_data))
        self.r = request.Request(auth_request_url,
                                 data=json.dumps(self.post_data).encode(),
                                 headers=self.headers)
        # self.url_open

    def get_auth_token(self):
        try:
            self.url_open = request.urlopen(self.r)
        except Exception as e:
            return e
        try:
            return json.loads(self.url_open.read())
        except Exception as e:
            return e


s = auth("someuser.somethingelse@gmail.com", "somepassword")
response = s.get_auth_token()
id_token = response['idToken']
expires_in = response['expiresIn']

I got the idToken which is a 924 character long string.

Now I am trying to write to firestore with the idToken that I received and valid for 3600(seconds I presume) using the Authorization': 'Bearer header as follows.

firestore_project_url = "https://firestore.googleapis.com/v1beta1/projects/{}/databases/(default)/documents:write".format(
    'newproject-30f72')

headers = {
    'Content-type': 'application/json',
    'Authorization': 'Bearer %s' % id_token,
}

test_data = '''
{
    "writes": [{
        "updateMask": {
            "fieldPaths": ["name"]
        },
        "update": {
            "name": "projects/newproject-30f72/databases/(default)/documents/testcollection/testdoc/",
            "fields": {
                "name": {
                    "stringValue": "test"
                }
            }
        }
    }]
}
'''

test_data_json_bytes = json.dumps(test_data).encode("utf-8")
req = request.Request(url=firestore_project_url,
                      data=test_data_json_bytes,
                      headers=headers,
                      method='POST')
print(headers)
f = request.urlopen(req)

I can see the headers as

{'Content-type': 'application/json', 'Authorization': 'Bearer eyJhbGciOiJSUzI1NiIsI........<snip>'}

but I am getting a urllib.error.HTTPError: HTTP Error 400: Bad Request error.

I was referring to Firebase Firestore REST example to check it using curl with custom headers to add the bearer token and to see if there is any possible verbosity, and I am seeing

*   Trying 74.125.68.95...
* TCP_NODELAY set
* Connected to firestore.googleapis.com (74.125.68.95) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*   CAfile: /etc/ssl/certs/ca-certificates.crt
  CApath: none
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
* TLSv1.3 (IN), TLS handshake, Certificate (11):
* TLSv1.3 (IN), TLS handshake, CERT verify (15):
* TLSv1.3 (IN), TLS handshake, Finished (20):
* TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.3 (OUT), TLS handshake, Finished (20):
* SSL connection using TLSv1.3 / TLS_AES_256_GCM_SHA384
* ALPN, server accepted to use h2
* Server certificate:
*  subject: C=US; ST=California; L=Mountain View; O=Google LLC; CN=*.googleapis.com
*  start date: May 14 13:35:00 2019 GMT
*  expire date: Aug  6 13:20:00 2019 GMT
*  subjectAltName: host "firestore.googleapis.com" matched cert's "*.googleapis.com"
*  issuer: C=US; O=Google Trust Services; CN=Google Internet Authority G3
*  SSL certificate verify ok.
* Using HTTP2, server supports multi-use
* Connection state changed (HTTP/2 confirmed)
* Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0
* Using Stream ID: 1 (easy handle 0x560e5884af30)
> POST /v1beta1/projects/newproject-30f72/databases/(default)/documents/testcollection3 HTTP/2
> Host: firestore.googleapis.com
> User-Agent: curl/7.64.1
> Accept: */*
> {Content-type: application/json, Authorization: Bearer eyJhbGciOiJSUzI1NiIsImtpZCI6........<snip>}
> Content-Type: application/json
> Content-Length: 258
> 
* We are completely uploaded and fine
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* old SSL session ID is stale, removing
* Connection state changed (MAX_CONCURRENT_STREAMS == 100)!
< HTTP/2 400 
< content-type: text/html; charset=UTF-8
< referrer-policy: no-referrer
< content-length: 1555
< date: Fri, 31 May 2019 14:31:36 GMT
< 
* HTTP/2 stream 0 was not closed cleanly: PROTOCOL_ERROR (err 1)
* stopped the pause stream!
* Connection #0 to host firestore.googleapis.com left intact
curl: (92) HTTP/2 stream 0 was not closed cleanly: PROTOCOL_ERROR (err 1)
* Closing connection 0

I was referring to the following links for references.

https://groups.google.com/forum/#!topic/google-cloud-firestore-discuss/4Le2RskC3cg https://firebase.google.com/docs/firestore/reference/rest/v1beta1/projects.databases.documents/commit

What I am trying to achieve is

use the email/password auth to get authenticated to firestore
create collection and write docs to it.

Any help is greatly appreciated.

Thank you very much.

I am fine with using firebase-admin too, but I couldn't find a documentation reference for signing in with idToken using emailid/password authentication (not with oauth or the json key file). Any reference or sample code to authentication and further CRUD operation is greatly appreciated. Thank you @Will — nohup, May 31 '19 at 19:05
Well, if client passes you idToken then use [Firebase Admin](https://firebase.google.com/docs/auth/admin/verify-id-tokens). If you client gives you email/password then use [REST API](https://firebase.google.com/docs/reference/rest/auth#section-sign-in-email-password). I've used the latter. — Will, Jun 02 '19 at 21:35
@Will Please correct me if I am wrong, I am using the REST API itself now, where the payload sets `returnSecureToken` to `True` and it returns an idToken. I am stuck from that point. I cannot find a doc that says how to use that idToken. Can you kindly point me to any docs or a reference that you've used, or a github repo, or a sample snippet that I can use as a reference? Thank you very much. — nohup, Jun 03 '19 at 04:20
ah, apologies for not understanding the question properly. You want Firestore to kick its rules in because you're basically impersonating a user? — Will, Jun 03 '19 at 10:14
OK, I've tested a GET using `Authorization: Bearer ` and it works fine. A 400 error implies something on your body is wonky. Have you tested that a simple GET on a collection works with auth passed the way you do? — Will, Jun 03 '19 at 11:16

Will · Accepted Answer · 2019-06-03T11:44:44.080

SO this is what worked for me (I used requests module but net effect is the same)

First the Auth, exactly as you do.

url = "https://www.googleapis.com/identitytoolkit/v3/relyingparty/verifyPassword?key=" + API_KEY
payload = {"email": email, "password": password, "returnSecureToken": True}
rsp = requests.post(url, data=payload)
id_token = rsp.json().get("idToken")

# User idToken from your auth for later requests
headers = {
    'Content-type': 'application/json',
    'Authorization': "Bearer %s" % id_token
}

Now create a document in a collection

# Create a doc in "testcoll"
url = "https://firestore.googleapis.com/v1beta1/projects/" + PROJECT + "/databases/(default)/documents/testcoll"
payload = {
  "fields": {
    "meaningOfLife": {"integerValue": 42}
  }
}
# Create Doc
rsp = requests.post(url, headers=headers, data = json.dumps(payload))
assert(rsp.status_code == 200)
docref = rsp.json().get("name")
print(rsp.json())

Now update the same document

# Update doc
url = "https://firestore.googleapis.com/v1beta1/" + docref
payload = {
  "fields": {
    "meaningOfLife": {"stringValue": '43'}
  }
}
rsp = requests.patch(url, headers=headers, data = json.dumps(payload))
assert(rsp.status_code == 200)
print(rsp.json())

Thank you very much @Will. You are a life saver. This worked. — nohup, Jun 03 '19 at 15:13
I'm glad I worked through it because I wasn't sure at the time if the idToken could be passed in REST calls. The SDKs and libraries don't support this at the moment. — Will, Jun 03 '19 at 16:57

Creating new collection and document with Firestore REST API returning HTTP 400

1 Answers1

Linked