Missing "access-control-allow-origin" in PUT response header of GCS resumable upload

Question

I am currently trying to implement resumable upload from a web browser to a google cloud storage bucket. I am using the documented procedure as follows:

1) initiate the resumable upload via a http POST send from the web server to GCS.

2) send the returned upload url to the browser and issue a Http PUT via XmlHttpRequest to upload the data.

Everything seems to work well until the very end. I.e. the browser first sends the OPTIONS preflight request which returns OK (200) and then a PUT request during which the file is uploaded. The PUT also return OK(200) but contains no Access-Control-Allow-Origin in the header. This will then cause the XmlHttpRequest to trigger an error.

I cannot figure out why this header field is not returned. I think from the GCS side the upload was successful as the file actually appears in the bucket. However, the browser believes an error occurred. Here is HTTP record taken right out of the Chrome developer console.

OPTIONS Request

Request URL: "https://" BUCKET.storage-upload.googleapis.com/1485967698353.e57?upload_id=AEnB2UohgRBG272hoHLZ9i-wLeTn45KKoMjTDEQGu-GoUl-1JQf5_sOnf7IjtpN0wuYzHgzEu3Qi9tpVHGrru--cwY7q2jNQdw
Request Method:OPTIONS
Status Code:200
Remote Address:[2607:f8b0:4009:811::2010]:443

OPTIONS Response Header

access-control-allow-credentials:true
access-control-allow-headers:content-range, content-type, x-upload-content-type
access-control-allow-methods:PUT
access-control-allow-origin:http://www.example.com
alt-svc:quic=":443"; ma=2592000; v="35,34"
content-length:0
content-type:text/html; charset=UTF-8
date:Wed, 01 Feb 2017 16:48:18 GMT
server:UploadServer
status:200

PUT Request

Request URL: "https:"//BUCKET.storage-upload.googleapis.com/1485967698353.e57?upload_id=AEnB2UohgRBG272hoHLZ9i-wLeTn45KKoMjTDEQGu-GoUl-1JQf5_sOnf7IjtpN0wuYzHgzEu3Qi9tpVHGrru--cwY7q2jNQdw
Request Method:PUT
Status Code:200
Remote Address:[2607:f8b0:4009:811::2010]:443

PUT Response Headers

alt-svc:quic=":443"; ma=2592000; v="35,34"
content-length:0
content-type:text/html; charset=UTF-8
date:Wed, 01 Feb 2017 16:48:19 GMT
etag:"c6f30652db5986aec4c00e80a9d00f25"
server:UploadServer
status:200
vary:Origin
x-goog-generation:1485967699029000
x-goog-hash:md5=xvMGUttZhq7EwA6AqdAPJQ==
x-goog-hash:crc32c=/vx4zQ==
x-goog-metageneration:1
x-goog-stored-content-encoding:identity
x-goog-stored-content-length:743424
x-guploader-uploadid:AEnB2UohgRBG272hoHLZ9i-wLeTn45KKoMjTDEQGu-GoUl-1JQf5_sOnf7IjtpN0wuYzHgzEu3Qi9tpVHGrru--cwY7q2jNQdw

So, there is no "access-control-allow-origin" in the PUT response. When I use a signed URL to perform an upload GCS actually returns the "Access-Control-Allow-Origin". However this will not be a resumable upload which I absolutely need.

This problem is almost identical to

XMLHttpRequest CORS to Google Cloud Storage only working in preflight request

but the solution given there has no effect. Here is the original request to initiate the upload (tokens are no real and there are quotes around "http/s" to scramble links). The "origin" is in the request.

POST "https"://BUCKET.storage-upload.googleapis.com/1485967698353.e57
Accept-Encoding: gzip
Authorization: Bearer 2342342234234234234234234234-234234234234234234234234234234234234234234234234sxx-FSpWkCqI0BCRWoG2342423s423423423A4234_234E23s423i423423423423423423423423423234234234234234234234234234234234234234234234234X
User-Agent: Google-HTTP-Java-Client/1.22.0 (gzip)
x-goog-resumable: start
origin: "http:"/www.example.com
Content-Length: 0

I can probably work around this by simply ignoring the XmlHttpRequest error, but that would result in some strange coding on the client. Ideally there is an answer to this...

Answer 1

A bit late to the party, but I was currently faced with the same issue. Here is how I manage to do the POST on the server and the PUT on the client.

1> Set CORS configuration on your bucket

Doc here: https://cloud.google.com/storage/docs/gsutil/commands/cors

First of all you must set the CORS correctly on the bucket you are using. For that you can use gsutils.

Create a file cors.json with your CORS configuration and set it on your bucket

$> cat cors.json
[{"method": ["PUT", "POST"], "origin": ["http://localhost:3000"]}]

$> gsutil cors set cors.json gs://your-bucket-name
$> gsutil cors get gs://user-upload
[{"method": ["PUT", "POST"], "origin": ["http://localhost:3000"]}]

Now your bucket will pass the right Header for PUT and POST on request coming from "http://localhost:3000". You do have to allow POST because you will later pass the origin header to it. Remember, it's really one upload broke in 2.

2> Pass the "Origin" header in the initial POST

Doc here: https://cloud.google.com/storage/docs/xml-api/resumable-upload#step_1wzxhzdk14initiate_the_resumable_upload

In python, your request should look like:

req = session.post(url, data="", headers={
  'Content-Length': '0',
  'x-goog-resumable': 'start',
  'Authorization': 'Bearer ' + creds.get_access_token().access_token,
  'Origin': 'http://localhost:3000'
})
req.raise_for_status()

return req.headers['location']

Note:

• Authorization may or may not be required depending on the ACL on your bucket. I personally would never open entirely a bucket to everyone, I would advise to have strict ACL that only your server can access and use the 'Authorization' header.

• req.headers['location'] hold the url you need to send to the client. The one you will use for the PUT request.

3> Make the request on the client

In javascript it will look something like this:

const xhr = new XMLHttpRequest();
xhr.open("PUT", signedUrl, true);
xhr.setRequestHeader('Content-type', file.type); # if you specified it when building the signed url

xhr.onload = function () {
   console.log("Success"
}
xhr.send(file);

If you still face some issue, try to change your CORS config to the following (only for debug purposes):

[{"method": ["*"], "origin": ["*"]}]

Answer 2

Before I forget about this post, I wanted to leave the solution I finally found. The problem was with these steps

1) initiate the resumable upload via a http POST sent from the web server to GCS.

2) send the returned upload url to the browser and issue a Http PUT via XmlHttpRequest to upload the data.

I ended up doing both 1) and 2) on the client. For that, I created a signed url for the initial post on the server and sent that to the browser. The browser then issues both, the POST to initiate the upload and the subsequent PUT to send the file.

Some solutions here suggest that you can perform the initial POST on the server, but that did not work for me.

Answer 3

When you initially request a resumable upload url (usually from your server, this is the "POST" above), you need to specify an "origin". This needs to be whatever the origin is that the browser will send when it starts to send the file, ie: the value of location.origin in the browser. And it needs to be exact.

In my code, I achieved this by implementing the request resumable upload function on the server, but getting the browser to pass location.origin to this function.

Note that in the question above, origin is being passed, but it doesn't look right (has some weird double quotes in it).