Facing issue while uploading to Google cloud storage using signed URL

Question

I am creating a signedURL in an app engine endpoint and then serving it to the client. But when the client tries to upload using the signed URL, the cloud storage throws the following error

Access denied. Anonymous users does not have storage.objects.create access to bucket

The app engine code to generate signed URL is as follows:

private String getSignedUrl() {
String encodedUrl = null;
String httpVerb = "PUT";
String contentMD5 = "";
String contentType = "image/rgb";

Calendar calendar = Calendar.getInstance();
calendar.add(Calendar.MINUTE, 10);
long expiration = calendar.getTimeInMillis() / 1000L;
String canonicalizedExtensionHeaders = "";
String canonicalizedResource =
    "/<bucket_name>/<folder_name>/";

String stringToSign =
    httpVerb + "\n" + contentMD5 + "\n" + contentType + "\n"
        + expiration + "\n" + canonicalizedExtensionHeaders
        + canonicalizedResource;

AppIdentityService service =
    AppIdentityServiceFactory.getAppIdentityService();
String googleAccessId = service.getServiceAccountName();

String baseURL =
    "http://storage.googleapis.com/<bucket_name>/<folder-name>/";
SigningResult signingResult =
    service.signForApp(stringToSign.getBytes());
String encodedSignature = null;
try {
    encodedSignature =
        URLEncoder.encode(
            new String(Base64.encodeBase64(
                signingResult.getSignature(), false),
                "UTF-8"), "UTF-8").toString();
} catch (UnsupportedEncodingException e) {
    // TODO Auto-generated catch block
    e.printStackTrace();
}
encodedUrl =
    baseURL + "?GoogleAccessId=" + googleAccessId + "&Expires="
        + expiration + "&Signature=" + encodedSignature;

return encodedUrl;
}

And after obtaining the signed URL, I am using cURL to test the upload. I use the following command to upload the file

curl -X PUT -H "Content-Type: multipart/form" -F file=@"<file_path>";type=image/rgb <signed_url>

I have tried both POST and PUT in the cURL with the same result. Am I missing something here?

I don't know if this is the problem, but in general you probably want to sign URLs for HTTPS instead of HTTP. — Brandon Yarbrough, Jun 09 '16 at 16:32
Even with https and after removing the folder name as per @Andrei suggestion, I am unable to upload from the client. — Sudarshan Murthy, Jun 10 '16 at 07:40

Andrei Volgin · Answer 1 · 2016-06-09T19:41:39.450

1

Writing from memory here, but I think you need to provide the object name. Plus, you included folder name in the baseURL - only bucket name should be included. "folder" name is part of the object name.

edited Jun 09 '16 at 19:41

answered Jun 09 '16 at 19:35

Andrei Volgin

40,755
6
49
58

I actually want to generate the signed URL so that clients can upload to GCS without having google account. Hence, there is no object name in the base URL. As you suggested, I have removed the folder name from both the `canonicalizedResource` and `baseURL` in my code. Now I am observing the following error response `MissingSecurityHeaderYour request was missing a required header.
Authorization
` – Sudarshan Murthy Jun 10 '16 at 07:37
Can you check the client to make sure it's sending the URL query parameters? That sounds like the sort of thing that would happen if it were stripping the Signature parameter along the way. Another thing worth trying is giving the client a known good signed URL to see if it works. The gsutil utility can generate them for you. – Brandon Yarbrough Jun 10 '16 at 16:42
@SudarshanMurthy: The workflow is this: ask a user to select a file to upload, get the name of the file, request a signed URL from the backend, set this URL to your file upload form, submit the file upload form. – Andrei Volgin Jun 10 '16 at 17:32
@BrandonYarbrough Thank you. The client (cURL) was indeed stripping the Signature and Expires from the URL. The `MissingSecurityHeader` error went away after I enclosed the entire signed URL within double quotes. But I am facing another issue. Please see the comment below – Sudarshan Murthy Jun 12 '16 at 14:54
@AndreiVolgin Thank you. Signed URL with object name works for GET request. But when I try to do same for PUT, I get the error: `The request signature we calculated does not match the signature you provided. Check your Google secret key and signing method.` I tried using latest SDK as per this [answer](http://stackoverflow.com/a/17799789/5916957) but still same issue. I think issue could be in request. I am using cURL to test & curl command is: `curl -X PUT -H "ContentType: multipart/form-data" -F file=@;type=image/rgb `. Could you please let me know if this is correct? – Sudarshan Murthy Jun 12 '16 at 14:56

score 1 · Answer 2 · answered Nov 22 '17 at 15:55

1

You also have to URL encode googleAccessId and encodedSignature because of the @ sign in the service account email and a base64 encoded string may have the + character which is decoded as a space by the server.

answered Nov 22 '17 at 15:55

seyisulu

406
6
13

Facing issue while uploading to Google cloud storage using signed URL

2 Answers2