0

I'm trying to figure out how to upload a file to google cloud storage using rest API , i don't want to use the client Library .

i read the documents but it was not helpful for a beginner in this flied ,

anyone can give me a step-by-step how to do this ? and how the URL/header/body format should look like , if also can give me an examples that would be very helpful .

Ali Aqrabawi
  • 143
  • 1
  • 14

2 Answers2

1

If you're not going to use any of the helper libraries and are also a beginner, the hardest part of implementing an upload to GCS will likely be authenticating yourself. Let's ignore that for now.

The simplest way to upload an object to Google Cloud Storage is to make an HTTPS call to storage.googleapis.com that looks like this:

PUT /your-bucket-name/your-object.txt HTTP/1.1
Authorization:  (YOUR ACCESS TOKEN GOES HERE)
Content-Length: 20
Content-Type: text/plain-or-whatever; charset=utf-8
Host: storage.googleapis.com
User-Agent: YourApplication/1.0

This is a test file

That will upload a file named "your-object.txt" of type "text/plain-or-whatever" to the bucket "your-bucket-name", with the contents "This is a test file."

If your bucket allows anonymous users to upload files (you shouldn't do that), then just don't include the Authorization line and you're done.

Now, since you really don't want to use any client libraries, and that presumably includes Google's OAuth libraries, you're going to need to implement authorization yourself, so let me give you an overview.

First, though, if you want to try this out immediately, install the "gcloud" tool, login with "gcloud auth login", and the print an access token with gcloud auth print-access-token. Then use the Authorization header Authorization: Bearer whatever.gcloudprintedout. That way you can be off and running with GCS quickly. But the token will only last an hour or so, so you'll need to implement OAuth for real.

Google Cloud APIs use OAuth to handle their requests, which is a powerful but not simple auth mechanism. There's extensive documentation on how OAuth with Google works: https://developers.google.com/identity/protocols/OAuth2

And there's also more general information on authorizing Google Cloud requests: https://cloud.google.com/docs/authentication

If you are running your application on a Google Cloud technology like App Engine or GCE, auth will be somewhat easier, but I will assume you're running this on your own machine. I will further assume that you want your application to have its own identity, rather than simply having you log in as part of the upload flow. For such a case, you'll need a service account, which will have an associated private key.

The basic flow for a service account is that you will create a JWT request for access credentials, then cryptographically sign that request with your private key, then send that signed request to Google. It will return you a token that may then be passed to your actual upload request later. You can keep using that token until it expires, at which time you'll need to build another JWT request to request another token.

Again, the client libraries entirely take care of this whole process for you. I am describing the approach of implementing everything exclusively on your own.

Brandon Yarbrough
  • 37,021
  • 23
  • 116
  • 145
  • thanks Brandon for the detailed answer , i went through the whole process of generating the token , i was able to get JWT , but when i add it to myurl as assertion variable it gave me a BAD request error : – Ali Aqrabawi Oct 21 '16 at 18:33
  • the error : { "error": "invalid_grant" "error_description": "Invalid JWT: Token must be a short-lived token and in a reasonable timeframe" "error_uri": "" } – Ali Aqrabawi Oct 21 '16 at 18:33
  • Your clock may be wrong, or you asked for too much time: http://stackoverflow.com/questions/30115933/access-token-and-refresh-token-giving-invalid-grant-in-google-plus-in-python/30117441#30117441 – Brandon Yarbrough Oct 21 '16 at 18:52
  • i checked on this on this , but my clock is correct , what i see in the header resposne : – Ali Aqrabawi Oct 21 '16 at 19:06
  • Content-Type: application/json; charset=UTF-8 Date: Fri, 21 Oct 2016 19:02:28 GMT Expires: Fri, 21 Oct 2016 19:02:28 GMT – Ali Aqrabawi Oct 21 '16 at 19:06
  • my JWT claim set is : "exp":1378550785,"iat":1328550785} , i believe i need to change the iat time , but not sure how can set it to match today :/ – Ali Aqrabawi Oct 21 '16 at 19:08
  • "iat" should be the number of seconds, as of the time you make the call, since January 1, 1970 UTC, i.e. epoch timestamp. For example, right now it's 1477078054. exp is the expiration time, so it should be iat + 3600 or less. – Brandon Yarbrough Oct 21 '16 at 19:28
  • alright , i'm almost there , the issue is that the signiture is invalid , – Ali Aqrabawi Oct 21 '16 at 20:53
  • this is how i generated the sig : 1)base64 encoded the {"alg":"RS256","typ":"JWT"} , and then encoded the JWT claim , i paste them as follow (head protected encoded).(JWTClaim encoded). i used this for encoding : http://www.url-encode-decode.com/base64-encode-decode/ 2)i generated the sig using this tool : http://www.online-convert.com/result/e0c0c68f9de7c7052eeb4eb2fea7b10e it gave a base64 result , i encoded that again by the same tool in step 1 , any recommended tool for this ? – Ali Aqrabawi Oct 21 '16 at 20:59
  • Are you encrypting the string using the RSA key for the service account, using RSASSA-PKCS1-V1_5-SIGN aka SHA256withRSA? – Brandon Yarbrough Oct 22 '16 at 06:13
  • i'm trying to do this through online tools , the tools never show that option , it only show me (SHA256 hash) as an option , and the field for the private key if i want to calculate HAMC, i'm just ran out of options , do you recommend any specific tool ? weatehr online of a cmd utility to do this ? does openssl can do such thing ? – Ali Aqrabawi Oct 22 '16 at 08:19
  • The third part of the JWT, after the base64-encoded header and the base64-encoded claim set, is the base64-encoded signature. The signature is the result of signing the base string of the first two parts. There are a variety of tools that can sign a string given an RSA key, for example `openssl rsautl -sign`. – Brandon Yarbrough Oct 23 '16 at 05:48
0

You can find the same example here:

https://stackoverflow.com/a/53955058/4345389

in which I already explained how to upload a file to google cloud storage using rest API.

Thanks

Jayoti Parkash
  • 868
  • 11
  • 26