Access GoogleCloudStorage using GoogleCloudEndpoints

Question

I'm working on this project in which I'm using a Google-App-Engine backend connected to an Android app via Google-Cloud-Endpoints. For Google-Cloud-Datastore access I'm using Objectify and everything works fine.

Now I decided to add the functionality to upload images to Google-Cloud-Storage but I couldn't find a clear explanation on how to do this using the Google-Cloud-Endpoints setup.

I found the following explanation how to use Google-Cloud-Storage with Google-App-Engine: https://cloud.google.com/appengine/docs/java/googlecloudstorageclient/app-engine-cloud-storage-sample

but instead of adding it to the Endpoints Api the article writes an additional servlet.

Furthermore I found this example of upload/download for Android:

github.com /thorrism/GoogleCloudExample

Sadly this is using the Google Cloud Storage API for direct access to the Google-Cloud-Storage and you need to add a P12-file to the asset folder, which seems unsecure.

My Google-App-Engine code looks like that:

@Api(
    name = "example",
    version = "v1", 
    scopes = { Constants.EMAIL_SCOPE }, 
    clientIds = { Constants.WEB_CLIENT_ID, Constants.ANDROID_CLIENT_ID, Constants.API_EXPLORER_CLIENT_ID }, 
    audiences = {Constants.ANDROID_AUDIENCE},
    description = "API for the Example Backend application."

)

public class ExampleApi{

@ApiMethod(name = "doSomething", path = "dosomething", httpMethod = HttpMethod.POST)
public String doSomething(@Named("text") String text){

  TestEntity test = new TestEntity(text);

   ofy().save().entity(test).now();

   return  test;

}

After I uploaded it I generated the Endpoints Client Library and imported it into my android project.

Then I'm calling Endpoints from Android like explained here:

https://cloud.google.com/appengine/docs/java/endpoints/calling-from-android#creating_the_service_object

public static com.appspot.******.example.Example buildServiceHandler(Context context, String email) {
    GoogleAccountCredential credential = GoogleAccountCredential.usingAudience(
            context, AppConstants.AUDIENCE);
    credential.setSelectedAccountName(email);

     com.appspot.******.example.Example.Builder builder  = new  com.appspot.******.example.Example.Builder(
            AppConstants.HTTP_TRANSPORT,
            AppConstants.JSON_FACTORY, null);
    builder.setApplicationName("example-server");
    return builder.build();
}

sApiServiceHandler = buildServiceHandlerWithAuth(context,email);

And each Api-Method I call like this:

com.appspot.******.example.Example.DoSomething doSomething = sApiServiceHandler.doSomething(someString);
doSomething.execute();

All of this works fine, but only for storing/receiving Datastore Entities. How would I go about uploading/downloading files to Google Cloud Storage using the Google Cloud Endpoints setup? Is it somehow possible to send a POST with my image data via Endpoints to the UploadServlet using the already build ServiceHandler ? Is it possible to call a servlet from an Endpoints Method? How am I supposed to send the Post to the Servlet and how would I go about the authentication?

Any help or advice would be greatly appreciated!

Answer 1

There are different ways to do this, but the most recommended way is to use Signed URLs, so that your Android app can upload the file securely to Google Cloud Storage directly, without going through your Endpoints backend. The basic process is:

1) Create an Endpoints method that creates a new signed URL and returns it to the Android client. Signing the URL on the server still requires a P12 key but is stored on App Engine, not on the client, so is secure. Try to use a short expiration for the URL, for example no more than 5 minutes.

2) Have the Android client upload the file directly to the signed URL, as you would doing a normal HTTP PUT to the Cloud Storage XML API to upload a file (resumable uploads with the JSON API are also supported, but not covered here).

Your Endpoints method might look like this:

@ApiMethod(name = "getUploadUrl", path = "getuploadurl", httpMethod = HttpMethod.GET)
public MyApiResponse getUploadUrl(@Named("fileName") String fileName
                                  @Named("contentType" String contentType)
{
        String stringToSign
            = "PUT\n" + contentType
            + "\n" + EXPIRATION_TIMESTAMP_EPOCH_SECONDS + "\n"
            + YOUR_GCS_BUCKET + "/" + fileName;

        // Load P12 key
        FileInputStream fileInputStream = new FileInputStream(PATH_TO_P12_KEY);
        KeyStore keyStore = KeyStore.getInstance("PKCS12");
        keyStore.load(fileInputStream, password);
        PrivateKey key = keyStore.getKey(privatekey", YOUR_P12_KEY_PASSWORD);

        // Get signature
        Signature signer = Signature.getInstance("SHA256withRSA");
        signer.initSign(key);
        signer.update(stringToSign.getBytes("UTF-8"));
        byte[] rawSignature = signer.sign();
        String signature = new String(Base64.encodeBase64(rawSignature, false), "UTF-8");

        // Construct signed url
        String url
            = "http://storage.googleapis.com/" + YOUR_GCS_BUCKET + fileName
            + "?GoogleAccessId=" + P12_KEY_SERVICE_ACCOUNT_CLIENT_ID
            + "&Expires=" + EXPIRATION_TIMESTAMP_EPOCH_SECONDS
            + "&Signature=" + URLEncoder.encode(signature, "UTF-8");

        // Endpoints doesn't let you return 'String' directly
        MyApiResponse response = new MyApiResponse();
        response.setString(url);
        return response;
}

On the Android side, you might use the method like this:

// Get the upload URL from the API
getUploadUrl = sApiServiceHandler.getUploadUrl(fileName, contentType);
MyApiResponse response = getUploadUrl.execute();
String uploadUrl = response.getString();

// Open connection to GCS
URL url = new URL(uploadUrl);
HttpURLConnection httpConnection = (HttpURLConnection) url.openConnection();
httpConnection.setDoOutput(true);
httpConnection.setRequestMethod("PUT");
httpConnection.setRequestProperty("Content-Type", contentType);

// Write file data
OutputStreamWriter out = new OutputStreamWriter(httpConnection.getOutputStream());
out.write(fileData);
out.flush();

// Get response, check status code etc.
InputStreamReader in = new InputStreamReader(httpConnection.getInputStream());
// ...

(Disclaimer: I'm just typing code freely into a text editor but not actually testing it, but it should be enough to give you a general idea.)