Try to download export from google vault, get 403 error using service account

Question

From documentation on https://developers.google.com/vault/guides/exports, I've been able to create, list, and retrieve exports, but I haven't found any way to download the exported data associated with a specific export. Is there any way to download the exported files via the API, or is this only available through the vault UI?

There is a cloudStorageSink key in the export metadata, but trying to use the values provided using the cloud storage API results in a generic permissions issue (403 Error).

There is error I got:

com.google.cloud.storage.StorageException: gcsadmin@gmail-acess-347301.iam.gserviceaccount.com does not have storage.objects.get access to the Google Cloud Storage object.
71e6f8ba-dc92-494f-b584-1a9675074c0b/exportly-f1c1ecd8-254c-4078-b0c2-5228905720d3/My_first_mail_accounts_export-metadata.csv  4d17606f-0ebc-46e7-8369-d8e4ffd12b44
    at com.google.cloud.storage.StorageException.translate(StorageException.java:118)
    at com.google.cloud.storage.spi.v1.HttpStorageRpc.translate(HttpStorageRpc.java:287)
    at com.google.cloud.storage.spi.v1.HttpStorageRpc.get(HttpStorageRpc.java:512)
    at com.google.cloud.storage.StorageImpl.lambda$get$6(StorageImpl.java:285)
    at com.google.api.gax.retrying.DirectRetryingExecutor.submit(DirectRetryingExecutor.java:103)
    at com.google.cloud.RetryHelper.run(RetryHelper.java:76)
    at com.google.cloud.RetryHelper.runWithRetries(RetryHelper.java:50)
    at com.google.cloud.storage.Retrying.run(Retrying.java:54)
    at com.google.cloud.storage.StorageImpl.run(StorageImpl.java:1406)
    at com.google.cloud.storage.StorageImpl.get(StorageImpl.java:284)
    at com.google.cloud.storage.StorageImpl.get(StorageImpl.java:290)
    at com.q1d.googlevaulttest.Quickstart.downloadObject(Quickstart.java:159)
    at com.q1d.googlevaulttest.Quickstart.main(Quickstart.java:215)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 403 Forbidden
GET https://storage.googleapis.com/storage/v1/b/4d17606f-0ebc-46e7-8369-d8e4ffd12b44/o/71e6f8ba-dc92-494f-b584-1a9675074c0b%2Fexportly-f1c1ecd8-254c-4078-b0c2-5228905720d3%2FMy_first_mail_accounts_export-1.zip?projection=full
{
  "code" : 403,
  "errors" : [ {
    "domain" : "global",
    "message" : "gcsadmin@gmail-acess-347301.iam.gserviceaccount.com does not have storage.objects.get access to the Google Cloud Storage object.",
    "reason" : "forbidden"
  } ],
  "message" : "gcsadmin@gmail-acess-347301.iam.gserviceaccount.com does not have storage.objects.get access to the Google Cloud Storage object."
}
    at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:146)
    at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:118)
    at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:37)
    at com.google.api.client.googleapis.services.AbstractGoogleClientRequest$1.interceptResponse(AbstractGoogleClientRequest.java:428)
    at com.google.api.client.http.HttpRequest.execute(HttpRequest.java:1111)
    at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:514)
    at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:455)
    at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:565)
    at com.google.cloud.storage.spi.v1.HttpStorageRpc.get(HttpStorageRpc.java:509)
    ... 10 more

As follow other user guide, I was trying to download the object not whole bucket, Here is the code I am using: (gcs.json key file is the service account I created in Google cloud which have domain-wide delegation)

     public static void downloadObject(String projectId, String bucketName, String objectName, 
       String destFilePath) throws IOException {
    
    // The ID of your GCP project
    // String projectId = "your-project-id";

    // The ID of your GCS bucket
    // String bucketName = "your-unique-bucket-name";

    // The ID of your GCS object
    // String objectName = "your-object-name";

    // The path to which the file should be downloaded
    // String destFilePath = "/local/path/to/file.txt";
    // Load client secrets.

    InputStream in = Quickstart.class.getResourceAsStream("/gcs.json");
    if (in == null) {
        throw new FileNotFoundException("Resource not found: " + "gcs.json");
    }
    Storage storage = StorageOptions.newBuilder().setCredentials(ServiceAccountCredentials.fromStream(in)).build()
            .getService();
    
    Blob blob = storage.get(BlobId.of(bucketName, objectName));
    //Blob blob = storage.get(BlobId.fromGsUtilUri(objectURI));
    blob.downloadTo(Paths.get(destFilePath));

    System.out
            .println("Downloaded object " + objectName + " from bucket name " + bucketName + " to " + destFilePath);
}

Was the `gcsadmin@gmail-acess-347301.iam.gserviceaccount.com` the account that created the export? — Puteri, Aug 25 '22 at 05:09

score 1 · Answer 1 · answered Aug 25 '22 at 03:07

1

This part of the error indicates how to solve the problem:

gcsadmin@gmail-acess-347301.iam.gserviceaccount.com does not have storage.objects.get access to the Google Cloud Storage object

The solution is to add a role that has the required permission. For example:

Storage Object Viewer (roles/storage.objectViewer)

The role must be added to the service account:

gcsadmin@gmail-acess-347301.iam.gserviceaccount.com

gcloud projects add-iam-policy-binding REPLACE_WITH_YOUR_PROJECT_ID \
  --member='serviceAccount:gcsadmin@gmail-acess-347301.iam.gserviceaccount.com' \
  --role='roles/storage.objectViewer'

Documentation for the command:

answered Aug 25 '22 at 03:07

John Hanley

74,467
6
95
159

1

This will not work. This is because the bucket where the export lives is part of a Google Managed project and not the OP's project. They need to make the download using the account that created the export in Google Vault. – Puteri Aug 25 '22 at 05:07
Thanks for the answers, I agree with @Ferregina Pelona; – James Zou Aug 25 '22 at 22:25
I found a workaround,I was able to use the default credentials to download the export( by use the Google Cloud SDK. Create Application Default Credentials with gcloud auth application-default login, not set the credentials when get cloud storage service), however I got this waring: WARNING: Your application has authenticated using end user credentials from Google Cloud SDK. We recommend that most server applications use service accounts instead. If your application continues to use end user credentials from Cloud SDK, you might receive a "quota exceeded" or "API not enabled" error. – James Zou Aug 25 '22 at 22:26
I still want to figure out if I can use the service account to download the export to avoid above warning – James Zou Aug 25 '22 at 22:27
You can do it if the SA was the responsible of the export, otherwise maybe this can work https://support.google.com/a/answer/9807615#AssignToUsers https://support.google.com/vault/answer/2799699?hl=en#privileges – Puteri Aug 25 '22 at 22:30
1

@JamesZou - Since the export was performed as a Google Workspace user, I think you can use a service account that has Domain Wide Delegation: https://developers.google.com/admin-sdk/directory/v1/guides/delegation That will allow a service account to impersonate the user that performed the export. – John Hanley Aug 25 '22 at 22:32
@Ferregina Pelona, as shown in code I was trying to use SA to download export, got 403 error, I did set the vault privileges for the SA, but that doesn't help – James Zou Aug 26 '22 at 14:11
@JohnHanley I want to try that, when authenticate with Google Vault, how do you use a SA json ? can you give me example. really appreciate your help. – James Zou Aug 26 '22 at 14:13
I provided a link for Domain Wide Delegation. There are examples on Stack Overflow and in Google's documentation. I do not know of an example for Vault, but setting up credentials are the same for most libraries. – John Hanley Aug 26 '22 at 18:00

Try to download export from google vault, get 403 error using service account

1 Answers1