Delete multiple objects fron google cloud storage using node.js

Question

I need to delete multiple objects from google cloud storage. I have deleted one object at a time.

This is my code:

var gcloud = require('gcloud')({
  projectId: "sampleProject1"
});
var gcs = gcloud.storage();
var myBucket = gcs.bucket('sampleBucket1');
var file = myBucket.file('1.png');

file.delete(function (err, apiResponse) {
  if (err) {
    console.log(err);
  }
  else {
    console.log("Deleted successfully");
  }
});

But I need to delete multiple objects simultaneously. Is it possible or not?

Stephen · Answer 1 · 2019-05-03T18:01:18.897

We do have bucket#deleteFiles that will handle throttling the requests for you. You can use the prefix option to target multiple images by a naming convention, like:

bucket.deleteFiles({ prefix: 'image-' }, callback);

If that doesn't work, we also have a guide that shows how you can do the throttling logic yourself. See "My requests are returning errors instructing me to retry the request": https://googlecloudplatform.github.io/gcloud-node/#/docs/v0.29.0/guides/troubleshooting

Edit to elaborate on how to do the throttling using async:

var async = require('async');
var PARALLEL_LIMIT = 10;

function deleteFile(file, callback) {
  file.delete(callback);
}

async.eachLimit(filesToDelete, PARALLEL_LIMIT, deleteFile, function(err) {
  if (!err) {
    // Files deleted!
  }
});

score 0 · Answer 2 · answered Apr 04 '16 at 16:18

0

There is no way to atomically delete multiple objects from GCS at the same time.

However, you can issue multiple requests with a single call, which returns the status of each individual operation, using the batch API.

answered Apr 04 '16 at 16:18

jterrace

64,866
22
157
202

score 0 · Answer 3 · answered Nov 07 '20 at 12:16

I created an array of all the objects that I wanted to delete and then iterated over the array, executing the "delete" function specified on the google cloud storage documentation.

Delete GCS object

const storage = new Storage({keyFilename: 'google-credentials.json'});
const imagesToDelete = ['fileName1', 'fileName2', 'fileName3'];
    
    imagesToDelete.map(async (image) => {
      await storage
        .bucket('yourbucketName')
        .file(image)
        .delete();
    });

It would really be better to `await` a `Promise.all()` call with the mapped array. — Amit Beckenstein, Sep 14 '21 at 07:14

score -3 · Answer 4 · answered Apr 05 '16 at 13:49

-3

    var gcloud = require('gcloud')({
    projectId: "sampleProject1"
     });
    var gcs = gcloud.storage();
   var myBucket = gcs.bucket('sampleBucket1');
   var collection = gcs.collection("Add file for delete");  
   collection.insert({'1.png'},{'2.png'});

   collection.delete(function (err, apiResponse) {
   if (err) {
    console.log(err);
    }
   else {
    console.log("Deleted successfully");
   }
   });

answered Apr 05 '16 at 13:49

Prabhat ranjan verma prarav

1
1

The code might work, but please explain a little bit about what it does exactly so that OP can understand more clearly why this code should be used to achieve his goal. Just edit your answer a bit and add some commentary there. – Ortund Apr 05 '16 at 13:57
Use collection like(list,dic,map etc) – Prabhat ranjan verma prarav Apr 05 '16 at 14:47
Where did `gcs.collection` come from? We don't have that method on our API: https://googlecloudplatform.github.io/gcloud-node/#/docs/v0.30.1/storage :) – Stephen Apr 05 '16 at 15:09

Delete multiple objects fron google cloud storage using node.js

4 Answers4

Linked