0

We can run a gsutils command to delete objects as follows:

gsutil rm -a gs://bucket/**

Now I would like to only remove certain folders following a wildcard pattern:

gs://bucket/folder/{WILDCARD-A}/folderA/{WILDCARD-B}/folderB

The idea is to delete all files with target folder B.

How can this be achieved with gsutils?

Hector Martinez Rodriguez
  • 277
  • 4
  • 11
WJA
  • 6,676
  • 16
  • 85
  • 152
  • 1
    Have you tried gsutil `rm -r gs://bucket/folder/{WILDCARD-A}/folderA/{WILDCARD-B}/folderB` applying your [wildcat patterns](https://cloud.google.com/storage/docs/gsutil/addlhelp/WildcardNames#efficiency-consideration:-using-wildcards-over-many-objects)? If you want to bulk delete a hundred thousand or more objects, avoid using `gsutil`, as the process takes a long time to complete. – Osvaldo May 24 '22 at 21:28
  • What is the alternative to remove objects? – WJA May 24 '22 at 21:29
  • @JohnAndrews Based on the context of your original question, that specific inquiry will require a new posted question as per Stack Overflow rules. – Hector Martinez Rodriguez May 24 '22 at 22:55
  • 1
    (@OsvaldoLópez: `wildcat`?) – greybeard Jun 08 '22 at 10:33
  • I’m voting to close this question because it's not a programming question - please read the tag wiki of [tag:google-cloud-storage] – pppery Jun 10 '22 at 03:27
  • @pppery Disagree, gsutil is a command line tool, widely used, and certainly programming related. See also https://github.com/GoogleCloudPlatform/gsutil – WJA Jun 10 '22 at 07:20

1 Answers1

1

Try gsutil rm -r gs://bucket/folder/{WILDCARD-A}/folderA/{WILDCARD-B}/folderB applying your wildcard patterns. If you have a large number of objects to remove, use the gsutil -m option, which enables multi-threading/multi-processing. In case you need to bulk delete a hundred thousand or more objects, avoid using gsutil, as the process takes a long time to complete. Instead use the Google Cloud console, which can delete up to several million objects, or Object Lifecycle Management, which can delete any number of objects.

Osvaldo
  • 473
  • 1
  • 12
  • Was this answer useful? – Osvaldo May 30 '22 at 16:46
  • This solved it. Basically I put * for every place in the pattern where a wildcard was needed. There are some notes on the performance, as also outlined in the link you provided. – WJA Jun 10 '22 at 07:22