how to count number of objects in a bucket?

Question

I try to use the stat command in gsutil like this:

$ gsutil -q stat gs://test

but it resulted in the following exception:

CommandException: The stat command only works with object URLs

Note: I don't want to use the ls command because there are too many files in the bucket.

So, how can I do this?

Mangu · Answer 1 · 2018-07-19T14:21:31.527

2

stat only gives details of objects, not buckets and their total of files.

So, your best shot would be using du in combination with wc:

$ gsutil du gs://your-bucket | wc -l

If you are running on Windows, check either this option or this one for the wc command.

edited Jul 19 '18 at 14:21

answered Jul 19 '18 at 13:14

Mangu

Thank you! But the command line show message: 'wc' is not an internal or external command, nor is it a runnable program or batch file. – yezhishu Jul 19 '18 at 14:02
2

You need to have `wc` installed, it's external to `gsutil` or `gcloud`. It looks, by your message error, that you are running on Windows. Check [this](https://stackoverflow.com/questions/247234/do-you-know-a-similar-program-for-wc-unix-word-count-command-on-windows) or [this](https://superuser.com/questions/959036/what-is-the-windows-equivalent-of-wc-l) – Mangu Jul 19 '18 at 14:09
Yes, I am running on windows. I got it now, thanks! – yezhishu Jul 19 '18 at 14:18
I'm glad it helped you. I will edit the answer to include the Windows bit. If you don't mind, please do accept the answer. – Mangu Jul 19 '18 at 14:20
The question asked how to count the number of objects in a bucket. gsutil du will report the number of bytes. – Mike Schwartz Jul 19 '18 at 16:29
You can also count the number of objects with du and wc. The OP explicitly mentioned not using ls. – Mangu Jul 19 '18 at 17:09
Sorry, I realised that my answer was wrong @MikeSchwartz : it counts the folders, where as yours doesn't. – Mangu Jul 20 '18 at 07:34

score 1 · Answer 2 · edited Jul 20 '18 at 11:06

1

You can count the number of objects using:

gsutil ls gs://your-bucket/** | wc -l

Adding the -l flag will only display the number of objects.

edited Jul 20 '18 at 11:06

Mangu

answered Jul 19 '18 at 16:30

Mike Schwartz

2 Answers2