FileNotFoundError when trying to access a file in google cloud that exists inside a bucket storage

Question

FileNotFoundError When trying to read/access a file or a folder that exists in the bucket in the google cloud by referencing gs://BUCKET_NAME/FolderName/.

I am using python 3 as the kernel with a jupyter notebook. I have a cluster configured in the google cloud linked to a bucket. When ever I try to read/upload a file I am getting the file not found error

def get_files(bucketName):
    files = [f for f in listdir(localFolder) if 
    isfile(join(localFolder, f))]
    for file in files:
        print("file path:", file)

get_files("agriculture-bucket-gl")

I should be able to access the folder contents or to reference any file that exists inside any folder in the bucket.

Error Message:

FileNotFoundError: [Errno 2] No such file or directory: 'gs://agriculture-bucket-gl/Data sets/'

Some piece of code are missing. How are you getting your file? If you hardcode the path, is there is still the problem? — guillaume blaquiere, Sep 29 '19 at 11:03
Yes I am hardcoding the path and the problem still resides. For example: localFolder = "gs://agriculture-bucket-gl/Data sets/' — SaraAwad, Sep 29 '19 at 13:51
Why are you passing `bucketName` as a parameter, when you never use it? — John Gordon, Sep 29 '19 at 20:17
When you call `open()` on a file named `gs://mybucket/file.txt`, the `open()` function doesn't magically know that the `gs://` prefix means "Go get this file from an Amazon bucket"; it just thinks that is a plain filename, which of course it isn't. — John Gordon, Sep 29 '19 at 20:21
Possible duplicate of [GCS - Read a text file from Google Cloud Storage directly into python](https://stackoverflow.com/questions/48279061/gcs-read-a-text-file-from-google-cloud-storage-directly-into-python) — Dustin Ingram, Sep 30 '19 at 20:34

score 1 · Answer 1 · answered Sep 30 '19 at 09:27

You need to access the bucket using the storage library, to get the file and then get content.

You may find this code template helpful.

from google.cloud import storage

# Instantiates a client
client = storage.Client()

bucket_name = 'your_bucket_name'

bucket = client.get_bucket(bucket_name)

blob = bucket.get_blob('route/to/file.txt')

downloaded_blob = blob.download_as_string()

print(downloaded_blob)

score 1 · Answer 2 · answered Jun 21 '21 at 10:19

To add to the the previous answers, the path in the Error Message: FileNotFoundError: [Errno 2] No such file or directory: 'gs://agriculture-bucket-gl/Data sets/' also contains some issues. I'd try fixing the following:

The folder name "Data sets" has a space. I'd try a name without the space.
There is the / sign is at the end of the path. The path should end without a slash.

score 0 · Answer 3 · answered Mar 05 '20 at 14:08

If you want to access from storage

from google.cloud import storage

bucket_name = 'your_bucket_name'
blob_path = 'storage/path/fileThatYouWantToAccess'

storage_client = storage.Client()
bucket = storage_client.get_bucket(bucket_name)
blob = bucket.blob(blob_path)

#this is optional if you want to download it to tmp folder
blob.download_to_filename('/tmp/fileThatYouWantToAccess')

FileNotFoundError when trying to access a file in google cloud that exists inside a bucket storage

3 Answers3