0

I have a CSV file staged daily in a Firebase storage bucket. I want to download this updated CSV file to my local machine on daily basis.

Using the following Python code, I am able to generate a link where I can click and the download the file. However, I am wondering if I can download it without any need of clicking the link generated.

import datetime
import firebase_admin
from firebase_admin import credentials
from firebase_admin import storage

### Fetch the service account key JSON file contents
cred = credentials.Certificate("SDK_key.json")

### Initialize the app with a service account, granting admin privileges
app = firebase_admin.initialize_app(cred, {'storageBucket': 'XYZ.appspot.com',}, name='storage')

bucket = storage.bucket(app=app)

### Address of file to downloaded
blob = bucket.blob("test.csv")

### Generate a link and clicking on the link downloads the file
print(blob.generate_signed_url(datetime.timedelta(seconds=300), method='GET'))
ChrisGPT was on strike
  • 127,765
  • 105
  • 273
  • 257
Harpreet Singh
  • 1
  • I found solution to my problem through selenium automation to open the generated link dynamically and downloads the file on my local machine. I can read it in python from local machine and proceed For .json or .text files, we can again use selenium in python along with BeautifulSoup to directly copy text/json data in a variable. Refer to https://stackoverflow.com/questions/26661808/how-to-grab-from-json-in-selenium-python – Harpreet Singh Jul 21 '20 at 08:48

2 Answers2

0

I think you'll find your answer here.

@James Daniels points out that getSignedURL() is the key component for retrieving your designated file, but you can do so by using the Firebase Admin SDK (acts as a wrapper for Google's Cloud Storage SDK).

Alex Douglas
  • 191
  • 7
  • Last command in above given python code is generating the download link for me. That's not a problem. I want to download the file from this link directly through python itself without any manual intervention. As of now, I have to open this URL manually and then file gets downloaded – Harpreet Singh Jul 16 '20 at 04:07
0

Sample Code

  • You don't need firebase-admin SDK to download a publically visible file
  • Make sure you make the folder location visible, other wise you need to attach the access token, Which can only be fetched via client sdk.

Sample Firebase Security Rule

rules_version = '2';
service firebase.storage {
  match /b/{bucket}/o {
     // Explicitly define rules for the 'publicpath' pattern
    match /publicpath/{allPaths}{
      allow  write: if request.auth != null;    // Only auth users can write
      allow  read: if request.auth == null;   //Publically Readable
    }
    // This will be defined for everything else
    match /{allPaths=**} {
      allow  write: if request.auth != null;  // Only auth users can write
      allow  read: if request.auth != null;  // Only auth users can read
    }
  }
}

Python code to save file as CSV

Use pandas, If not installed use following code in cmd to install the package

pip install pandas

Sample python code

import pandas as pd

def download_File(url):
    response = requests.get(url=url)
    path="downfile.csv"
    print(response)
    if response.status_code == 200:
        print("File Downloaded")

        df = pd.read_csv(url)   
        df.to_csv('outfile.csv')

        # with open('down.csv', 'w') as f:
        #     writer = csv.writer(f)
        #     for line in response.iter_lines():
        #         writer.writerow(line.decode('utf-8').split(','))
    else:
        print("Something went wrong")

def main:
    storageBucket='XYZ.appspot.com'
    # publicpath/test.csv should be changed to 
    # publicpath%2Ftest.csv

    bucket_path="publicpath%2Ftest.csv"
    firebase_storageURL = 'https://firebasestorage.googleapis.com/v0/b/{}/o/{}?alt=media'.format(storageBucket, bucket_path)

    download_File(firebase_storageURL)


#Main program
if __name__ == '__main__':
    main()

Remember to update the firebase storage rules
Dr.House
  • 784
  • 5
  • 11