Automating running Python code using Azure services

Question

Hi everyone on Stackoverflow,

I wrote two python scripts. One script is for picking up local files and sending them to GCS (Google Cloud Storage). Another one is opposite - for taking files from GCS that were uploaded and saving locally.

I want to automate process using Azure.

What would you recommend to use? Azure Function App, Azure Logic App or other services?

* I'm now trying to use Logic App. I made .exe file using pyinstaller and looking for connector in Logic App that will run my program (.exe file). I have trigger in Logic App - "When a file is added or modified", but now I stack when selecting next step (connector)..

Kind regards, Anna

Adding code as requested:

from google.cloud import storage
import os
import glob
import json

    # Finding path to config file that is called "gcs_config.json" in directory C:/
    def find_config(name, path):
        for root, dirs, files in os.walk(path):
            if name in files:
                return os.path.join(root, name)
    
    def upload_files(config_file):
        # Reading 3 Parameters for upload from JSON file
        with open(config_file, "r") as file:
            contents = json.loads(file.read())
            print(contents)
    
        # Setting up login credentials
        os.environ["GOOGLE_APPLICATION_CREDENTIALS"] = contents['login_credentials']
        # The ID of GCS bucket
        bucket_name = contents['bucket_name']
        # Setting path to files
        LOCAL_PATH = contents['folder_from']
    
        for source_file_name in glob.glob(LOCAL_PATH + '/**'):
    
        # For multiple files upload
        # Setting destination folder according to file name 
            if os.path.isfile(source_file_name):
                partitioned_file_name = os.path.split(source_file_name)[-1].partition("-")
                file_type_name = partitioned_file_name[0]
    
                # Setting folder where files will be uploaded
                destination_blob_name = file_type_name + "/" + os.path.split(source_file_name)[-1]
    
                # Setting up required variables for GCS 
                storage_client = storage.Client()
                bucket = storage_client.bucket(bucket_name)
                blob = bucket.blob(destination_blob_name)
    
                # Running upload and printing confirmation message
                blob.upload_from_filename(source_file_name)
                print("File from {} uploaded to {} in bucket {}.".format(
                    source_file_name, destination_blob_name, bucket_name
                ))
    
    config_file = find_config("gcs_config.json", "C:/")
    
    upload_files(config_file)

config.json:

{
"login_credentials": "C:/Users/AS/Downloads/bright-velocity-___-53840b2f9bb4.json",
"bucket_name": "staging.bright-velocity-___.appspot.com",
"folder_from": "C:/Users/AS/Documents/Test2/",
"folder_for_downloaded_files": "C:/Users/AnnaShepilova/Documents/DownloadedFromGCS2/",

"given_date": "",
"given_prefix": ["Customer", "Account"] }

I connected to GCS ok using Python (credentials json and I use "google" library). Now I need to run Python script — AnnaSh, Feb 16 '22 at 14:11
Added code, but I made standalone program from it (.exe) I want to run UploadToGCS.exe using Azure LogicApp — AnnaSh, Feb 16 '22 at 14:27
you talk about azure, so if you have a virtual machine in azure, you can certainly setup croj jobs to automate the process. — Lei Yang, Feb 16 '22 at 14:31

score 0 · Answer 1 · answered Feb 18 '22 at 09:14

Currently, there is no built-in connector in Logic Apps for interacting with Google Cloud Services. however, you can use Google Cloud Storage does provide REST API in your Logic app or Function app.

enter image description here

But my suggestion is you can use the Azure Function to do these things. Because the azure Function can be more flexible to write your own flow to do the task.

Refer to run your .exe file in the Azure function. If you are using Local EXE or using Cloud Environment exe.

Refer here for more information

Automating running Python code using Azure services

1 Answers1