Copy directory in blob storage across different storage accounts

Question

I am working on a data transfer service, which copies data in a specific directory of the container to the destination container. This service will be fed with src-container-URL with SAS token and the directory which needs to be copied as inputs. (Assume that destination container URL is static)

I have tried using azcopy shell command using the subprocess module and was able to do that. But is there any other way to do the same using python blob store SDK library without first listing all the blobs in the directory & copying it one by one serially?

Is there any azure cloud service that performs the copy activity on some trigger (API call back)?

Have you referred this thread: https://stackoverflow.com/questions/57651890/how-to-copy-a-file-in-azure-from-a-one-storage-account-to-another-using-python — RamaraoAdapa, Nov 30 '21 at 08:48
@RamaraoAdapa-MT The above thread talks about a single blob. But in my case a directory in a container(may contain multiple blobs) — harish chava, Nov 30 '21 at 09:12

score 1 · Answer 1 · answered Dec 01 '21 at 07:25

I have tested in my environment.

Without listing the blobs in a directory, we cannot copy all the blob files in a directory of a storage account to another storage account.

We can copy the blobs in a directory from one storage account to another storage account using below python code:

from azure.storage.blob import BlobServiceClient, BlobClient
source_connect_str = "source_storage_account_connection_string"
destination_connect_str="destination_storage_account_connection_string"
source_blob_service_client = BlobServiceClient.from_connection_string(source_connect_str)
destination_blob_service_client = BlobServiceClient.from_connection_string(destination_connect_str)
source_container_client = source_blob_service_client.get_container_client("containerName")
my_blobs = source_container_client.list_blobs(name_starts_with="directory/")
for my_blob in my_blobs:
    source_blob = BlobClient(source_blob_service_client.url,container_name="containerName",blob_name=my_blob.name,credential="sas-token-of-source-storage-account")
    destination_blob = destination_blob_service_client.get_blob_client("containerName",my_blob.name)
    destination_blob.start_copy_from_url(source_blob.url)

Is there any azure cloud service that performs the copy activity on some trigger (API call back)?

You can use Azure Logic App in this case to perform the copy activity on a trigger

Copy directory in blob storage across different storage accounts

1 Answers1