move or copy files(blob) between storage accounts python (azure function)?

Question

I want to move (or copy then delete) files/blobs between two storage accounts using python (in an azure function). I've used methods like this. However this works for older SDKs, does anyone know a way for new SDK?

Something like this but between two storage accounts rather than containers.

if you use the first link I mentioned, I'll get : `no module name .make_blob_url` and early on in the code `cannot import name 'BlockBlobService'` — mas, Mar 26 '21 at 06:05
Have you referred to https://github.com/Azure/azure-sdk-for-python/blob/master/sdk/storage/azure-storage-blob/samples/blob_samples_common.py#L193? — Jim Xu, Mar 26 '21 at 06:08
yeah, how do you get the link for [source blob](https://github.com/Azure/azure-sdk-for-python/blob/0caa4ba9c6d058872fcbf0bf0db4637ffcb52650/sdk/storage/azure-storage-blob/samples/blob_samples_common.py#L195). here is hard coded — mas, Mar 26 '21 at 06:15
Now. You want to know how get make source blob URL. Right? Let me test it. — Jim Xu, Mar 26 '21 at 06:19
yep, confused how I can get a link of source blob. I know you can get `.url` from this: ```blob_service_client = BlobServiceClient.from_connection_string(connection_string) blob_client = blob_service_client.get_blob_client(container=source_container_name, blob=source_file_name) url = blob_client.url ``` — mas, Mar 26 '21 at 06:26
Let us [continue this discussion in chat](https://chat.stackoverflow.com/rooms/230398/discussion-between-mas-and-jim-xu). — mas, Mar 26 '21 at 06:34

score 1 · Accepted Answer · answered Mar 26 '21 at 07:28

If you want to copy blob across Azure storage account, please refer to the following code

from azure.storage.blob import ResourceTypes, AccountSasPermissions, generate_account_sas, BlobServiceClient
from datetime import datetime, timedelta
source_key = ''
des_key = ''
source_account_name = ''
des_account_name = '23storage23'
# genearte account sas token for source account
sas_token = generate_account_sas(account_name=source_account_name, account_key=source_key,
                                 resource_types=ResourceTypes(
                                     service=True, container=True, object=True),
                                 permission=AccountSasPermissions(read=True),
                                 expiry=datetime.utcnow() + timedelta(hours=1))
source_blob_service_client = BlobServiceClient(
    account_url=f'https://{source_account_name}.blob.core.windows.net/', credential=source_key)
des_blob_service_client = BlobServiceClient(
    account_url=f'https://{des_account_name}.blob.core.windows.net/', credential=des_key)

source_container_client = source_blob_service_client.get_container_client(
    'copy')

source_blob = source_container_client.get_blob_client('Capture.PNG')
source_url = source_blob.url+'?'+sas_token
# copy
des_blob_service_client.get_blob_client(
    'test', source_blob.blob_name).start_copy_from_url(source_url)

Besides, if the access level of the source container is public, we can simplify the code as below

from azure.storage.blob import BlobServiceClient
source_key = ''
des_key = ''
source_account_name = ''
des_account_name = '23storage23'
source_blob_service_client = BlobServiceClient(
    account_url=f'https://{source_account_name}.blob.core.windows.net/', credential=source_key)
des_blob_service_client = BlobServiceClient(
    account_url=f'https://{des_account_name}.blob.core.windows.net/', credential=des_key)

source_container_client = source_blob_service_client.get_container_client(
    'input')

source_blob = source_container_client.get_blob_client('file.json')
source_url = source_blob.url
# copy
des_blob_service_client.get_blob_client(
    'test', source_blob.blob_name).start_copy_from_url(source_url)

For more details, please refer to here

this is using same storage account – Dariusz Krynicki Jan 16 '23 at 16:18 — Dariusz Krynicki, Jan 16 '23 at 16:18

score 1 · Answer 2 · answered Nov 18 '21 at 21:50

I see this answer is pretty good too, so leaving it here. It is from here

import json
import logging
import os

import azure.functions as func
from azure.storage.blob import BlobServiceClient, generate_blob_sas, AccessPolicy, BlobSasPermissions
from azure.core.exceptions import ResourceExistsError
from datetime import datetime, timedelta

def main(event: func.EventGridEvent):
    result = json.dumps({
        'id': event.id,
        'data': event.get_json(),
        'topic': event.topic,
        'subject': event.subject,
        'event_type': event.event_type,
    })

    logging.info('Python EventGrid trigger processed an event: %s', result)

    blob_service_client = BlobServiceClient.from_connection_string(
        os.environ.get('ARCHIVE_STORAGE_CONNECTION_STRING'))

    # Get the URL and extract the name of the file and container
    blob_url = event.get_json().get('url')
    logging.info('blob URL: %s', blob_url)
    blob_name = blob_url.split("/")[-1].split("?")[0]
    container_name = blob_url.split("/")[-2].split("?")[0]
    archived_container_name = container_name + '-' + os.environ.get('AZURE_STORAGE_ARCHIVE_CONTAINER')

    blob_service_client_origin = BlobServiceClient.from_connection_string(os.environ.get('ORIGIN_STORAGE_CONNECTION_STRING'))

    blob_to_copy = blob_service_client_origin.get_blob_client(container=container_name, blob=blob_name)

    sas_token = generate_blob_sas(
        blob_to_copy.account_name,
        blob_to_copy.container_name,
        blob_to_copy.blob_name,
        account_key=blob_service_client_origin.credential.account_key,
        permission=BlobSasPermissions(read=True),
        start=datetime.utcnow() + timedelta(seconds=1),
        expiry=datetime.utcnow() + timedelta(hours=1))

    logging.info('sas token: %s',sas_token)

    archived_container = blob_service_client.get_container_client(archived_container_name)

    # Create new Container
    try:
        archived_container.create_container()
    except ResourceExistsError:
        pass

    copied_blob = blob_service_client.get_blob_client(
        archived_container_name, blob_name)

    blob_to_copy_url = blob_url + '?' + sas_token

    logging.info('blob url: ' + blob_to_copy_url)

    # Start copy
    copied_blob.start_copy_from_url(blob_to_copy_url)

move or copy files(blob) between storage accounts python (azure function)?

2 Answers2

Linked