Python: How to move or copy Azure Blob from one container to another

Question

I am using Microsoft Azure SDK for Python in project. I want to move or copy Blob from one container to another. for exmaple

https://demostorage.blob.core.windows.net/image-container/pretty.jpg

I want to move this blob to

https://demostorage.blob.core.windows.net/demo-container/

I have found following method in python SDK but unable to understand it.

def copy_blob(self, container_name, blob_name,...):

How can I do this? Thank you

score 23 · Accepted Answer · edited Oct 22 '18 at 13:50

23

I have done in this way.

from azure.storage.blob import BlobService

def copy_azure_files(self):

        blob_service = BlobService(account_name='account_name', account_key='account_key')
        blob_name = 'pretty.jpg'
        copy_from_container = 'image-container'
        copy_to_container = 'demo-container'

        blob_url = blob_service.make_blob_url(copy_from_container, blob_name)
        # blob_url:https://demostorage.blob.core.windows.net/image-container/pretty.jpg

        blob_service.copy_blob(copy_to_container, blob_name, blob_url)

        #for move the file use this line
        blob_service.delete_blob(copy_from_container, blob_name)

I have not found any Blob Move method yet. So I have used the copy method and then execute Blob function.

This is my solution. If you have better way to handle all this please share with me.

Note: I have not used any custom method all these methods are included in SDK.

edited Oct 22 '18 at 13:50

pietroppeter

1,433
13
30

answered Sep 10 '15 at 13:00

Shoaib Ijaz

5,347
12
56
84

3

Move is not natively supported in Azure Storage. You can accomplish move operation by performing copy operation, wait for the copy to finish (very important because copy is async operation) and then performing the delete operation on the source blob. – Gaurav Mantri Sep 10 '15 at 13:23
3

Think you meant to write `blob_url = blob_service.make_blob_url(copy_from_container,blob_name)` & `blob_service.delete_blob(copy_from_container, blob_name)` – Alex S Sep 10 '15 at 13:32
2

Should the initial import (`from azure.storage.blob import BlobService`) be changed to `from azure.storage.blob.baseblobservice import BaseBlobService`? I cannot find any reference of `BlobService` object in [Azure SDK for python](https://azure-storage.readthedocs.io/ref/azure.storage.blob.baseblobservice.html) so I would assume they refactored it to `BaseBlobService`... – pietroppeter Oct 22 '18 at 13:31
2

I think you need to import this `from azure.storage.blob import BlockBlobService` – Amir Imani May 10 '19 at 16:51
The copy operation on blob service messes up with the metadata keys case. so beware if your blob metadata keys have camel case or something else because it will be converted to lower case by copy operation. – Achilles Jun 12 '20 at 14:26

score 15 · Answer 2 · answered Jun 16 '20 at 05:57

15

With the current version of azure-storage-blob (at the moment v12.3.2) you will get an ImportError:

cannot import name 'BlockBlobService' from 'azure.storage.blob'

This code is working in my case:

from azure.storage.blob import BlobServiceClient

# Azure
# Get this from Settings/Access keys in your Storage account on Azure portal
account_name = "YOUR_AZURE_ACCOUNT_NAME"
connection_string = "YOUR_AZURE_CONNECTION_STRING"

# Source
source_container_name = "sourcecontainer"
source_file_path = "soure.jpg"
blob_service_client = BlobServiceClient.from_connection_string(connection_string)
source_blob = (f"https://{account_name}.blob.core.windows.net/{source_container_name}/{source_file_path}")

# Target
target_container_name = "targetcontainer"
target_file_path = "target.jpg"
copied_blob = blob_service_client.get_blob_client(target_container_name, target_file_path)
copied_blob.start_copy_from_url(source_blob)

# If you would like to delete the source file
remove_blob = blob_service_client.get_blob_client(source_container_name, source_file_path)
remove_blob.delete_blob()

answered Jun 16 '20 at 05:57

gailmichi

181
1
4

I am getting "ImportError: cannot import name 'BlobServiceClient' from 'azure.storage.blob'" upon trying to use it in Azure Function. My version of azure-storage-blob is 12.3.2 Any idea how can I avoid this? – RB17 Jun 19 '20 at 01:17
Maybe there are some problems with your python environment? Are you using a virtual enviroment? Open your function in VSCode and try 'pip freeze' in the VSCode Terminal. You will see the used library versions. Check if there is the current version. – gailmichi Jun 22 '20 at 10:56
is there a way to copy from one storage account to another? – mas Mar 25 '21 at 23:39
1

You can get the `source_blob` url natively as well using `source_blob_client = blob_service_client.get_blob_client(source_container_name, source_file_path)` and then the `BlobClient` has a property "url" that you can pass in place of `source_blob` as `source_blob_client.url` – deko Sep 29 '22 at 17:21

openwonk · Answer 3 · 2019-07-23T18:45:05.610

5

Using the most current Azure Blob Storage SDK.

from azure.storage.blob import BlockBlobService

account_name = "demostorage"
account_key = "lkjASDRwelriJfou3lkjksdfjLj349u9LJfasdjfs/dlkjfjLKSjdfi8ulksjdfAlkjsdfkL762FDSDFSDAfju==" 

source_container_name = "image-container"
source_file_path = "pretty.jpg"

target_container_name = "demo-container"
target_file_path = "pretty_copy.jpg"

service = BlockBlobService(account_name, account_key)

service.copy_blob(
    target_container_name, 
    target_file_path, 
    f"https://{account_name}.blob.core.windows.net/{source_container_name}/{source_file_path}",
)

edited Jul 23 '19 at 18:45

answered Jul 23 '19 at 18:35

openwonk

14,023
7
43
39

is there any way to do it in bulk? I would like to copy blobs in bulk from one folder to folder. – shary.sharath Feb 12 '21 at 13:33

score 3 · Answer 4 · answered Oct 09 '20 at 18:24

sdk version 12.0.0b, they change the copy_blob to start_copy_from_url. You can see it here about the method

Here is a snapshot on how you can do the moving mechanism without downloading the data.

    from azure.storage.blob import BlobServiceClient

    blob_service_client = BlobServiceClient.from_connection_string(
            os.getenv("AZURE_STORAGE_CONNECTION_STRING")
        ).get_container_client(self.container)

    # azure storage sdk does not have _move method
    source_blob = blob_service_client.get_blob_client(source_blob_path)
    dest_blob = blob_service_client.get_blob_client(dest_blob_path)

    dest_blob.start_copy_from_url(source_blob.url, requires_sync=True)
    copy_properties = dest_blob.get_blob_properties().copy

    if copy_properties.status != "success":
        dest_blob.abort_copy(copy_properties.id)
        raise Exception(
            f"Unable to copy blob %s with status %s"
            % (source_blob_path, copy_properties.status)
        )
    source_blob.delete_blob()

Python: How to move or copy Azure Blob from one container to another

4 Answers4

Linked