0

I was trying to download azure blob from python using the azure.storage.blob BlobServiceClient. There are various blobs inside the containers which are big in size, the error happens when the code is trying to download files around 100mb or higher.

The code works fine for small blobs, only the bigger blobs are throwing an error: error message :- enter image description here

Error while trying to download a 700mb+ blob enter image description here

I have also tried download_to_stream, readinto and other methods in the MS documentation, but everything returns with same error.

My code :-

with open(path, "wb") as file:
    data = blobclient.download_blob()
    for stream in data.chunks():
        file.write(stream)
Rumi
  • 44
  • 5

1 Answers1

0

trying to download files around 100mb or higher

I have reproduced in my environment. and got expected results by using below code:

Blob in Storage Account:

enter image description here

from azure.storage.blob import BlobServiceClient, BlobClient, ContainerClient    
blob_service_client = BlobServiceClient.from_connection_string('DefaultEndpointsProtocol=https;AccountNam8zxS3g==;EndpointSuffix=core.windows.net')   
container_name = 'pool'   
container_client = blob_service_client.get_container_client(container_name)   
print(container_client.download_blob("100MB (1).bin").readall())

enter image description here

I have got 100mb file downloaded in my environment. Here i used print(container_client.download_blob("100MB (1).bin").readall()) to show the file content.

References taken from:

RithwikBojja
  • 5,069
  • 2
  • 3
  • 7
  • I see you are using the readall method which is just fine for small files, can you do the same using chunks() ? Because I am using chunks method to download big blobs and that's when I face the error. – Rumi Jan 16 '23 at 12:07
  • I have used readall method for 100mb+ file and it worked fine for me. – RithwikBojja Jan 16 '23 at 12:21
  • Even when i use chunks i dont see any error https://i.imgur.com/ucAKjIK.png, Chunks works fine for me – RithwikBojja Jan 16 '23 at 12:37
  • I tried the same code you provided to download a 700mb blob, and this is the error I'm facing "azure.core.exceptions.ResourceModifiedError: The condition specified using HTTP conditional header(s) is not met." any suggestions that explains the situation? – Rumi Jan 16 '23 at 15:53
  • The issue is explained in [SO-thread](https://stackoverflow.com/questions/65649972/azure-python-download-storage-blob-returns-the-condition-specified-using-http-c). I guess this should help. – RithwikBojja Jan 16 '23 at 15:56
  • 1
    Thankyou so much the thread helped!!! Damnn this problem was so tricky, getting a lock on the blob using a lease that never expires did the trick for me. Thanks once again. Crushed water out of stone, pheww!! Can you please add the thread to your answer? – Rumi Jan 16 '23 at 16:16
  • Sure ill add it. – RithwikBojja Jan 16 '23 at 16:18