I want to upload a file to azure blob using python in blocks. (using-put_block)

Question

I am uploading a file to the azure blob using python. I want to do the same using put_block by splitting the file into small parts then uploading them then combining. so that in case of large files, it does not fail. Can someone help me with this? I tried multiple solutions on StackOverflow. Nothing is working

from  azure.storage.blob import BlockBlobService

def uploadFile():
    accountey="account-key"
    accountName="account-name"
    containerName="container-name"

    blobService =BlockBlobService(account_name=accountName, account_key=accountey )
    blobService.create_container(containerName)
    blobService.create_blob_from_path(containerName, "image1.jpg", "./images/python.jpg")

uploadFile()

I tried this also this is not working:

def upload():
    blob_service.create_container(container_name, None, None, False)
    #blob_service.put_block_blob(container_name, blob_name, '', 'BlockBlob')

    chunk_size = 65536
    block_ids = []
    index = 0
    with open(file_path, 'rb') as f:
        while True:
            data = f.read(chunk_size)
            if data:
                length = len(data)
                block_id = base64.b64encode(bytes(index))
                blob_service.put_block(container_name, blob_name, data, block_id)
                block_ids.append(block_id)
                index += 1
            else:
                break

    resp = blob_service.put_block_list(container_name, blob_name, block_ids)

Please edit your question and include what you have tried (because you mentioned you tried certain things) and what's not working. — Gaurav Mantri, Jul 19 '19 at 13:43
@GauravMantri I added the code which i tried but its not working — Shakir Shakeel, Jul 19 '19 at 18:53
I am guessing that you're getting `Specified blob or block content is not valid` error. — Gaurav Mantri, Jul 19 '19 at 19:35

score 1 · Accepted Answer · answered Jul 19 '19 at 19:57

One problem I see with your code is that you're using your index variable for creating block id. Please note that the block ids must be of same length. In your case, 1st 10 blocks (0 - 9) will have one length and then next 90 blocks (10 - 99) blocks will have different lengths and that will cause your blob upload to fail.

Two possible solutions:

Pad your index variable: You can pad the index variable with appropriate number of zeros so that they are of equal length. For example, you could do something like 0000, 0001....0009, 0010...
Use UUID for creating block ids. Please see this post for more details: How to create a GUID/UUID in Python.

I wrote a blog post about it some time ago that you might find useful: https://gauravmantri.com/2013/05/18/windows-azure-blob-storage-dealing-with-the-specified-blob-or-block-content-is-invalid-error/.

Might be worth noting that the question code is probably not working as is because blob_service.put_block_list expects `BlobBlock` instances to access the list of block_ids to commit... Probably need `block_ids.append(BlobBlock(block_id))` instead. — chaami, Aug 23 '19 at 09:52

I want to upload a file to azure blob using python in blocks. (using-put_block)

1 Answers1