Can I upload a stream to Azure blob storage without specifying its length upfront?

Question

I don't know if it's relevant, but I am using Java with the azure-storage-android-0.2.0.aar for the upload.

I can upload files to Microsoft Azure blob storage

CloudBlockBlob blob = container.getBlockBlobReference("filename.ext");
blob.upload(inputStream, n);

where n is the length of the inputStream when it is derived from the file.

Here's my problem: I would like to stream directly, for example from the camera, which apparently isn't possible as Azure requires the length parameter for the upload, which is unknown while still streaming.

Is there a reason why I need to specify the length? (MD5?) And is there a way to upload while the stream is still being produced (which obviously is the idea of an InputStream in Java, the reason why InputStream does not have a length property)?

score 12 · Accepted Answer · answered Jul 07 '14 at 23:31

12

We’ll log feature request to enable uploading from a stream without specifying length. For now, you may want to use the openOutputStream method which has a write method taking a byte[] or an int. An example using the int method is below:

    CloudBlockBlob blockBlob = container.getBlockBlobReference(‘myblob’); // assuming container was already created

    BlobOutputStream blobOutputStream = blockBlob.openOutputStream();
    ByteArrayInputStream inputStream = new ByteArrayInputStream(buffer); // assuming buffer is a byte[] with your data

    int next = inputStream.read();
    while (next != -1) {
          blobOutputStream.write(next);
          next = inputStream.read();
    }

    blobOutputStream.close();

answered Jul 07 '14 at 23:31

Emily Gerner

2,427
16
16

1

@trailmax Hopefully that makes your aforementioned C# code easier. :) If you have any other issues like what you mentioned before, please do post an issue/feature request on our [GitHub page](https://github.com/Azure/azure-storage-net) or here on Stack Overflow. We monitor both regularly and are happy to answer questions and take feedback. – Emily Gerner Jul 08 '14 at 02:05
@EmilyGerner-Microsoft : I was curious whether Blob storage works the same way the service client work. DO I need to add a seperate async task to get the work done. – Terril Thomas Sep 26 '14 at 07:08
@TerrilThomas Could you please clarify? Which storage library are you using (language and version)? Do you have a code snippet? Perhaps open a new question with this information on Stack Overflow or the appropriate GitHub repository so we can better help you. – Emily Gerner Sep 26 '14 at 18:50
@EmilyGerner-Microsoft I have used your demo to upload image. its working fine. But now, after uploading image i need its url to display, how i can do that? I mean how i can get URL of the uploaded image? – Shreyash Mahajan Aug 07 '15 at 12:17
@iDroidExplorer This should be really simple. CloudBlockBlob has a getUri method. Is this what you need? – Emily Gerner Aug 07 '15 at 16:47
Hey, just wanted to comment it would still nice to be able to upload without needing the length to provide parity between the AWS sdk and our own. Is it as simple as just adding a new method, `upload` that takes a single input of a stream, where the function is the code you have provided here? – Dan Ciborowski - MSFT Jan 20 '16 at 22:44
1

Not at all or we'd have done it. :) The length determines how we have to upload. Smaller blobs are done in a single put whereas larger blobs require chunking. If we don't know length we can't make an appropriate decision. – Emily Gerner Jan 20 '16 at 23:13
So the method your giving here should only be used if you know your file is small? – Dan Ciborowski - MSFT Jan 22 '16 at 18:04
What do you mean? The library does chunking for big blobs so using any of these methods works for all sizes. – Emily Gerner Jan 22 '16 at 18:23

score 1 · Answer 2 · answered Jul 23 '21 at 18:43

I did it using BlobAsyncClient below is the code for that:

BlobServiceAsyncClient blobServiceAsyncClient = blobServiceClientBuilder.buildAsyncClient();

BlobContainerAsyncClient blobContainerAsyncClient =
    blobServiceAsyncClient.getBlobContainerAsyncClient("container name");

BlobAsyncClient blobAsyncClient =
    blobContainerAsyncClient.getBlobAsyncClient("blob name");

ParallelTransferOptions parallelTransferOptions = new ParallelTransferOptions()
    .setBlockSizeLong(blockSize).setMaxConcurrency(5)
    .setProgressReceiver(bytesTransferred -> log.info("Uploaded bytes:{}", bytesTransferred));

Flux<ByteBuffer> data = Flux.just(ByteBuffer.wrap(byteArray));

blobAsyncClient.upload(data, parallelTransferOptions, true)
    .doOnError(throwable -> log.error("Error occurred while uploading:{}",
        throwable.getMessage()))
    .doOnSuccess(success -> {
      log.info("Blob is uploaded successfully!!:{}", blobAsyncClient.getBlobUrl());
    })
    .subscribe();

Can I upload a stream to Azure blob storage without specifying its length upfront?

2 Answers2

Linked