why I can not upload a large file to azure storage blob though java?

Question

this is my code to upload file to azure

BlobAsyncClient blobAsyncClient = AzureUtils.getBlobAsyncClient(containerName, blobName);
long blockSize = 10 * 1024 * 1024;
ParallelTransferOptions parallelTransferOptions = new ParallelTransferOptions().setBlockSizeLong(blockSize)
            .setMaxConcurrency(10)
            .setProgressReceiver(bytesTransferred -> System.out.println("uploaded:" + bytesTransferred));
Flux<ByteBuffer> data = Flux.just(ByteBuffer.wrap(IOUtils.toByteArray(fileStream)));
blobAsyncClient.upload(data, parallelTransferOptions, true).block();

it's work fine when I upload a file with the size of 181 M, but when I tried to upload a file with the size of 498 M, it was blocked when it was uploaded to 160 M, and never been done.this is the log

What should I do?

It's very grateful if you have any advices.

Are you getting any error message? Please edit your question and include that. — Gaurav Mantri, May 22 '21 at 10:15
What is not working according to logs it uploaded your file. — Vova Bilyachat, May 23 '21 at 23:46
@GauravMantri No, the wired thing is that there is nothing, no info, no warning, no error. It just stop at 'uploaded:167772160' and never update status until I stop the program — Riven, May 24 '21 at 02:57
@Riven can add full example where fileStream is coming from? — Vova Bilyachat, May 24 '21 at 03:53

score 1 · Answer 1 · answered May 25 '21 at 02:34

If you want to upload from local path, please try this code below:

public static void uploadFilesByChunk() {
                String connString = "<conn str>";
                String containerName = "<container name>";
                String blobName = "<your-blob-name>";
                String filePath = "<your-file-path>" + blobName;

                BlobServiceClient client = new BlobServiceClientBuilder().connectionString(connString).buildClient();
                BlobClient blobClient = client.getBlobContainerClient(containerName).getBlobClient(blobName);
                long blockSize = 2 * 1024 * 1024; //2MB
                ParallelTransferOptions parallelTransferOptions = new ParallelTransferOptions()
                                .setBlockSizeLong(blockSize).setMaxConcurrency(2)
                                .setProgressReceiver(new ProgressReceiver() {
                                        @Override
                                        public void reportProgress(long bytesTransferred) {
                                                System.out.println("uploaded:" + bytesTransferred);
                                        }
                                });

                BlobHttpHeaders headers = new BlobHttpHeaders().setContentLanguage("en-US").setContentType("binary");

                blobClient.uploadFromFile(filePath, parallelTransferOptions, headers, null, AccessTier.HOT,
                                new BlobRequestConditions(), Duration.ofMinutes(30));
        }

If you want to upload from a stream, using this:

BlockBlobClient blockBlobClient = blobContainerClient.getBlobClient("myblockblob").getBlockBlobClient();
String dataSample = "samples";
try (ByteArrayInputStream dataStream = new ByteArrayInputStream(dataSample.getBytes())) {
    blockBlobClient.upload(dataStream, dataSample.length());
} catch (IOException e) {
    e.printStackTrace();
}

Refer to :

https://github.com/Azure/azure-sdk-for-java/tree/master/sdk/storage/azure-storage-blob#examples

https://stackoverflow.com/a/65403169/13586071

why I can not upload a large file to azure storage blob though java?

1 Answers1