Azure Block Blob: "The specified block list is invalid" when committing previously committed blocks

Question

Using the Azure.Storage.Blob .NET SDK v12, I'm trying to append to a block blob. Based on various documentation, it seems the way to do this is to commit a list of previously committed block IDs along with new block IDs. I've made sure that the block IDs are all the same length. But when I try to commit a block ID that has already been committed, I get a "400: The specified block list is invalid" error.

Here is some simplified code which illustrates the problem:

// Create a blob container and a new block blob
string connectionString = @"...";
BlobServiceClient serviceClient = new BlobServiceClient(connectionString);
string containerName = Guid.NewGuid().ToString();
BlobContainerClient containerClient = serviceClient.CreateBlobContainer(containerName);
BlockBlobClient blobClient = containerClient.GetBlockBlobClient("some-blob");

// Stage and commit a block containing some dummy data
byte[] dummyData = new byte[1024];
byte[] firstBlockID = Encoding.UTF8.GetBytes("0");
string firstIDBase64 = Convert.ToBase64String(firstBlockID); // "MA=="
var stageResponse = blobClient.StageBlock(firstIDBase64, new MemoryStream(dummyData));
var responseInfo = stageResponse.GetRawResponse(); // 201: Created
var contentResponse = blobClient.CommitBlockList(new[] { firstIDBase64 });
responseInfo = contentResponse.GetRawResponse(); // 201: Created

// Stage a second block
byte[] secondBlockID = Encoding.UTF8.GetBytes("1");
string secondIDBase64 = Convert.ToBase64String(secondBlockID); // "MQ=="
stageResponse = blobClient.StageBlock(secondIDBase64, new MemoryStream(dummyData));
responseInfo = stageResponse.GetRawResponse(); // 201: Created

// Sanity check: 
// Viewing the block list in the debugger shows both the committed block ID 
// "MA==" and uncommitted block ID "MQ==", as expected.
BlockList blockList = blobClient.GetBlockList(BlockListTypes.All).Value;

// Commit both the previously committed block. and the new uncommitted one.
// This results in the the error:
//    Status: 400(The specified block list is invalid.)
//    ErrorCode: InvalidBlockList
blobClient.CommitBlockList(new[] { firstIDBase64, secondIDBase64 });

Answer 1

Great question! I believe the issue is with the way CommitBlockList method is implemented. I checked the request/response in Fiddler and this is what is being sent to Storage service:

<BlockList>
    <Uncommitted>MA==</Uncommitted>
    <Uncommitted>MQ==</Uncommitted>
</BlockList>

If you notice, even though block with MA== block id has been committed, SDK is still sending it as Uncommitted and that's causing the problem.

Then I looked at the documentation of this method here, and this is what I noticed for base64BlockIds parameter:

Specify the Uncommitted Base64 encoded block IDs to indicate that the blob service should search only the uncommitted block list for the named blocks. If the block is not found in the uncommitted block list, it will not be written as part of the blob, and a RequestFailedException will be thrown.

Because the block with MA== block id is already committed and you're sending it as uncommitted block, Storage Service is throwing the exception.

IMHO, the method implementation is not in compliance with Put Block List REST API operation. SDK should be modified to consider this scenario considering it is entirely possible to do so using REST API. I would recommend opening up an issue on SDK Repository for this.

Answer 2

What happened to you was that you commited block list with the first Id and then you commit again with the first and the second Id.

Like @Gaurav Mantri-AIS said: the first Id was already commited so it throwed you that exception.

In your second commit if you wanted only to add the second block you dont need to put the Id of the first block, you should include only the second block ID.

If you want to commit everything again then you should destroy the blob and do the process all over again.