6

I've combed through at least a dozen different versions of this problem but none of them have so far produced an answer to the one I'm having. I'm attempting to develop a proof-of-concept for retrieving an image from Azure BLOB storage using Azurite and a service SAS with a corresponding storage account key. I'm using the well-known storage account credentials for Azurite as indicated here. There appear to be innumerable ways to do this kind of thing in C# but I opted to loosely follow this example and ended up with something like this:

public Uri GetSharedKeySasUrl()
{
    BlobServiceClient serviceClient = new BlobServiceClient(
        new Uri("http://127.0.0.1:10000/devstoreaccount1/"),
        new Azure.Storage.StorageSharedKeyCredential(
            "devstoreaccount1",
            "Eby8vdM02xNOcqFlqUwJPLlmEtlCDXJ1OUzFT50uSRZ6IFsuFq2UVErCz4I6tq/K1SZFPTOtr/KBHBeksoGMGw=="
        )
    );

    var blobContainer = serviceClient.GetBlobContainerClient("images");

    var blobClient = blobContainer.GetBlobClient("00000-pano.jpg");

    BlobSasBuilder sasBuilder = new BlobSasBuilder()
    {
        BlobContainerName = "images",
        BlobName = "00000-pano.jpg",
        Resource = "b",
        StartsOn = DateTimeOffset.UtcNow,
        ExpiresOn = DateTimeOffset.UtcNow.AddHours(1)
    };

    sasBuilder.SetPermissions(BlobSasPermissions.Read);

    Uri sasUri = blobClient.GenerateSasUri(sasBuilder);
    Debug.WriteLine("SAS URI for blob is: {0}", sasUri);

    return sasUri;
}

This produces a URL that looks as though it should work just fine, but whenever I paste it into a browser and attempt to access it I consistently get the following:

<Error>
    <Code>AuthorizationFailure</Code>
    <Message>Server failed to authenticate the request. Make sure the value of the Authorization header is formed correctly including the signature. RequestId:551902dd-ecb8-4736-9cf1-32406d98c02f Time:2022-04-11T20:18:06.368Z</Message>
</Error>

Conversely, if I generate a SAS signature through the Microsoft Azure Storage Explorer, that URL works perfectly. If I compare this URL to the URL this method generates, both seem to be perfectly well-formatted - I can't see anything that would suggest mine is malformed.

Generated by the method:
http://127.0.0.1:10000/devstoreaccount1/images/00000-pano.jpg?sv=2021-04-10&st=2022-04-11T20%3A17%3A59Z&se=2022-04-11T21%3A17%3A59Z&sr=b&sp=r&sig=bUHI2562NmvvtflOqT5kr0E%2BnZv7Q12PlR%2FGNPmEhL8%3D

Generated in Storage Explorer:
http://127.0.0.1:10000/devstoreaccount1/images/00000-pano.jpg?sv=2018-03-28&st=2022-04-11T20%3A06%3A47Z&se=2022-04-12T20%3A06%3A47Z&sr=b&sp=r&sig=V6N7uWDGgoVx8wirM%2FP1ou2kbg05PB4D%2BG8YQdvS5RU%3D

The only notable difference seems to be the version of the service each one is using - the explorer uses 2018-03-28 while the method uses 2021-04-10. I've tried assigning the version in the BlobSasBuilder but it just gets ignored, which is consistent with Microsoft's own remarks about the property being deprecated and the class always using the latest supported version of the service regardless of what you specify here. I still don't think that explains the issue, though.

Any ideas? Thanks in advance for the help.

EDIT

I remain hopelessly stuck on this. I've made numerous attempts to reformulate the code I'm using to construct the SAS URI in hopes that something might work. Not a single one has worked so far. The methods I've tried:

  • Using the BlobUriBuilder, SasBuilder and StorageSharedKeyCredential to generate the URI
  • Using the BlobServiceClient with a StorageSharedKeyCredential to derive the BlobContainerClient and in turn the BlobClient, and then the SasBuilder to generate the URI
  • Same as above using a connection string with the BlobServiceClient in place of a StorageSharedKeyCredential
  • Same as above using "UseDevelopmentStorage=true" as the connection string with the BlobServiceClient
  • Signing the string manually using HMACSHA256 and a NameValueCollection to construct the query parameters

I've also experimented with running Azurite using different startup options, including azurite --loose and azurite --skipApiVersionCheck. Neither one makes any difference.

Other things I've tried:

  • Updating the version of Azurite I'm using
  • Running Azurite from within Visual Studio as a service dependency

I simply can't understand why this would be such an insurmountable problem. Please help.

Dumas.DED
  • 466
  • 5
  • 17
  • please make sure you have Assigned Storage Blob Data Contributor role application. Navigate to Storage account -> Access Control (IAM) -> Add role assignment – kavyaS Apr 12 '22 at 10:26
  • Hi @dumas.ded I have since answered this re-testing it with 12.12.0 without issues, and for fun tried 12.11.0 without issues as looks like the bug is fixed, can you try it and let me know. so I can update my answer. – Maytham Fahmi May 10 '22 at 20:09
  • btw I pulled a fresh and latest docker image for azurite in the new test, it could also be that fix the issue as well. – Maytham Fahmi May 10 '22 at 20:17

4 Answers4

8

Updated Solution:

I have tested Azure.Storage.Blobs package version 12.11.0, and gives the exact issue OP described in the question.

Changing this to version 12.10.0 solved the problem.

This makes a conclusion that version 12.11.0 must either be a side effect of a breaking change or a bug.

So until Microsoft comes up with a solution or answer, the solution is to use version 12.10.0.


Original answer:

I have tested your code and it works, I do not get the error you have. I guess you either need to reinstall the latest version of Azurite or do it the way I did it and it worked for me.

To ensure the consistency of my work, I used docker to install azurite.

I use Windows 11 and Docker Desktop, more about installation check Microsoft doc.

docker pull mcr.microsoft.com/azure-storage/azurite

Then

docker run -p 10000:10000 -p 10001:10001 -p 10002:10002 -v c:/azurite:/data mcr.microsoft.com/azure-storage/azurite

Now we have docker up and running, I use Microsoft Azure Storage Explorer to create a blob container and upload the image 00000-pano.jpg (it is a flower image).

enter image description here

Now I take your code without changing anything in it and run it in my console application with package reference

<PackageReference Include="Azure.Storage.Blobs" Version="12.10.0" />

And running the code, I get the following URL:

http://127.0.0.1:10000/devstoreaccount1/images/00000-pano.jpg?sv=2020-08-04&st=2022-04-20T10%3A47%3A00Z&se=2022-04-20T11%3A47%3A00Z&sr=b&sp=r&sig=RHHe5NJRjhbOjoTeCONqhrRHjjJCygOa8urcwzOWpeE%3D

enter image description here

I copy and paste the URL into my browser and I get the flower without authentication issues:

enter image description here

The following is my console for docker and logs of what I was doing with the image:

enter image description here

Disclaimer: I have some articles about Azurite and docker, testing Azure code with Azurite, and mocking Azure storage this might add value to your further work. (https://itbackyard.com/tag/azurite/)

Maytham Fahmi
  • 31,138
  • 14
  • 118
  • 137
  • 1
    I followed your example and tested my code in a dedicated console app, but still encountered the same problem. I noticed in your case that you were using version 12.10.0 of Azure.Storage.Blobs, where I was using 12.11.0. I tried installing the earlier version, and it worked. – Dumas.DED Apr 20 '22 at 15:20
  • Ahha I guess there must be a breaking change some where, I will fun try to change my version number and give or a try. If I make any found I will let you know – Maytham Fahmi Apr 20 '22 at 15:40
  • Please do! I'll be very interested to see if you can replicate the same thing I've been experiencing. – Dumas.DED Apr 20 '22 at 15:59
  • 1
    Many thanks, Maytham! I'm so glad it wasn't just me going crazy! – Dumas.DED Apr 21 '22 at 19:14
  • I know the feeling you are welcome when I at some point get more updates regarding this issue I will add the update to the answer. Good wind ☺️ – Maytham Fahmi Apr 21 '22 at 20:09
5

Update

I found out after submitting a bug to the Azure SDK team that the fix for this is actually being handled on the Azurite side, per these previously identified issues. It appears Azurite simply doesn't support SAS versions 2020-10-02 and newer at this time. Worth keeping an eye on this one for those wishing to develop locally using the latest SAS versions.

Original Answer

It was a versioning issue. For whatever reason the latest version of Azure.Storage.Blobs (12.11.0, at the time of writing) absolutely refuses to work with Azurite, but version 12.10.0 works just fine.

It appears there was some kind of breaking change with the new version. I've submitted a bug to the SDK team, just to add some visibility. In the meantime I suppose I'll just be sticking to version 12.10.0.

Dumas.DED
  • 466
  • 5
  • 17
  • This happened for me too but not with Azurite but Azure Compute Emulator installed from the Visual Studio 2022 installer. It worked fine with Azure Storage Emulator from 2019 installer. – dbmuller Apr 21 '22 at 00:27
  • Interesting. Were you using the same version of the Azure SDK in both cases? Which version was it? – Dumas.DED Apr 22 '22 at 13:35
  • I had the same version between both, but I have no idea how to determine which version it is exactly! – dbmuller Apr 24 '22 at 01:56
3

Please check the below points:

please make sure you have Assigned Storage Blob Data Contributor role or storage blob data reader role . Navigate to Storage account -> Access Control (IAM) -> Add role assignment or ask the adimin to assign that role.

Also do check the Access level of the container and change access level to read blob ,if it is private. In my case,i am also admin to my storage account and has read , write and create sas permissions enabled. enter image description here

-Please heck if there is any timezone difference of local computerw(out of sync time stamp).

Also please check if your storage account is firewall enabled.

  • Azure Portal -> Storage Account -> Networking -> Check Allow Access From (All Networks / Selected Networks)
  • If it is "Selected Networks" - It means the storage account is firewall enabled.
  • If the storage account is firewall enabled ,that may be the cause for the error , so pleasecheck your storage is whitelisted to access as by default its ip is 127.0.0.1:10000.
  • Also check if SAS has expired.

Please check with https endpoint : SO reference :add a file to Azurite

Easiest way is to use connection string in app config file

<appSettings>
  <add key="StorageConnectionString" value="UseDevelopmentStorage=true" />
</appSettings>

NOTE: Keep a track similar ongoing Azurite/issues in github.

With connection string :

    var connectionString = "DefaultEndpointsProtocol=https;AccountName=.........";
    BlobServiceClient _blobServiceClient = new BlobServiceClient(connectionString);
   // var blobUri = _blobServiceClient.GenerateAccountSasUri(
   AccountSasPermissions.Read, DateTimeOffset.UtcNow.AddDays(10),
   AccountSasResourceTypes.Object);

   // var sasCredential = new AzureSasCredential(blobUri.Query);
    
    var blobContainerClient = _blobServiceClient.GetBlobContainerClient("imagecont");  //container name
    var blobClient = blobContainerClient.GetBlobClient("computeimage.jpg"); // my image blob name
                                                                     
    var sasBuilder = new BlobSasBuilder()
    {
        BlobContainerName = blobClient.BlobContainerName,
        BlobName = blobClient.Name,
        Resource = "b", // b for blob, c for container
        StartsOn = DateTimeOffset.UtcNow,
        ExpiresOn = DateTimeOffset.UtcNow.AddHours(2),
    };
  
     
    sasBuilder.SetPermissions(BlobSasPermissions.Read); // read permissions
                                                       
    Uri sasUri = blobClient.GenerateSasUri(sasBuilder);
  
    Console.WriteLine("SAS URI for blob is: {0}", sasUri);

With connectionstring and (SAS key)Account key get SAS token of blob :

var connectionString = "DefaultEndpointsProtocol=h.......";
             BlobServiceClient _blobServiceClient = new BlobServiceClient(connectionString);
              var storageAccountName = " ";

var blobContainerClient = _blobServiceClient.GetBlobContainerClient("imagecont");
            var blobClient = blobContainerClient.GetBlobClient("computeimage.jpg"); // blob name


//Here we have used SAS  Key StorageSharedKeyCredential(storageAccountName,Account key);

            StorageSharedKeyCredential sharedKeyCredential = new StorageSharedKeyCredential(storageAccountName, "hmjkxxxxxxX3A==");
            var sasBuilder = new BlobSasBuilder()
            {
                BlobContainerName = blobClient.BlobContainerName,
                BlobName = blobClient.Name,
                Resource = "b", // b for blob
                StartsOn = DateTimeOffset.UtcNow,
                ExpiresOn = DateTimeOffset.UtcNow.AddHours(2),
            };
            sasBuilder.SetPermissions(BlobSasPermissions.Read); // read permissions
            string sasToken = sasBuilder.ToSasQueryParameters(sharedKeyCredential).ToString();
Console.WriteLine("SAS token for blob is: {0}", sasToken);

OUTPUT SAS TOKEN:

enter image description here

Form url https://<storage account name>.blob.core.windows.net/<container>/<blob name ex: computeimage.jpg >?<give sas token here> And use the endpoint to get /retreive the image


OUTPUT: RETRIEVED IMAGE BOTH WAYS (FROM URL AND BY FORMING URL FROM SAS TOKEN)

![enter image description here

REFERENCES:

  1. azure-storage-net/issues(GITHUB)
  2. How to access Azure blob using SAS in C#
  3. How to get a Shared Access Signature on a Blob
kavyaS
  • 8,026
  • 1
  • 7
  • 19
  • Thanks so much for the extremely detailed response. I believe your instructions regarding the Storage Blob Data Contributor role et al pertain to the Azure portal - is there an equivalent in Azurite? I'm not able to find one. – Dumas.DED Apr 13 '22 at 12:42
  • Also I apologize for overlooking this in my original post, I've since edited it for clarity - the URL I've generated in the Storage Explorer works perfectly, while the URL I've generated in code does not work at all. This suggests that the Azurite environment is configured correctly, but there's something wrong with how I'm generating the SAS. I've taken your code example and tried to get that to work in my own environment, but the URL produces the same error. Still stuck, I'm afraid. – Dumas.DED Apr 13 '22 at 12:44
1

I experienced the exact same problem as OP but with slightly different technologies:

  • Azurite 3.16.0 (Docker image)
  • Azure Storage Blob Java SDK 12.15.0
  • @azure/storage-blob JavaScript SDK 12.9.0

In my application, I generate the SAS using the Java SDK and then pass it to a JavaScript app running in the browser which then uploads a file to an Azure Storage blob container.
My SAS generator code is explicitly using BlobServiceVersion.V2021_04_10

My SAS generator code:

BlobContainerClient container = new BlobContainerClientBuilder()
  .credential(azuriteStorageCredential())
  .endpoint(endpoint)
  .containerName(containerName)
  .serviceVersion(BlobServiceVersion.V2021_04_10)
  .buildClient();
BlobClient blob = container.getBlobClient(blobName);
OffsetDateTime expiration = OffsetDateTime
  .now()
  .plusMinutes(30);
BlobSasPermission permissions = new BlobSasPermission()
  .setCreatePermission(true);
BlobServiceSasSignatureValues values = new BlobServiceSasSignatureValues(expiration, permissions)
  .setContentType("video/mp4")
  .setProtocol(SasProtocol.HTTPS_HTTP);
String sas = blob.generateSas(values);
String url = blob.getBlobUrl() + "?" + sas; // the final blob url passed to my javascript app

JavaScript code

const options = {
  blobHTTPHeaders: {
    blobContentType: 'video/mp4'
  }
};
const client = new BlockBlobClient(url);
client.uploadData(file, options);

I too used the well-known storage account credentials for Azurite and kept getting Server failed to authenticate the request. Make sure the value of the Authorization header is formed correctly including the signature and couldn't for the life of me, figure out why Azurite was rejecting the SAS.

I turned on verbose logging in Azurite and noticed the signatures didn't match but couldn't determine why.

I too went through the process of trying a bunch of different ways to do the same thing and then eventually stumbled upon this StackOverflow post and decided to try upgrading the libraries.

I upgraded to:

  • Azurite 3.17.1
  • Azure Storage Blob Java SDK 12.16.0

and amazingly... it finally worked!
So it turned out the problem was either in Azurite or in the Java SDK...

I didn't bother to independently downgrade them to try to figure out which was actually the problem but I figured I would post this here for anyone experiencing a similar issue using these same SDKs + Azurite.

Update: based on this comment, it looks like the bug was in Azurite and was fixed in 3.17.0

bpossolo
  • 849
  • 5
  • 6