I have the following C# code that uses an async function, however the await foreach (...)
returns immediately without processing any of the BlobHieararchyItems
that I know exist.
I refactored this from a synchronous call to an async call in an attempt to make it use multiple threads to hopefully return quicker. There are nearly a million blobs in the referenced container.
using System;
using System.Collections.Generic;
using Azure.Storage.Blobs;
using Azure.Storage.Blobs.Models;
using Azure.Storage.Blobs.Specialized;
using System.Threading.Tasks;
namespace ListBlobs
{
class Program
{
static void Main(string[] args)
{
List<BlobHierarchyItem> items = new List<BlobHierarchyItem>();
Task _task = GetBlobs(items);
foreach (var item in items)
{
Console.WriteLine($"{item.Blob.Name} - created {item.Blob.Properties.CreatedOn}, size = {item.Blob.Properties.ContentLength}");
}
}
private static async Task GetBlobs(List<BlobHierarchyItem> items)
{
try
{
// I've obfuscated the actual connection string here, however I copied it directly from Azure Storage Explorer, where it works correctly.
var storageConnectionString = "DefaultEndpointsProtocol=....file.core.windows.net/;";
var _blobServiceClient = new BlobServiceClient(storageConnectionString);
var container = _blobServiceClient.GetBlobContainerClient("extractor-archive");
// the prefix here has been copied directly from Azure Storage Explorer, which lists nearly 1,000,000 blobs, so I know the prefix should work.
// unless I misunderstand the purpose of the prefix
var blobs = container.GetBlobsByHierarchyAsync(delimiter:"/", prefix: "Prod/entities/GetHOSByDriverId/").AsPages(continuationToken:default, pageSizeHint:50000);
await foreach (Azure.Page<BlobHierarchyItem> page in blobs) // this foreach exits at the 'in'
{
var continuation = page.ContinuationToken;
Console.WriteLine($"{DateTime.Now:u)} - Received {page.Values.Count} items.");
foreach (var blob in page.Values)
{
items.Add(blob);
}
}
}
catch (Exception ex)
{
Console.WriteLine(ex.ToString());
};
}
}
}
The code never reaches the catch
- i.e. there seems to be no exception happening, so I'm assuming I'm using the wrong approach here. Having said that, it seems to match up with the example from Microsoft docs.
I'm using the Azure.Storage.Blobs 12.9.1 package from nuget, which is the current version.
If I modify the main()
method to be async (I'm using VS2019 so it should work), I get:
Program does not contain a static 'Main' method suitable for an entry point ListBlobs
This is the modified main()
method:
static async void Main(string[] args)
{
List<BlobHierarchyItem> items = new List<BlobHierarchyItem>();
await GetBlobs(items);
foreach (var item in items)
{
Console.WriteLine($"{item.Blob.Name} - created {item.Blob.Properties.CreatedOn}, size = {item.Blob.Properties.ContentLength}");
}
}