2

I have the following C# code that uses an async function, however the await foreach (...) returns immediately without processing any of the BlobHieararchyItems that I know exist.

I refactored this from a synchronous call to an async call in an attempt to make it use multiple threads to hopefully return quicker. There are nearly a million blobs in the referenced container.

using System;
using System.Collections.Generic;
using Azure.Storage.Blobs;
using Azure.Storage.Blobs.Models;
using Azure.Storage.Blobs.Specialized;
using System.Threading.Tasks;

namespace ListBlobs
{
    class Program
    {
        static void Main(string[] args)
        {
            List<BlobHierarchyItem> items = new List<BlobHierarchyItem>();
            Task _task = GetBlobs(items);
            foreach (var item in items)
            {
                Console.WriteLine($"{item.Blob.Name} - created {item.Blob.Properties.CreatedOn}, size = {item.Blob.Properties.ContentLength}");
            }
        }

        private static async Task GetBlobs(List<BlobHierarchyItem> items) 
        {
            try
            {
                // I've obfuscated the actual connection string here, however I copied it directly from Azure Storage Explorer, where it works correctly.
                var storageConnectionString = "DefaultEndpointsProtocol=....file.core.windows.net/;";
                var _blobServiceClient = new BlobServiceClient(storageConnectionString);

                var container = _blobServiceClient.GetBlobContainerClient("extractor-archive");
                // the prefix here has been copied directly from Azure Storage Explorer, which lists nearly 1,000,000 blobs, so I know the prefix should work.
                // unless I misunderstand the purpose of the prefix
                var blobs = container.GetBlobsByHierarchyAsync(delimiter:"/", prefix: "Prod/entities/GetHOSByDriverId/").AsPages(continuationToken:default, pageSizeHint:50000);
                await foreach (Azure.Page<BlobHierarchyItem> page in blobs) // this foreach exits at the 'in'
                {
                    var continuation = page.ContinuationToken;
                    Console.WriteLine($"{DateTime.Now:u)} - Received {page.Values.Count} items.");
                    foreach (var blob in page.Values)
                    {
                        items.Add(blob);
                    }
                }
            }
            catch (Exception ex)
            {
                Console.WriteLine(ex.ToString());
            };
        }
    }
}

The code never reaches the catch - i.e. there seems to be no exception happening, so I'm assuming I'm using the wrong approach here. Having said that, it seems to match up with the example from Microsoft docs.

I'm using the Azure.Storage.Blobs 12.9.1 package from nuget, which is the current version.

If I modify the main() method to be async (I'm using VS2019 so it should work), I get:

Program does not contain a static 'Main' method suitable for an entry point    ListBlobs

This is the modified main() method:

        static async void Main(string[] args)
        {
            List<BlobHierarchyItem> items = new List<BlobHierarchyItem>();
            await GetBlobs(items);
            foreach (var item in items)
            {
                Console.WriteLine($"{item.Blob.Name} - created {item.Blob.Properties.CreatedOn}, size = {item.Blob.Properties.ContentLength}");
            }
        }
Hannah Vernon
  • 3,367
  • 1
  • 26
  • 48
  • 1
    Did you forget to `await` `GetBlobs`? https://stackoverflow.com/questions/9208921/cant-specify-the-async-modifier-on-the-main-method-of-a-console-app – mjwills Jul 09 '21 at 00:35
  • if I add `static async void Main(string[] args)` I get an error `Program does not contain a static "main" method suitable for an entry point` – Hannah Vernon Jul 09 '21 at 00:40
  • `async Task Main(...)`. See https://stackoverflow.com/a/45908959. Do make sure you haven't accidentally targeted a "legacy" version of the C# language/framework. – Peter Duniho Jul 09 '21 at 00:45
  • _"I VtC'd it as a dupe of your linked question"_ -- I guess that's fine. Technically the original issue wasn't the `Main()` method declaration, but the lack of awaiting the task returned by `GetBlobs()`. But, sure...to fix that issue (essentially a typo), you'll need the necessary change to the method signature too. – Peter Duniho Jul 09 '21 at 00:57
  • @PeterDuniho - so the thread was aborted by the lack of an await? Is that really what happened? – Hannah Vernon Jul 09 '21 at 01:56
  • 2
    Your program exited, because there was nothing in it to tell it to wait for the task to finish. It depends on how the task is implemented, but in the vast majority of cases there either is not a thread at all, or the thread is a background thread. Only foreground threads keep a process alive. So if you don't explicitly wait for a task to finish, the process can exit right away, abandoning whatever work was started. – Peter Duniho Jul 09 '21 at 01:58
  • Let us [continue this discussion in chat](https://chat.stackoverflow.com/rooms/234678/discussion-between-hannah-vernon-and-peter-duniho). – Hannah Vernon Jul 09 '21 at 02:10

0 Answers0