How to calculate the progress of hashing System.IO.Hashing?

Question

I want to move from the unofficial implementations of hash functions to the official System.IO.Hashing. But I absolutely do not understand how to make a report on the progress of hashing? I intend to use the XxHash3 algorithm, but I think it doesn’t matter.

At the moment, I calculate the hash like this (without displaying progress):

private string GetXxHash3(string filename)
{
    var hashAlgorithm = new XxHash3();
    using (Stream entryStream = System.IO.File.OpenRead(filename))
        hashAlgorithm.Append(entryStream);
    return BitConverter.ToString(hashAlgorithm.GetHashAndReset()).Replace("-", string.Empty);
}

What exactly are you trying to report progress on? `xxhash` is supposed to be "real time" so are you just asking how to use a [`stopwatch`](https://learn.microsoft.com/en-us/dotnet/api/system.diagnostics.stopwatch?view=net-7.0)? — BurnsBA, Mar 15 '23 at 21:55
@BurnsBA, Calculation of the hash of a large file does not occur in real time. — Raf-9600, Mar 16 '23 at 09:38
Does this answer your question? [How to report progress from FileStream](https://stackoverflow.com/questions/57432133/how-to-report-progress-from-filestream) — NineBerry, Mar 16 '23 at 11:41

score 1 · Accepted Answer · answered Mar 16 '23 at 15:56

I found the answer:

string GetXxHash3(string filename, IProgress<long> progress)
{
    var hashAlgorithm = new XxHash3();
    using (Stream entryStream = File.OpenRead(filename))
    {
        byte[] buffer = ArrayPool<byte>.Shared.Rent(4096); // use whatever chunk size you want
        int bytesRead;
        long totalRead = 0;
        while ((bytesRead = entryStream.Read(buffer)) > 0)
        {
            hashAlgorithm.Append(buffer.AsSpan(0, bytesRead));
            totalRead += bytesRead;
            progress.Report(totalRead);
        }
        ArrayPool<byte>.Shared.Return(buffer);
    }
    return Convert.ToHexString(hashAlgorithm.GetHashAndReset());
}

(c) stephentoub

score 0 · Answer 2 · answered Mar 15 '23 at 23:50

I might suggest instead of appending the whole stream to the hashAlgorithm, step over it in chunks and then you can report on the progress of those chunks.

private string GetXxHash3(string filename)

{

    var hashAlgorithm = new XxHash3();
    var buffer = new Span<byte>(new byte[512]);
    using (Stream entryStream = System.IO.File.OpenRead(filename)) {
           while (entryStream.Read(buffer) > 0) {
               hashAlgorithm.Append(buffer);
               // report on progress based on entryStream.Position and the file's total length
            }
    }

    return BitConverter.ToString(hashAlgorithm.GetHashAndReset()).Replace("-", string.Empty);

}

Ah yes, you need to make sure you don't send the whole buffer when you only read a subsection. — Rob G, Mar 19 '23 at 20:05

How to calculate the progress of hashing System.IO.Hashing?

2 Answers2