C# Compare two byte[] arrays containing text as lowercase

Question

When comparing individual byte values from two separate byte[] sources (arrays / pointers), how would one perform a case INSENSITIVE comparison?

I have one very large array of bytes containing the "haystack" of strings I am accessing through pointer and I am comparing it to a "needle" pattern, but currently it's only returning when there is an exact case sensitive match.

Is it possible to create a lookup dictionary containing the upper-to-lower values and use that in the comparing loop or is there a faster way? (performance-wise)

Edit1:

The strings are UTF8 encoded.

The desired behavior would be: return true when comparing either a,a; A,A; or a,A. But since 'A' in UTF8 has a value of 65 and 'a' has a value of 97, I cannot do case-insensitive comparison.

Answer 1

Convert the byte array to a string and then do a case-insensitive comparison. Something like:

bool caseInsensitiveByteArrayComparison(byte[] a, byte[] b) {
    string aString = System.Text.Encoding.UTF8.GetString(a);
    string bString = System.Text.Encoding.UTF8.GetString(b);
    return string.Equals(aString, bString, StringComparison.CurrentCultureIgnoreCase);
}

Code shamelessly stolen from SO. See:

1. How to convert UTF-8 byte[] to string?
2. Is there a C# case insensitive equals operator?

Answer 2

Lowercase and uppercase ACSII and UTF8 code's byte representation have an offset of 32 (or hex20), so you can implement a comparison of x == byte[x] || x == byte[x+32] with x=uppercase char value.

edit:

suppose you really have to deal only with small and capital english letters, you can hack around with bitwise operations to speed things up, as you can handle 8 byte / chars at once, because those only differ by the 3rd most significant bit:

'b' & 0b_1101_1111 == 'B' & 0b_1101_1111

so you could handle the byte array in 8 byte chunks:

void Main()
{
    byte[] a = "ASDADAGF".Select(x => (byte)(x) ).ToArray();
    byte[] b = "asdAdAGF".Select(x => (byte)(x) ).ToArray();
    bitCompared(a,b).Dump();
}

static bool bitCompared( byte[] b1, byte[]b2)
{
    UInt64 a = BitConverter.ToUInt64(b1, 0); //loop over the index
    UInt64 b = BitConverter.ToUInt64(b2, 0);
    UInt64 mask =0b_1101_1111_1101_1111_1101_1111_1101_1111_1101_1111_1101_1111_1101_1111_1101_1111;
    return (a &= mask) == (b &= mask);
}

afaik there are also even more ways to optimize with SIMD and other low level "hacks".....