Converting HEX to (extended) ASCII 0-255 with NO UTF for DOS C#

Question

Firstly, I don't want to be 'That Guy' when I ask this long question, even though I know it's been asked plenty of times in different way, but I'm having significant problems in getting a date format to store in a string correctly.

Some minor background.

I am using the DOS FileTime Date format that needs to be stored in an 8 character HEX format - as seen here: https://doubleblak.com/blogPosts.php?id=7

In short, the time and date are captured, then arranged in binary bits, and then converted to HEX.

What I need is to be able to do now, is store those HEX Values as a string, and be able to pass them to tagLib sharp to write a custom APE tag in an MP3 file. Easier said than done...

Writing the custom tags is easy, as it's basically just a matter of this:

TagLib.File file = TagLib.File.Create(filename);
TagLib.Ape.Tag ape_tag = (TagLib.Ape.Tag)file.GetTag(TagLib.TagTypes.Ape, true);

// Write - for my example
/* declarations: 
    public void SetValue(string key, string value);
    public void SetValue(string key, uint number, uint count);
    public void SetValue(string key, string[] value);
*/
ape_tag.SetValue("XLastPlayed", history );

So, on to the actual problem:

After the conversion of the date to the correct HEX Values, I get the following result:

928C9D51

However, to make this work and store it correctly, I need to convert it to ASCII values, so that it can then be stored by TagLibSharp.

If I convert this to ASCII, then I get the following: (which is wrong), as it should only be 4 ASCII characters long - even if they are unprintable, or sit in the > 127 character range.

"\u0092\u008c\u009dQ"

You can see in this image the extra HEX values that have been stored, which is incorrect.

This is a sample of the code of I've been trying to use, (in various forms) to get this to work.

string FirstHistory = "7D8C9D51";
 
String test1 = "";

for (int i = 0; i < FirstHistory.Length; i += 2)
{
    string hs = FirstHistory.Substring(i, 2);
     
    var enc = Encoding.GetEncoding("iso-8859-1"); //.ASCII;// .GetEncoding(437); 
    var bytes1 = enc.GetBytes(string.Format("{0:x1}", Convert.ToChar(Convert.ToUInt16(hs, 16)))); 
    string unicodeString = enc.GetString(bytes1); 
    Console.WriteLine(unicodeString);  
    test1 = test1 + unicodeString;
}

// needs to be "00 00 00 21" for the standard date array for this file format.
byte[] bytesArray = { 0, 0, 0, 33 }; // A byte array containing non-printable characters
     
string s1 = "";  
string history = ""; 

// Basically what the history will look like
// "???!???!???!???!???!???!???!???!???!???!???!???!???!???!???!???!???!"

for (int i =0; i < 18; i++)
{            
    if(i==0) {
        history = test1; // Write the first value.
    } 

    s1 = Encoding.UTF8.GetString(bytesArray); // encoding on this string won't effect the array date values
    history = history + s1;     
}

ape_tag.SetValue("XLastPlayed", history );

I am aware there are multiple encodings, and I've basically tried all that I can, and have read things, but I'm not getting anywhere.

Sometimes I think I've got it, but then when I look at the file I'm saving, it slips in a "C2" HEX value, when it shouldn't, and this is the unicode breaking everything. I've included an image of what it should be without these C2 Hex Values, and you can actually see the DOS Time and Date time appear correctly in the HxD Hex viewer.

I've tried various encodings such as 437, ios-8859-1, ASCII, and different methods such as using string builder, char, bytes, etc. and sometimes I get a date and time stamp where the values are correct, where the HEX values don't exceed in to the extended ASCII ranges, but then I run it again, and I'm back to square 1. It ALWAYS inserts those extended values as UTF8 entries and breaks regardless of what I do.

I'm sure there's not a bug in VS, but I'm running Microsoft Visual Studio Community 2019, Version 16.8.2 if that adds to the case.

I can not seem to find a way around this. Does anyone have any thoughts on this?

Thanks in advance.

*** UPDATE ***

This the update thanks to @xanatos

public static byte[] ConvertHexStringToByteArray(string str)
{
    Dictionary<string, byte> hexindex = new Dictionary<string, byte>();
    for (int i = 0; i <= 255; i++)
        hexindex.Add(i.ToString("X2"), (byte)i);
    List<byte> hexres = new List<byte>();
    for (int i = 0; i < str.Length; i += 2)
        hexres.Add(hexindex[str.Substring(i, 2)]);
    return hexres.ToArray();
}

string FirstHistory = "7D8C9D51";
 
string s1 = "";  
string history = ""; 
byte[] bytes = { 0, 0, 33, 0 }; // A byte array contains non-ASCII (or non-readable) characters
for (int i =0; i < 18; i++)
{   
    s1 = Encoding.UTF8.GetString(bytes); // ???
    history = history + s1;
}

var theArray_SO = ConvertHexStringToByteArray(FirstHistory);
ape_tag.SetItem(new TagLib.Ape.Item("XLastPlayed", (new TagLib.ByteVector(theArray_SO)) + history));

*** UPDATE 2 - 30th Jan 2021 ***

After editing other values and resaving them, I ran into some trouble. It seems that there could be data corruption with TagLib and custom APE tags, specifically for this ByteVector data. If you just use the save method for editing other custom values then it's not a problem, but if you have custom values with these values with ByteVector values, you will most probably run in to trouble. This is what I still used for saving the files.

TagLib.File file = TagLib.File.Create(filename);
// changes
file.save();

However, to overcome this data corruption, I read (searched) the file first as a FileStream to locate the value I needed, and then put the values of 72 bytes after the found value to a new byte array, then save it back to the file.

I discovered that reading ByteVector data in through a string failed spectacularly and had results all over the place.

TagLib.Ape.Item item_Duration = ape_tag.GetItem("XLastScheduled");

While this can probably be rewritten a thousand ways, here's my code that works.

int foundlocation = 0;
int loop1 = 0;                
byte[] sevenItems = new byte[80] { 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 };
string match = "XLastScheduled";
byte[] matchBytes = Encoding.ASCII.GetBytes(match);
{
    using (var fs = new FileStream(filename, FileMode.Open))
    {
        int i = 0;
        int readByte;
        while ((readByte = fs.ReadByte()) != -1)
        {

            if (foundlocation == 0)
            {
                if (matchBytes[i] == readByte)
                {
                    i++;
                }
                else
                {
                    i = 0;
                }
            }

            if (i == matchBytes.Length)
            {
                //Console.WriteLine("It found between {0} and {1}.", fs.Position - matchBytes.Length, fs.Position);
                // set to true.
                foundlocation = 1;
            }

            if (foundlocation==1)
            {
                //if (loop1 > 1)
                {
                    // Start adding it at 2 bytes after it's found.
                    sevenItems[loop1] = (byte)readByte;
                }

                loop1++;

                if(loop1 > 79) 
                {

                    fs.Close();
                    Console.WriteLine("Found the XLastScheduled data");
                    // 72/4 = 18 date/times
                    break;
                }
            }
            // Then, I can save those values back as a vector byte array, instead of a string - hopefully...

        }
        fs.Close();
    }
}

byte[] dst = new byte[sevenItems.Length - 8];
Array.Copy(sevenItems, 2, dst, 0, dst.Length);
 




TagLib.File file = TagLib.File.Create(filename); 
// Get the APEv2 tag if it exists.
TagLib.Ape.Tag ape_tag = (TagLib.Ape.Tag)file.GetTag(TagLib.TagTypes.Ape, true);




// Save the new byteVector.
ape_tag.SetItem(new TagLib.Ape.Item("XLastScheduled", (new TagLib.ByteVector(dst))));
Console.WriteLine("XLastScheduled: set"  );

"as it should only be 4 ASCII characters long" - the string `"\u0092\u008c\u009dQ"` is 4 characters long: e.g. '\u0092' is a single character (with code 0x92) — Klaus Gütter, Dec 29 '20 at 07:47
Yes, but it doesn't store that way to file. The "\u0092" store them as 4 bytes, not 2 bytes... — vr_driver, Dec 29 '20 at 07:58
I don't know the TaglibSharp library and how you can pass in binary tag values there, but maybe this can be of help: https://stackoverflow.com/questions/34507982/adding-custom-tag-using-taglib-sharp-library — Klaus Gütter, Dec 29 '20 at 08:00
@KlausGütter, yeah, this is essentially what I was doing. Thanks for your suggestions though. — vr_driver, Dec 29 '20 at 12:31

xanatos · Accepted Answer · 2020-12-29T09:17:42.370

There is another method for binary data:

var bytes = new byte[4] { 0xFF, 0, 0, 0xFF };
ape_tag.SetItem(new TagLib.Ape.Item("XLastPlayed", new ByteVector(bytes)));

Unclear if you need methods to convert from/to DOS FileTime:

public static uint ToDosFileTimeDate(DateTime dt)
{
    ushort date = (ushort)(((dt.Year - 1980) << 9) | (dt.Month << 5) | (dt.Day));
    ushort time = (ushort)((dt.Hour << 11) | (dt.Minute << 5) | (dt.Second >> 1));

    uint dateTime = ((uint)date << 16) | time;
    return dateTime;
}

public static DateTime FromDosFileTimeDate(uint ui)
{
    ushort date = (ushort)(ui >> 16);
    ushort time = (ushort)(ui & 0xFFFF);

    var year = (date >> 9) + 1980;
    var month = (date >> 5) & 0xF;
    var day = date & 0x1F;

    var hour = time >> 11;
    var minute = (time >> 5) & 0x3F;
    var second = (time & 0x1F) << 1;

    return new DateTime(year, month, day, hour, minute, second, DateTimeKind.Local);
}

and to convert the uint to byte[4] arrays there are

uint ui = BitConverter.ToUInt32(bytes);

and

byte[] bytes = BitConverter.GetBytes(ui);

You are a genius. Directing me to new TagLib.ByteVector(bytes) was the key. Thank you so much. — vr_driver, Dec 29 '20 at 12:30

Converting HEX to (extended) ASCII 0-255 with NO UTF for DOS C#

1 Answers1