Converting a hex string to base 64 in PowerShell

Question

I'm trying to replicate the functionality of the following Python snippit in PowerShell:

allowed_mac_separators = [':', '-', '.']
for sep in allowed_mac_separators:
    if sep in mac_address:
        test = codecs.decode(mac_address.replace(sep, ''), 'hex')
        b64_mac_address = codecs.encode(test, 'base64')
        address = codecs.decode(b64_mac_address, 'utf-8').rstrip()

It takes a MAC address, removes the separators, converts it to hex, and then base64. (I did not write the Python function and have no control over it or how it works.)

For example, the MAC address AA:BB:CC:DD:E2:00 would be converted to AABBCCDDE200, then to b'\xaa\xbb\xcc\xdd\xe2\x00', and finally as output b'qrvM3eIA'. I tried doing something like:

$bytes = 'AABBCCDDE200' | Format-Hex
[System.BitConverter]::ToString($bytes);

but that produces MethodException: Cannot find an overload for "ToString" and the argument count: "1". and I'm not really sure what it's looking for. All the examples I've found utilizing that call only have one argument. This works:

[System.Convert]::ToBase64String([System.Text.Encoding]::UTF8.GetBytes('AABBCCDDE200'))

but obviously doesn't convert it to hex first and thus yields the incorrect result. Any help is appreciated.

What about using the `.Bytes` method of `$bytes`? For example running: `[System.Convert]::ToBase64String($bytes.Bytes)`. — Alex_P, Jul 23 '21 at 18:02
MAC addresses are already in hex. The python script looks like it **decodes** the MAC from hex to UTF8, then encodes the bytes as base64 just like your `[System.Convert]` code in powershell. — Cpt.Whale, Jul 23 '21 at 18:13
Also, using `$bytes.Bytes` will fix the error you're seeing, but it's equivalent to the `UTF8.GetBytes` method — Cpt.Whale, Jul 23 '21 at 18:16
Good to know about `.Bytes`, @Cpt.Whale; note that in _Windows PowerShell_ it is the - lossy - `ASCII.GetBytes` method. While this distinction doesn't matter in the case at hand, note that using `.Bytes` would _not_ work in this case, because you'd effectively get byte values reflecting the *ASCII code points* of characters such as `A` - whereas what's needed is the interpretation of these characters as _hex digits_. — mklement0, Jul 23 '21 at 19:38

mklement0 · Accepted Answer · 2021-12-12T16:56:08.333

# Remove everything except word characters from the string.
# In effect, this removes any punctuation ('-', ':', '.')
$sanitizedHexStr = 'AA:BB:CC:DD:E2:00' -replace '\W'

# Convert all hex-digit pairs in the string to an array of bytes.
$bytes = [byte[]] -split ($sanitizedHexStr -replace '..', '0x$& ')

# Get the Base64 encoding of the byte array.
[System.Convert]::ToBase64String($bytes)

For an explanation of the technique used to create the $bytes array, as well as a simpler PowerShell (Core) 7.1+ / .NET 5+ alternative (in short: [System.Convert]::FromHexString('AABBCCDDE200')), see this answer.

As for what you tried:

Format-Hex does not return an array of bytes (directly), its primary purpose is to visualize the input data in hex format for the human observer.

In general, Format-* cmdlets output objects whose sole purpose is to provide formatting instructions to PowerShell's output-formatting system - see this answer. In short: only ever use Format-* cmdlets to format data for display, never for subsequent programmatic processing.

That said, in the particular case of Format-Hex the output objects, which are of type [Microsoft.PowerShell.Commands.ByteCollection], do contain useful data, and do contain the bytes of the transcoded characters of input strings .Bytes property, as Cpt.Whale points out.

However, $bytes = ($sanitizedHexStr | Format-Hex).Bytes would not work in your case, because you'd effectively get byte values reflecting the ASCII code points of characters such as A (see below) - whereas what you need is the interpretation of these characters as hex digits.

But even in general I suggest not relying on Format-Hex for to-byte-array conversions:

On a philosophical note, as stated, the purpose of Format-* cmdlets is to produce for-display output, not data, and it's worth observing this distinction, this exception notwithstanding - the type of the output object could be considered an implementation detail.
Format-Hex converts strings to bytes based on first applying a fixed character transcoding (e.g., you couldn't get the byte representation of a .NET string as-is, based on UTF-16 code units), and that fixed transcoding differs between Windows PowerShell and PowerShell (Core):
- In Windows PowerShell, the .NET string is transcoded to ASCII(!), resulting in the loss of non-ASCII-range characters - they are transcoded to literal ?
- In PowerShell (Core), that problem is avoided by transcoding to UTF-8.

The System.BitConverter.ToString failed, because $bytes in your code wasn't itself a byte array ([byte[]]), only its .Bytes property value was (but didn't contain the values of interest).

That said, you're not looking to reconvert bytes to a string, you're looking to convert the bytes directly to Base64-encoding, as shown above.

good to know. I definitely felt like it should be standard, so I'm glad they updated the API to support it. PS Core is easily the future. Thanks again for the answer. — mikeboharsik, Nov 07 '21 at 19:57

Converting a hex string to base 64 in PowerShell

1 Answers1