Write HEX values to file in Windows batch

Question

In Linux we can do

echo -n -e '\x66\x6f\x6f' > test.txt

to write HEX values to a file.

How can this be done simply in Windows batch?

Answer 1

I am assuming you want the ability to write all possible binary bytes in the range \x00 through \xFF. Unfortunately, pure batch does not provide a simple mechanism to do this.

But there are a number of options that are not too difficult.

Undocumented !=ExitCodeASCII! dynamic variable

The !=ExitCodeASCII! dynamic variable reports the ASCII value of the return code of the most recently run external command. But it is limited to ASCII codes from \x20 through \x7E.

I use delayed expansion so that I don't have to worry about poison characters.

The simplest mechanism to return a specific error code is to use cmd /c exit N, where N must be a decimal value. If you want to pass in a hex value, then the value must first be converted to decimal.

Windows batch uses 0xNN notation to specify a hex value. As others have noted, you can use set /a val=0x66 to do the conversion. Another option is to use for /l %%N in (0x66 1 0x66) do ..., the advantage being you don't need to define an intermediate environment variable to hold the value.

@echo off
setlocal enableDelayedExpansion
set "str="
for %%H in (0x66 0x6f 0x6f) do (
  for /l %%N in (%%H 1 %%H) do cmd /c exit %%N
  set "str=!str!!=ExitCodeASCII!"
)
>test.txt echo(!str!

Advantages:

• Pure batch

Disadvantages:

• Must build string one character at a time
• Range limited to 0x20 - 0x7E
  
  

FORFILES

The FORFILES command supports 0xNN syntax, so it can generate most characters. But the string must pass through CMD /C, so it cannot be used to generate \0x00, \0x0A, or \0x0D. (I haven't tested, but I believe all other values work, provided poison characters are appropriately quoted or escaped)

@echo off
forfiles /p "%~dp0." /m "%~nx0" /c "cmd /c echo(0x660x6f0x6f"

Advantages:

• Pure batch
• Can process an entire string in one pass

Disadvantages:

• No native XP support
• No support for 0x00, 0x0A, or 0x0D
• Escaping / quoting of poison characters can be tricky
  
  

CERTUTIL

The CERTUTIL supports a -decodeHex verb that can read hex values and write directly to a file

@echo off
>temp.txt echo(66 6f 6f 0d 0a
certutil -f -decodehex temp.txt test.txt >nul
del temp.txt

Advantages:

• Pure batch
• All possible bytes codes supported
• Absolute control of newlines and carriage returns
• Can process an entire string (or file!) in one pass
• Fast

Disadvantages:

• Requires a temp file
  
  

Hybrid JScript / batch - simple solution

It is very easy to embed and execute JScript within a batch script. And JScript has native ability to interpret many escape sequences, including \xNN. However, the \xNN escape sequences actually map to Unicode code points, so some of the high order byte codes do not map to the correct character values. And the results for high order bytes can vary depending on your machines default character set.

Below I define a :jWrite subroutine that can write lines with embedded escape sequences. Simply change the WriteLine to Write in the JScript code if you want to write strings without the newline characters.

@if (@X)==(@Y) @end /* Harmless hybrid line that begins a JScript comment
:: -------- Batch code --------------
@echo off
call :jWrite "\x66\x6f\x6f" >test.txt
exit /b

:jWrite
cscript.exe //E:JScript //nologo "%~f0" %1
exit /b
:: --------- JScript code -----------*/
WScript.StdOut.WriteLine(eval('"'+WScript.Arguments.Unnamed(0)+'"'));

Advantages:

• Pure script that runs natively on all Windows machines from XP onward
• Very simple JScript code
• Can process entire strings in one pass
• Many other escape sequences available, such as \\, \t, \r, \n, etc.
• Easy to mix ordinary text with escape sequences, but double quote must be \x22

Disadvantages:

• Not all high order bytes give the correct result
• Results are dependent on your machine's default character set. Best if Windows 1252
  
  

Hybrid JScript / batch - more robust, but complex solution

It is not too difficult to write some JScript code to properly interpret all \xNN codes to give the correct byte as long as your machine defaults to Windows-1252. And if your command session's active code page also matches Windows 1252, then you can freely mix in normal text.

@if (@X)==(@Y) @end /* Harmless hybrid line that begins a JScript comment
:: -------- Batch code --------------
@echo off
call :jWrite "\x66\x6f\x6f"
call :jWrite "Hello\nworld\x80"
exit /b

:jWrite
cscript.exe //E:JScript //nologo "%~f0" %1
exit /b
:: --------- JScript code -----------*/
WScript.StdOut.WriteLine(WScript.Arguments.Unnamed(0).replace(
  /\\(\\|b|f|n|r|t|v|x[0-9a-fA-F]{2}|u[0-9a-fA-F]{4})/g,
  function($0,$1) {
    switch ($1.toLowerCase()) {
      case 'x80': return '\u20AC';
      case 'x82': return '\u201A';
      case 'x83': return '\u0192';
      case 'x84': return '\u201E';
      case 'x85': return '\u2026';
      case 'x86': return '\u2020';
      case 'x87': return '\u2021';
      case 'x88': return '\u02C6';
      case 'x89': return '\u2030';
      case 'x8a': return '\u0160';
      case 'x8b': return '\u2039';
      case 'x8c': return '\u0152';
      case 'x8e': return '\u017D';
      case 'x91': return '\u2018';
      case 'x92': return '\u2019';
      case 'x93': return '\u201C';
      case 'x94': return '\u201D';
      case 'x95': return '\u2022';
      case 'x96': return '\u2013';
      case 'x97': return '\u2014';
      case 'x98': return '\u02DC';
      case 'x99': return '\u2122';
      case 'x9a': return '\u0161';
      case 'x9b': return '\u203A';
      case 'x9c': return '\u0153';
      case 'x9d': return '\u009D';
      case 'x9e': return '\u017E';
      case 'x9f': return '\u0178';
      default:    return eval('"'+$0+'"');
    }
  }
));

Advantages:

• Pure script that runs natively on all Windows machines from XP onward
• Can process entire strings in one pass
• All bytes codes from \x00 through \xFF are supported
• Many other escape sequences available, such as \\, \t, \r, \n, etc.
• Easy to mix ordinary text with escape sequences, but double quote must be \x22

Disadvantages:

• Only gives correct results for all bytes if your machine defaults to Windows-1252
• Moderately complex JScript code
  
  

JREPL.BAT - Hybrid JScript/batch regular expression text processing utility

My JREPL.BAT utility was originally designed to perform regular expression search and replace operations on text files. But it has options that allow it to easily write strings with embedded escape sequences, and it can give the correct result for all byte codes no matter what default character set your machine uses.

If your machine defaults to any single byte character set, then you can safely use the following with all possible escape sequences from \x00 through \xFF, and you can freely mix in normal text along with escape sequences.

call jrepl $ "\x66\x6f\x6f" /s "=" /x /o test.txt

The /s "=" option specifies an undefined environment variable as the input source, which is interpreted as an empty string. The first $ argument matches the end of the empty string. The second "\x66\x6f\x6f" argument specifies the replacement value. The /x option enables escape sequences within the replacement string, and the /o test.txt option specifies the output file.

If you want to append to test.txt, then add the /APP option.

If you want \n end-of-line instead of \r\n, (Unix style instead of Windows) then add the /U option

If you don't want any new line terminators, then add the /M option.

Lastly, if your machine does not default to a single byte character set, you can still force the correct result for all escape sequences by specifying a single byte character set like Windows-1252 for the output format. However, if the specified character set does not match your command session's active code page, then only escape sequences are guaranteed to work - some normal text characters may give the wrong result.

call jrepl $ "\x66\x6f\x6f" /s "=" /x /o "test.txt|Windows-1252"

Advantages:

• Pure script that runs on any Windows machine from XP onward
• Can process entire strings in one pass
• All bytes codes from \x00 through \xFF are supported
• Many other escape sequences available, such as \\, \t, \r, \n, etc.
• Easy to mix ordinary text with escape sequences, but double quote must be \x22 or \q
• Default character set agnostic - can always give the correct result
• Once you have JREPL in your arsenal of tools, you will find many uses for it. It is a powerful tool.

Disadvantages:

• Requires download of a 3rd party script

Answer 2

To write "HEX values to file":

set a=.
for /l %a in (0,1,255) do (
set /a dec=%a >nul
call cmd /c exit /b !dec!
set hex=!=exitcode!
set a=!a! !hex:~-2!
)
echo %a:~2% > file.txt

To create a binary file from batch with all the values, see here:
https://www.dostips.com/forum/viewtopic.php?f=3&t=5326

certutil can do both: create a hex file from a binary, and create a binary file from a hex coded file:

certutil -f -decodeHex hex.txt out.bin >nul
certutil -f -encodeHex out.bin out.txt >nul

To write a binary file with given (decoded) hex value bytes, from CMD:

set data="66 6F 6F"
echo Dim File : Set File = CreateObject("Scripting.FileSystemObject").CreateTextFile("test.txt") > tmp.vbs
echo data = %data% >> tmp.vbs
echo data = Split (data) >> tmp.vbs
echo for each x in data >> tmp.vbs
echo File.write chr("&H" ^& x) >> tmp.vbs
echo next >> tmp.vbs
echo File.Close >> tmp.vbs
cscript //nologo tmp.vbs

If you're just after writing printable (from 0x20 through 0x7E) characters, then !=ExitCodeASCII! is a good/faster (batch) solution.

Answer 3

In addition to @dbenham's excellent and comprehensive answer, using PowerShell Core 6.x+ (available on all supported Windows systems, MacOS and Linux at http://github.com/PowerShell/Powershell) would be another way to do it in a batch-file run by cmd.

pwsh.exe -NoLogo -NoProfile -Command Write-Host "`u{64}`u{65}`u{66}" >test.txt

Please note that this works with PowerShell Core. I have yet to find a way to make it work with Windows PowerShell 5.x that is the most recent version as of this date that is delivered as part of Microsoft Windows systems.