Use PowerShell to find and replace hex values in binary files

Question

UPDATE: I got a working script to accomplish the task. I needed to batch process a bunch of files so it accepts a csv file formatted as FileName,OriginalHEX,CorrectedHEX. It's very slow even after limiting the search to the first 512 bytes. It could probably be written better and made faster. Thanks for the help.

UPDATE 2: Revised the search method to be faster but it's nowhere near as fast as a dedicated hex editor. Be aware that it's memory intensive. peaks around 32X the size of the file in RAM. 10MB=320MB of RAM. 100MB=3.2GB of RAM. I don't recommend for big files. It also saves to a new file instead of overwriting. Original file renamed as File.ext_original@date-time.

Import-CSV $PSScriptRoot\HEXCorrection.csv | ForEach-Object {
  $File = $_.'FileName'
  $Find = $_.'OriginalHEX'
  $Replace = $_.'CorrectedHEX'


  IF (([System.IO.File]::Exists("$PSScriptRoot\$File"))) {
    $Target = (Get-ChildItem -Path $PSScriptRoot\$File)
  } ELSE {
    Write-Host $File "- File Not Found`n" -ForegroundColor 'Red'
    RETURN
  }

  Write-Host "File:   "$Target.Name`n"Find:    "$Find`n"Replace: "$Replace
  $TargetLWT = $Target.LastWriteTime
  $TargetCT = $Target.CreationTime
  IF ($Target.IsReadOnly) {
    Write-Host $Target.Name "- Is Read-Only`n" -ForegroundColor 'Red'
    RETURN
  }
  $FindLen = $Find.Length
  $ReplaceLen = $Replace.Length
  $TargetLen = (1..$Target.Length)

  IF (!($FindLen %2 -eq 0) -OR !($ReplaceLen %2 -eq 0) -OR 
  [String]::IsNullOrEmpty($FindLen) -OR [String]::IsNullOrEmpty($ReplaceLen)) {
    Write-Host "Input hex values are not even or empty" -ForegroundColor 'DarkRed'
    RETURN
  } ELSEIF (
    $FindLen -ne $ReplaceLen) {
    Write-Host "Input hex values are different lengths" -ForegroundColor 'DarkYellow'
    RETURN
  }

  $FindAsBytes = New-Object System.Collections.ArrayList
  $Find -split '(.{2})' | ? {$_} | % { $FindAsBytes += [Convert]::ToInt64($_,16) }
  $ReplaceAsBytes = New-Object System.Collections.ArrayList
  $Replace -split '(.{2})' | ? {$_} | % { $ReplaceAsBytes += [Convert]::ToInt64($_,16) }
  # ^-- convert to base 10

  Write-Host "Starting Search"
  $FileBytes = [IO.File]::ReadAllBytes($Target)
  FOREACH ($Byte in $FileBytes) {
        $ByteCounter++
        IF ($Byte -eq [INT64]$FindAsBytes[0]) { TRY {
             (1..([INT64]$FindAsBytes.Count-1)) | % {
               $Test = ($FileBytes[[INT64]$ByteCounter-1+[INT64]$_] -eq $FindAsBytes[$_])
               IF ($Test -ne 'True') {
                 THROW
               }
  }
    Write-Host "Found at Byte:" $ByteCounter -ForegroundColor 'Green'
    (0..($ReplaceAsBytes.Count-1)) | % {
      $FileBytes[[INT64]$ByteCounter+[INT64]$_-1] = $ReplaceAsBytes[$_]}
  $Found = 'True'
  $BytesReplaces = $BytesReplaces + [INT64]$ReplaceAsBytes.Count
  }
CATCH {}
}

}
  IF ($Found -eq 'True'){
    [IO.File]::WriteAllBytes("$Target-temp", $FileBytes)
    $OriginalName = $Target.Name+'_Original'+'@'+(Get-Date).ToString('yyMMdd-HHmmss')
    Rename-Item -LiteralPath $Target.FullName -NewName $OriginalName
    Rename-Item $Target"-temp" -NewName $Target.Name
    #Preserve Last Modified Time
    $Target.LastWriteTime = $TargetLWT
    $Target.CreationTime = $TargetCT
    Write-Host $BytesReplaces "Bytes Replaced" -ForegroundColor 'Green'
    Write-Host "Original saved as:" $OriginalName
  } ELSE {
      Write-Host "No Matches" -ForegroundColor 'Red'}
    Write-Host "Finished Search`n"
    Remove-Variable -Name * -ErrorAction SilentlyContinue
} # end foreach from line 1

PAUSE

Original: This has been asked before but found no solutions to perform a simple and straight up find hex value and replace hex value on large files, 100MB+. Even better would be any recommendations for a hex editor with command line support for this task.

Possible duplicate of [delete some sequence of bytes in Powershell](https://stackoverflow.com/questions/55295131/delete-some-sequence-of-bytes-in-powershell) or [Methods to hex edit binary files via Powershell](https://stackoverflow.com/questions/20935356/methods-to-hex-edit-binary-files-via-powershell) — Theo, Aug 03 '19 at 09:13
@Fr3nZy, if you do not agree that this is a duplicate, please see [How to Ask](https://stackoverflow.com/help/how-to-ask). — iRon, Aug 03 '19 at 13:25
There's options over here. https://stackoverflow.com/questions/20935356/methods-to-hex-edit-binary-files-via-powershell But you have to be careful how many times the search and replace happens. — js2010, Aug 04 '19 at 13:53
@js2010 Thanks. This helped me work out the 'replace' part of the script. — Fr3nZy, Aug 05 '19 at 06:29

js2010 · Answer 1 · 2019-08-17T20:54:41.287

2

Here's a first crack at it:

(get-content -encoding byte file) -replace '\b10\b',11 -as 'byte[]'

I was checking those other links, but the only answer that does search and replace has some bugs. I voted to reopen. The mklement0 one is close. None of them search and then print the position of the replacement.

Nevermind. Yours is faster and uses less memory.

edited Aug 17 '19 at 20:54

answered Aug 03 '19 at 14:45

js2010

23,033
6
64
66

That's clever. You can speed it up dramatically by adding the `-Raw` switch, which reads the entire file at once and directly returns a `byte[]` array, rather than processing the bytes one by one and collecting the result in a regular `[object[]]` array. Also, use `-as [byte[]]` instead of `-as 'byte[]'`. – mklement0 Aug 03 '19 at 17:26
1

Good start but how would I search and replace multiple consecutive bytes in an array? – Fr3nZy Aug 04 '19 at 05:28
@Fr3nZy: See https://stackoverflow.com/a/57342311/45375 – mklement0 Aug 04 '19 at 23:50
`+1` for the idea. If you take this one step further and convert the input byte array to a single string, I guess you can just simplify this to: `"$bInput" -Replace "\b$bOriginal\b", "$bSubstitute" -Split '\s+' -as [Byte[]]`, see the [example in the duplicate](https://stackoverflow.com/a/57353901/1701026) – iRon Aug 05 '19 at 07:39
Sorry about the quotes. My original csv file name and header names had spaces in them. Thanks for beautifying it. I'm new to PS. Need to work on my indenting. – Fr3nZy Aug 06 '19 at 00:04
1

Revised the scripted and fixed the issue of it skipping the first byte. – Fr3nZy Aug 07 '19 at 19:46
Maybe you should post it over here? https://stackoverflow.com/questions/20935356/methods-to-hex-edit-binary-files-via-powershell – js2010 Aug 07 '19 at 19:56

Use PowerShell to find and replace hex values in binary files

1 Answers1

Linked