How to replace multiple strings in a file using PowerShell

Question

I am writing a script for customising a configuration file. I want to replace multiple instances of strings within this file, and I tried using PowerShell to do the job.

It works fine for a single replace, but doing multiple replaces is very slow because each time it has to parse the whole file again, and this file is very large. The script looks like this:

$original_file = 'path\filename.abc'
$destination_file =  'path\filename.abc.new'
(Get-Content $original_file) | Foreach-Object {
    $_ -replace 'something1', 'something1new'
    } | Set-Content $destination_file

I want something like this, but I don't know how to write it:

$original_file = 'path\filename.abc'
$destination_file =  'path\filename.abc.new'
(Get-Content $original_file) | Foreach-Object {
    $_ -replace 'something1', 'something1aa'
    $_ -replace 'something2', 'something2bb'
    $_ -replace 'something3', 'something3cc'
    $_ -replace 'something4', 'something4dd'
    $_ -replace 'something5', 'something5dsf'
    $_ -replace 'something6', 'something6dfsfds'
    } | Set-Content $destination_file

score 193 · Accepted Answer · edited Jul 13 '15 at 18:48

193

One option is to chain the -replace operations together. The ` at the end of each line escapes the newline, causing PowerShell to continue parsing the expression on the next line:

$original_file = 'path\filename.abc'
$destination_file =  'path\filename.abc.new'
(Get-Content $original_file) | Foreach-Object {
    $_ -replace 'something1', 'something1aa' `
       -replace 'something2', 'something2bb' `
       -replace 'something3', 'something3cc' `
       -replace 'something4', 'something4dd' `
       -replace 'something5', 'something5dsf' `
       -replace 'something6', 'something6dfsfds'
    } | Set-Content $destination_file

Another option would be to assign an intermediate variable:

$x = $_ -replace 'something1', 'something1aa'
$x = $x -replace 'something2', 'something2bb'
...
$x

edited Jul 13 '15 at 18:48

KyleMit

30,350
66
462
664

answered Aug 04 '10 at 07:10

dahlbyk

75,175
8
100
122

Can $original_file == $destination_file? As in I am modifying the same file as my source? – cquadrini May 08 '13 at 02:58
Because of the way PowerShell cmdlets stream their input/ouput, I don't believe it would work to write out to the same file in the same pipeline. However, you could do something like `$c = Get-Content $original_file; $c | ... | Set-Content $original_file`. – dahlbyk May 10 '13 at 13:34
Have you problems about file encoding using **Set-Content** that not mantains the original encoding? UTF-8 or ANSI encodings for example. – Kiquenet Jan 27 '15 at 06:57
3

Yeah PowerShell is...unhelpful like that. You have to detect encoding yourself, e.g. https://github.com/dahlbyk/posh-git/blob/869d4c5159797755bc04749db47b166136e59132/install.ps1#L23-L37 – dahlbyk Jan 30 '15 at 03:41
This solution has failed for me. It took a longer time to process and it generated a very big file instead of a file with 30 short lines that is in the existing file. – SouthSun Jan 27 '22 at 22:16

score 31 · Answer 2 · edited Sep 28 '17 at 22:22

31

To get the post by George Howarth working properly with more than one replacement you need to remove the break, assign the output to a variable ($line) and then output the variable:

$lookupTable = @{
    'something1' = 'something1aa'
    'something2' = 'something2bb'
    'something3' = 'something3cc'
    'something4' = 'something4dd'
    'something5' = 'something5dsf'
    'something6' = 'something6dfsfds'
}

$original_file = 'path\filename.abc'
$destination_file =  'path\filename.abc.new'

Get-Content -Path $original_file | ForEach-Object {
    $line = $_

    $lookupTable.GetEnumerator() | ForEach-Object {
        if ($line -match $_.Key)
        {
            $line = $line -replace $_.Key, $_.Value
        }
    }
   $line
} | Set-Content -Path $destination_file

edited Sep 28 '17 at 22:22

Peter Mortensen

30,738
21
105
131

answered Feb 17 '12 at 10:42

TroyBramley

584
4
7

2

This is by far the best approach I've seen so far. The only issue is that I had to read the entire file contents to a variable first in order to use the same source/destination file paths. – angularsen May 19 '15 at 11:25
this looks like the best answer, though I've seen some weird behavior with it matching incorrectly. i.e. in the case where you have a hash table with hex values as strings (0x0, 0x1, 0x100, 0x10000) and 0x10000 will match 0x1. – Sep 21 '16 at 00:50

score 18 · Answer 3 · edited Sep 28 '17 at 22:23

18

With version 3 of PowerShell you can chain the replace calls together:

 (Get-Content $sourceFile) | ForEach-Object {
    $_.replace('something1', 'something1').replace('somethingElse1', 'somethingElse2')
 } | Set-Content $destinationFile

edited Sep 28 '17 at 22:23

Peter Mortensen

30,738
21
105
131

answered Jan 10 '16 at 11:54

Ian Robertson

2,652
3
28
36

Works fine + fluent flavor – hdoghmen Feb 25 '18 at 18:15
As long as you don't need RegEx – nloewen Feb 08 '21 at 21:53
Where does the question mention regex? – Ian Robertson Feb 09 '21 at 07:44

score 13 · Answer 4 · edited Sep 28 '17 at 22:21

Assuming you can only have one 'something1' or 'something2', etc. per line, you can use a lookup table:

$lookupTable = @{
    'something1' = 'something1aa'
    'something2' = 'something2bb'
    'something3' = 'something3cc'
    'something4' = 'something4dd'
    'something5' = 'something5dsf'
    'something6' = 'something6dfsfds'
}

$original_file = 'path\filename.abc'
$destination_file =  'path\filename.abc.new'

Get-Content -Path $original_file | ForEach-Object {
    $line = $_

    $lookupTable.GetEnumerator() | ForEach-Object {
        if ($line -match $_.Key)
        {
            $line -replace $_.Key, $_.Value
            break
        }
    }
} | Set-Content -Path $destination_file

If you can have more than one of those, just remove the break in the if statement.

I see TroyBramley added $line just before the last line to write any line that had no changes in it. Okay. In my case I only changed every line needing replacements. — cliffclof, Jan 23 '20 at 22:32

score 11 · Answer 5 · answered Jan 04 '14 at 04:38

A third option, for a pipelined one-liner is to nest the -replaces:

PS> ("ABC" -replace "B","C") -replace "C","D"
ADD

And:

PS> ("ABC" -replace "C","D") -replace "B","C"
ACD

This preserves execution order, is easy to read, and fits neatly into a pipeline. I prefer to use parentheses for explicit control, self-documentation, etc. It works without them, but how far do you trust that?

-Replace is a Comparison Operator, which accepts an object and returns a presumably modified object. This is why you can stack or nest them as shown above.

Please see:

help about_operators

iRon · Answer 6 · 2021-08-05T10:17:08.287

Just a general reusable solution:

function Replace-String {
    [CmdletBinding()][OutputType([string])] param(
        [Parameter(Mandatory = $True, ValueFromPipeLine = $True)]$InputObject,
        [Parameter(Mandatory = $True, Position = 0)][Array]$Pair,
        [Alias('CaseSensitive')][switch]$MatchCase
    )
    for ($i = 0; $i -lt $Pair.get_Count()) {
        if ($Pair[$i] -is [Array]) {
            $InputObject = $InputObject |Replace-String -MatchCase:$MatchCase $Pair[$i++]
        }
        else {
            $Regex = $Pair[$i++]
            $Substitute = if ($i -lt $Pair.get_Count() -and $Pair[$i] -isnot [Array]) { $Pair[$i++] }
            if ($MatchCase) { $InputObject = $InputObject -cReplace $Regex, $Substitute }
            else            { $InputObject = $InputObject -iReplace $Regex, $Substitute }
        }
    }
    $InputObject
}; Set-Alias Replace Replace-String

Usage:

$lookupTable |Replace 'something1', 'something1aa', 'something2', 'something2bb', 'something3', 'something3cc'

or:

$lookupTable |Replace ('something1', 'something1aa'), ('something2', 'something2bb'), ('something3', 'something3cc')

Example:

'hello world' |Replace ('h','H'), ' ', ('w','W')
HelloWorld

I have created a formal PowerShell request for this: [`#15876` Make -Replace operator support multiple Regex/Substitution pairs](https://github.com/PowerShell/PowerShell/issues/15876) — iRon, Aug 05 '21 at 11:09

How to replace multiple strings in a file using PowerShell

6 Answers6

Linked

Related